Commit Graph

14 Commits (377e10815230c604f84020696cc05e8bf890af9a)

Author SHA1 Message Date
Dom Dwyer 928a4d163e
build: remove unused dependencies from crates
This commit fixes loads of crates (47!) had unused dependencies, or
mis-configured dependencies (test deps as normal deps).

I added the "unused_crate_dependencies" to all crates to help prevent
this mess from growing again!

    https://doc.rust-lang.org/beta/nightly-rustc/rustc_lint_defs/builtin/static.UNUSED_CRATE_DEPENDENCIES.html

This has the minor downside of false-positives when specifying
dev-dependencies for test/bench binaries - these are files in /test or
/benches (not normal tests). This commit includes a workaround,
importing them in lib.rs (gated by a feature flag). I think the
trade-off of better dependency management is worth it!
2023-05-23 14:55:43 +02:00
kodiakhq[bot] b9bcaf1aa0
Merge branch 'main' into savage/wal-regenerate-lp-cli-command 2023-05-22 16:18:44 +00:00
Andrew Lamb 6344fe8c3f
chore: Add rationale for `clippy::future_not_send` (#7822)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-18 16:58:56 +00:00
Fraser Savage fd5d5e0758
fix(wal): Assert WriteOpDecoder handles tail-corrupt WAL files correctly
WAL read errors encountered by the new WAL WriteOpDecoder were being
discarded and presented as a "happy path" end of file to callers due
to a bug in handling a nested result type. This moves the test for
decoding a tail-corrupt WAL into the crate itself and asserts the
error is reported correctly.
2023-05-18 17:18:19 +01:00
Fraser Savage 21d6460453
refactor(wal_inspect): Put the ref inside the option 2023-05-17 11:37:49 +01:00
Fraser Savage a56eb6efa6
refactor(wal_inspect): Take generic iterator rather than map as table_batch parameter
This allows for callers to use whatever collection implementation they
need, so we can ordered iterate in tests.
2023-05-17 11:31:05 +01:00
Fraser Savage d530b728d4
feat(wal_inspect): Make table_name_index optional for line proto writing
This will allow for decoding a WAL into line protocol without connecting
to a catalog service (or having an equivalent name lookup mechanism).
2023-05-17 10:48:14 +01:00
Fraser Savage e7a0ed4b24
refactor(wal_inspect): Make namespaced table batch writing a trait
This change allows callers to treat LineProtoWriter instances with
different write sink generators the same using generic parameters.
2023-05-17 10:48:13 +01:00
Fraser Savage cfb6abf05b
fix(wal_inspect): Make LineProtoWriter flush best effort 2023-05-16 12:26:59 +01:00
Fraser Savage a4a22b2732
refactor(wal): Tidy up WriteOpEntryDecoder next() body
Refactor out op comparison for `wal` decode tests to be
more general.
2023-05-16 12:06:54 +01:00
Fraser Savage fcd80060be
feat(wal): Make `wal` WriteOpEntryDecoder an interator
Also, implement drop on `wal_inspect`'s LineProtoWriter and
bubble up flush to the caller.
2023-05-15 20:46:56 +01:00
Fraser Savage 6cdc95e49d
refactor(wal): Use a separate DecodeError type for WriteOpEntryDecoder
Having a ginormous error enum returned for this method means that
the catch-all behaviour gets leaked into the error naming and
semantics of callers. The decoder is a new type and could benefit from
not adding to the existing error enum.
2023-05-04 12:36:23 +01:00
Fraser Savage b2e5ea2266
refactor(wal): Add test & docs for WriteOpEntryDecoder
Adds some documentation for the WriteOpEntryDecoder and
a unit test that asserts it skips over non write entries
and can continue to be consumed from.
2023-05-03 11:51:00 +01:00
Fraser Savage f6dea224e8
feat(wal): Add wal_inspect crate & a write op entry decoder
This adds a new crate with a type capable of converting
decoded WAL Write Op entries to line protocol and writing
the result to a namespaced destination. The wal crate
now exports a type which reads the sequenced wal ops and
decodes them as namespaced table batch writes.
2023-05-03 11:50:59 +01:00