- fuse two RUN commands into one, they're not cached anyways
- remove pointless `RUN ls ...`
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* feat: Specialises test output formatting for each language
* Also fixes an error uncovered in the `write_columnar` when tag
columns are `NULL`
Closes#7145
* chore: Run cargo hakari tasks
* chore: Add sorted output until #7513 is addressed
* chore: clippy 📋
* feat: Add `options` to `write_columnar`
* Added ability to configure border rendering, including removing
borders. This helps avoid variable width issues with EXPLAIN output,
which tends to vary and cause flaky test failures.
* chore: rustfmt 🧹
* chore: update expected output
* chore: clarify what "this" is
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
* refactor: move logic for knowing how much to buffer into GapFiller
* chore: clippy
* chore: add some clarifying comments
* refactor: clean up relationships between gap filling types
* refactor: remove use of RefCell from BufferedInput
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
When pre-warming the catalog cache before the ingester responses have
returned, we don't have any ingester parquet file counts. This is
different than asking the ingesters for the parquet file counts and not
getting any. So keep the Option to be able to treat "not present"
differently from "present but empty".
* feat: support `database`, `bucket`, and `bucket-name` as grpc header names
* chore: lint
* chore: update doc to accept `database`, `bucket`, and `bucket-name` as parameter names
* chore: update doc to only show `database` as the parameter name
* refactor: consolidate header names into a const vec and update comments on database
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
This was "internal". The mapping works like this: we take the
`DataFusionError` and call `find_root` which should traverse the
`External(...)` chain (even through Arrow) to find the last error that
is not within the Arrow/DataFusion land. This is then mapped by us.
`DataFusionError::External(...)` is no further inspected and mapped
straight to "internal". I think this if fine because in the end we're
mostly dealing w/ DataFusion stuff anyways.
I've slightly changed the error mapping in the planner to emit
`DataFusionError::Plan(...)` instead which we map to "invalid argument".
I think this is way better for the user.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Adds a single-line & batched write request benchmarks to ingester2.
When using the WAL, a floor of 10ms is added to all writes (the WAL
linger time, configurable). With the WAL disabled in code, benchmarked
writes complete in less than a millisecond:
single row/write
time: [1.9138 µs 1.9168 µs 1.9197 µs]
thrpt: [520.92 Kelem/s 521.70 Kelem/s 522.51 Kelem/s]
batched/write/1000
time: [129.15 µs 129.97 µs 131.28 µs]
thrpt: [7.6173 Melem/s 7.6941 Melem/s 7.7429 Melem/s]
Note these benchmarks exclude network I/O, and measure single-threaded,
synchronous write client performance.
Moves the ingester2 TestContext & builder to its own crate for reuse
between integration tests & benchmarks (and more?!)
This allows us to DRY & reuse the test code with it's nice API for
benchmarks too, without forcing it all to be in the "prod" build of
ingester2 (and the dependencies, and their dependencies, etc).
This also means this code can be built and cached instead of being
rebuilt all the time during normal development.
* chore: increasing concurrency a little more
This raises the threshold for single threading compactions to 100 column partitions. With the non-linear scaling, 70 column partitions would take 49% of the concurrency limit (allowing only 2 of such sized partitions to compact concurrently). Anything over 70 can only compact with something smaller than itself.
I'm gradually walking these up, partly to avoid causing OOMs in prod, and partly because I want to get a feel for how reactive the average concurrency is to these changes.
* chore: fix comment typo
kodiakhq[bot]
Dom Dwyer
Chunchun Ye
Marco Neumann
Andrew Lamb
dependabot[bot]
Stuart Carnie
Christopher M. Wolff
Carol (Nichols || Goulding)
Joe-Blount
Fraser Savage