Implements a map of K -> Arc<V> with exactly-once initialisation
semantics.
This map can be used to ensure a given key maps to singleton instances
of V; exactly what all the nodes in the ingester "buffer tree" of shard
-> namespace -> table -> partition require.
This impl contains unused funcs (silenced with an allow(dead_code)) due
to it being picked from a future branch.
The DataFusionToExprError::UnsupportedOperants variant contains twice as
many Expr to the second biggest variant. This is of a sufficient size
difference to cause a lint warning.
* fix: Parse regular expressions starting with possible escape sequence
This was failing because the previous combinator, `is_not`, would return
an error if it consumed no input when identifying one of the characters
in its set. This case would then prevent the remainder of the
`regex_literal` parser from identifying and ignoring sequences like "\w"
* fix: Parse microsecond duration literals with correct unit suffix
* fix: Parse a var ref as a 3-part, segmented identifier
Closes#6033
* chore: Address lint warnings
* chore: Additional test cases per feedback
* feat: Add object_store handler to querier
* test: end to end test for get-table from querier
* fix: doc links
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
This commit is part of a two-part change in order to add the table &
namespace IDs to the write buffer wire format. This commit forms the
first half; changing the producer to send the IDs.
In this commit the new ID values are never read on the consumer side,
ensuring there is no consumer dependency on them. This ensures they
remain operational during a rollout, where the consumer may be updated
to the latest code dependent on the IDs before the producer is updated
to send them. This also ensures we have a window of time where where the
consumers can be rolled back after being updated, and still handle
replaying messages in Kafka.
Changes the DML handler transformers to pass through the TableId once it
has been resolved during schema validation.
This value is collated by shard, and then unused. This collated TableId
map will be used in a follow-up PR.
* test: check server exit status on `TestServer` drop
* fix: handle recursing limit in querier<>ingester comm
Fixes#5974.
* docs: improve
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
* test: simplify
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
* refactor: simplify `QueryChunk` data access
We have only two types for chunks (now that the RUB is gone):
1. In-memory RecordBatches
2. Parquet files
Loads of logic is duplicated in the different `read_filter`
implementations. Also `read_filter` hides a solid amount of logic from
DataFusion, which will prevent certain (future) optimizations. To enable #5897
and to simplify the interface, let the chunks return the data (batches
or metadata for parquet files) directly and let `iox_query` perform the
actual heavy-lifting.
* docs: improve
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
* docs: improve
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
With #5963 merged, all chunks now provide a summary (even though it may
not contain data for all columns). So let's make it mandatory, which
also removes a few 🙈-style `.except(...)` calls.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Fixes#6001.
The generator can create multiple agents that all write in parallel to
the same file, which results in garbage.
Share the same File instance with a Mutex around it and lock the file
until you've written one whole line.
kodiakhq[bot]
YIXIAO SHI
Dom
Andrew Lamb
Stuart Carnie
Marco Neumann
Nga Tran
Rowan Hamilton
Cannon Palms
dependabot[bot]
Carol (Nichols || Goulding)