Configures the release CI pipeline to generate a binary with
tokio-console support.
This drops the compile-time log filter to TRACE, which may have some
minor overhead. All our log filters in prod are debug or higher, so the
runtime filter will remove the TRACE events and no additional logs will
be emitted because of this change.
This commit adds an optional feature (disabled by default) to the IOx
server binary named "tokio_console". When enabled, it adds support for
the tokio-console to IOx:
https://github.com/tokio-rs/console
Enabling this feature drops the compile-time log filter to TRACE, and
enables the necessary dependencies to support the instrumentation needed
for the console to function.
Unfortunately, this feature uses tonic 0.7 (latest) while we use tonic
0.6, so we wind up with two tonic versions being compiled in when this
feature is enabled.
Changes the trogging::Builder to return a tracing Layer impl when
build() is called, so that it can be composed together with other Layer
impls rather than forcing complete control of the tracing pipeline.
Callers of the Builder::install_global() (basically everything) are
unaffected by this change.
This commit unifies RUSTFLAGS config in the CircleCI config.
Previously in the CircleCI config file for the "build_release" job, we
specify an env var (RUSTFLAGS) in the env config section, but then
override / use a different value when passing flags to the docker build
steps.
"end-user -> querier" and "querier -> ingester" should use a single
Flight client implementation. The difference is just the request and
response metadata.
This changes our default Flight client to use protobuf instead of JSON
for the ticket format.
In parquet files written after compaction, use the catalog sort key but
filter it to only those columns that appear in the merged schema.
Panic if there are any columns in the merged schema's primary key that
aren't in the catalog sort key; that shouldn't happen.
Because we decided a panic was ok to do if the catalog doesn't have a
sort key for the partition, move the panic earlier to catch it before
doing other work.
* feat: Add basic Querier <--> Ingester "Service Configuration"
* docs: update comments in test
* refactor: cleanup tests a little
* refactor: make trait more consistent
* docs: improve comments in IngesterPartition
* fix: not to add IOxReadFilterNode for no data of non-duplicated chunks if there is already scan node for overlapped/duplicated chunks
* refactor: address review comments
* chore: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Removes the old stream_in_sequenced_entries() write buffer handler,
replacing it with the SequencedStreamHandler introduced in #4203.
This change will affect the metrics emitted by an ingester as outlined
in #4243.
Removes the Sync bound SequencedStreamHandler input stream type, as the
BoxStream returned by the WriteBufferStreamHandler is not Sync.
This change means the SequencedStreamHandler is not Sync either, but is
still Send and therefore can be moved into tokio tasks.
This commit adds an adaptor (IngestSinkAdaptor) that provides a DmlSink
implementation for the existing write path (IngesterData). With this,
the existing write path becomes compatible with the new
op stream handler (SequencedStreamHandler).
Dom Dwyer
kodiakhq[bot]
Andrew Lamb
dependabot[bot]
Marco Neumann
Carol (Nichols || Goulding)
Carol (Nichols || Goulding)
Nga Tran