0a2cb36ddf
fix: workspace-hack ( #3593 )
2022-02-01 12:32:58 +00:00
22778a3a80
chore: upgrade rskafka and parking_lot ( #3592 )
2022-02-01 11:50:42 +00:00
175732c3ca
feat: basic sqlparser -> RPCNode
2022-02-01 10:26:03 +00:00
b326b62b44
feat: buffer writes when writing to RSKafka ( #3520 )
2022-02-01 10:07:52 +00:00
8bef2c105c
Merge branch 'main' into cn/persist
2022-01-31 18:50:45 +00:00
7b96a37165
chore: Update datafusion ( #3586 )
...
* chore: update DataFusion to f849968057ddddccc9aa19915ef3ea56bf14d80d
* fix: reduce overhead of creating physical expressions
* chore: use MemTrackingMetrics
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-31 18:15:28 +00:00
bf89162fa5
refactor: Move IoxMetadata to parquet_file
2022-01-31 10:36:33 -05:00
dd9620da0c
feat: Create a new proto definition for the new design's IoxMetadata
2022-01-31 10:36:32 -05:00
5e0e0d8aa7
feat: Write parquet to object storage in a similar way as parquet_file::Storage
2022-01-31 10:36:32 -05:00
c633c9bc5c
feat: Wire object store into ingester persistence
2022-01-31 10:36:30 -05:00
4101d16f71
chore: feature flag consistency ( #3574 )
...
* chore: feature flag consistency
* chore: add aarch64-apple-darwin to hakari
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-28 16:38:59 +00:00
8735ede74f
feat: IoxMetadata for parquet file ( #3547 )
...
* feat: IoxMetadata for parquet file
* fix: typos
* refactor: address review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-28 14:41:59 +00:00
1669acc0c2
build: update tokio ( #3566 )
...
Release notes:
https://github.com/tokio-rs/tokio/releases/tag/tokio-1.16.0
2022-01-28 10:36:12 +00:00
b486258dfb
chore: run cargo update ( #3559 )
...
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 21:05:31 +00:00
9201023ea4
feat: schema validation for MutableBuffer instances ( #3554 )
...
* refactor: Debug bounds on Catalog trait
* feat: validate MutableBatch schema
Changes the schema validation code to validate MutableBatch instances
(coming from a pre-parsed LP write, and non-LP-based writes) instead of
parsed LP lines.
* refactor: Send bound on boxed errors
* refactor: clippy
Allow assert_eq!(bool, bool) for readability.
* refactor: no PartialEq<MB Column> for ColumnSchema
Remove the PartialEq<mutable_buffer::Column> for ColumnSchema - it's
definitely more readable as a method call.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 20:55:18 +00:00
4a96e52290
feat: router2 sharder benchmarking ( #3558 )
...
* feat: benchmarking the router2 sharder
* chore: added throughput to sharder benchmarks; vary num buckets
2022-01-27 18:09:16 +00:00
2062267d0f
chore: Update hashbrown ( #3551 )
...
* chore: Update hashbrown
* fix: hakari
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 15:34:10 +00:00
ce568ab447
build(iox_catalog): remove unused dependencies ( #3552 )
...
* build: don't pull in all of tokio
We already specify the tokio features we need so "full" (all features)
is not necessary.
* build: remove chrono dependency
Appears unused.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 15:14:42 +00:00
21c1824a7a
refactor: remove table_names from Predicate ( #3545 )
...
* refactor: remove table_names from Predicate
* chore: fix benchmarks
* chore: review feedback
Co-authored-by: Edd Robinson <me@edd.io>
* chore: review feedback
* chore: replace Default::default with InfluxRpcPredicate::default()
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 14:44:49 +00:00
5488c257d1
chore: Update datafusion, upgrade to arrow/parqet/arrow-flight 8.0.0 ( #3517 )
...
* chore: Update datafusion
* chore: update to arrow 8
* fix: update to use new DataFusion APIs
* fix: update case for sortedness
* fix: cargo hakari
2022-01-27 13:33:27 +00:00
7261571abf
fix: Revert "chore: temporarily hack around datafusion tempfiles" ( #3525 )
...
This reverts commit ae5763c1cb6bb4a98ffe0779a3a35f6daaf10971.
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 12:08:49 +00:00
bc44d33108
feat: Implement a snapshot method on DataBuffer ( #3518 )
...
* feat: Implement a snapshot method on DataBuffer
Fixes #3510 .
* test: Add a test snapshotting batches with different but compatible schemas
* fix: Simplify min/max sequencer number collection
The first batch should always have the min sequencer number. The last
batch should always have the max sequencer number. The min should always
be less than (or equal to, in case there's only one batch) the max.
2022-01-26 15:22:51 +00:00
2928254c0f
fix: test logging ( #3536 )
...
- Use a more standard way to setup the tracing subsystem (as described
in tracing-subscriber docs)
- Also capture content from `log` crate
- Play nice w/ Rust's libtest message capture
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-26 10:28:51 +00:00
b846ead320
feat(router2): shard writes/deletes into write buffer ( #3499 )
...
* feat: Sequencer wrapper
This type wraps an underlying WriteBufferWriter implementation, tagging
it with a sequencer ID it should use when enqueuing operations to the
buffer.
* feat: mock sharder
Implements a mock Sharder impl that returns pre-configured responses to
shard(), and captures the input to the call.
* feat: sharded write buffer
Implements sharding of ops into an underlying WriteBuffer.
Writes are sharded by some abstract Sharder impl, collated per shard to
maximise the size of each op (and therefore compression efficiency),
converted into a DML operation and then enqueued in parallel to the
underlying WriteBuffer implementation.
Deletes are modelled as being mapped to a single write buffer shard,
which is the case while we support sharding based on the table &
namespace only. Deletes will be extended to support (potentially)
multiple shards when column overrides are implemented.
* refactor: runtime write buffers
Switch from using static dispatch, to using a runtime specified
WriteBufferWriting implementation.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-25 15:19:48 +00:00
51a0f8a56a
chore: temporarily hack around datafusion tempfiles ( #3524 )
2022-01-25 12:30:29 +00:00
f9c1e80a7f
chore: update thread_local
...
chore: update thread_local
2022-01-24 13:37:52 -05:00
797ba459b9
chore: merge main to branch
2022-01-24 12:06:23 -05:00
5f98a07b7f
chore: add Corgo.lock
2022-01-24 12:03:02 -05:00
bb893510a0
feat: Add scaffolding for ingester server
...
* Adds a new ingester command to start an ingester server
* Moves previous ingester server over to handler
* Skeleton for gRPC and HTTP handlers
2022-01-21 18:02:19 -05:00
9615feacb3
fix(InfluxQL): Support RegEx with escape sequences not supported by Rust regex ( #3502 )
...
* fix(InfluxQL): Translate unsupported meta characters
* fix: remove debugging
* fix: clippy sacrifice
* docs: Add additional background and rationale for rewriting
* fix: doc link
2022-01-21 14:40:10 +00:00
5b22abf6a2
Merge branch 'main' into crepererum/wb_rskafka
2022-01-20 16:48:10 +00:00
9b6e626626
chore: Update datafusion (and get fix for influxql test failure) ( #3484 )
...
* test: add tests for comparing dictionary arrays
* chore: update datafusion deps
* refactor: Update code for DataFusion API changes
* fix: update test
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-20 14:01:47 +00:00
76dd62a6c2
feat: RSKafka-driven write buffer
2022-01-20 12:36:10 +01:00
36d50d083f
feat: sharder trait & impl
...
This commit defines the Sharder trait that should allow us to implement
multiple sharding strategies over a defined set of input types (such as
a MutableBatch for writes, DeletePredicate for deletes, etc).
This commit also includes a jump hash implementation that consistently
shards (table name, namespace) tuples to a given shard for all input
types.
2022-01-20 11:10:37 +00:00
d825dab8e2
fix: hakari workspace hack
2022-01-19 14:48:00 -05:00
41038721e1
feat: Add parquet file records to iox_catalog
...
* Adds ParquetFile and scaffolding to IOx catalog
* Changed the file_location in parquet_file to object_store_id which is a uuid
2022-01-19 14:14:54 -05:00
edb97f51cf
refactor: add persisting struct
2022-01-19 12:36:18 -05:00
8a17e1c132
refactor: address review comments
2022-01-19 11:20:20 -05:00
e8294d21ec
fix: add .lock
2022-01-18 22:02:27 -05:00
b57f027e35
refactor: address review comments
2022-01-18 20:57:13 -05:00
367a9fb812
fix: add workspace-hack
2022-01-18 18:10:42 -05:00
125285ae9a
feat: commit in order to pull and merge new commit from main
2022-01-18 16:11:25 -05:00
23290fd2ff
fix: new data structures suggested by reviewers
2022-01-18 14:04:07 -05:00
ef336b4659
feat: add ingester crate and a few basic data structures for its data lifecycle
2022-01-17 15:38:03 -05:00
02d45bbae0
Merge branch 'main' into dom/delete-handler
2022-01-17 15:26:33 +00:00
1843476651
chore: Update datafusion deps ( #3471 )
...
* chore: Update datafusion
* refactor: Update to use new Exec plan APIs
* fix: error message
* fix: fixup last bit
* fix: clippy
* fix: doclink
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-17 15:26:19 +00:00
d4d4b4c087
fix: workspace hack after adding iox_catalog crate
2022-01-17 10:04:24 -05:00
aa6f118487
feat: iox_catalog sequencers ( #3465 )
...
* refactor: ensure sequencers are unique
Adds a unique constraint to ensure only one sequencer record exists for
each Kafka (topic, partition).
* test: use DSN from env for integration tests
Removes the hard-coded DSN, instead sourcing it from the DATABASE_URL
environment variable.
* docs: integration testing for iox_catalog
Documents the required steps in order to run the Postgres integration
tests for the iox_catalog crate.
* feat(iox_catalog): create & list sequencers
Adds support for interacting with the "sequencer" table.
* chore: update lockfile
Running cargo in iox_catalog generates a lockfile diff.
2022-01-17 10:00:31 -05:00
6f2e10cab6
feat(router2): implement delete API handler
...
Adds support for the HTTP v2 delete API endpoint.
2022-01-17 14:57:36 +00:00
885c831aff
refactor: avoid constructing DmlOperation
...
Instead of converting the set of MutableBatches into a DmlOperation to
shard into more DmlOperation instances, the sharder can operate directly
on the MutableBatches.
2022-01-17 11:57:32 +00:00
40a290f6f7
feat: router2 HTTP handlers
...
Implements the HTTP v2 write API endpoint for router2.
2022-01-17 11:57:28 +00:00
80b12d417c
feat: abstract DML handler
...
Defines the DmlHandler trait responsible for processing a request in
some abstract way, decoupling the HTTP/gRPC request handlers from the
underlying routing logic.
2022-01-17 11:56:04 +00:00
c399e676ca
chore: upgrade clap to v3
2022-01-17 12:12:46 +01:00
89db894df4
fix: serde_json `arbitrary_precision` ( #3458 ) ( #3469 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-17 09:50:54 +00:00
dd23056efd
chore: update datafusion, arrow, prost, tonic, pbjson, etc ( #3455 )
...
* chore: update datafusion, arrow, prost, tonic, etc
* fix: update pprof as well
* chore: update hakari
* fix: update pbjson
* chore: update heappy
* fix: hakari
* fix: workaround https://github.com/influxdata/influxdb_iox/issues/3458
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-13 17:07:15 +00:00
b9bee7f735
build: update workspace-hack
2022-01-12 15:09:06 +00:00
a8cb8755de
feat: new router2 crate
...
This commit adds an almost-empty router2 crate containing enough of a
skeleton to plumb into the IOx CLI/server runner.
2022-01-12 14:43:10 +00:00
530240cb5a
chore: Update datafusion dependency ( #3443 )
2022-01-11 20:08:25 +00:00
f3f6f335a9
chore: upgrade to snafu 0.7 ( #3440 )
2022-01-11 19:22:36 +00:00
37bb7f2120
chore: `cargo update`
...
dependabot currently doesn't work due to
https://github.com/dependabot/dependabot-core/issues/4574
Excluded `quote` due to
https://github.com/dtolnay/quote/issues/204
2022-01-11 14:57:51 +01:00
f9174c483b
refactor: Extract server::db into its own crate ( #3417 )
...
* refactor: Extract JobRegistry from the server crate
Both the server crate and a db crate that I'm about to extract depend on
JobRegistry, so to avoid making circular dependencies, extract the
JobRegistry to its own crate.
* refactor: Move db out of server into its own crate
Fixes #2821 .
2021-12-23 22:01:17 +00:00
527885f7f8
chore: Update datafusion ( #3413 )
...
* chore: Update datafusion and update code to handle timezone aware timestamps
* fix: cargo hakari
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-23 14:52:12 +00:00
ccba68fe3e
chore: Update datafusion and sqlparser, and arrow ( #3385 )
...
* chore: Update datafusion and sqlparser, and arrow
* fix: cargo hakari
* test: fix metadata size
* fix: update some metadata sizes
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-16 12:13:40 +00:00
b0c6713bd7
fix: Don't swallow exit code when hakari test fails
2021-12-14 14:49:14 +01:00
01e86a031e
feat: add regex to bucket writers assignment in data generator
...
This adds the ability to specify a regex to match against database names when specifying what agents should write to which buckets in the data generator.
A default has also been added for ratio so that it doesn't need to be specified if only a single database writer is defined.
2021-12-09 13:39:33 -05:00
876a9af35a
fix: limit number of S3 connections
...
Otherwise the whole thing blows up when starting a server that has many
DBs registerd, because we potentially create 1 connection per DB (e.g.
to read out the preserved catalog).
Fixes #3336 .
2021-12-08 19:06:02 +01:00
218042784f
feat: add `system.queries `system table ( #3328 )
...
* feat: Add query log and system table
* docs: Add docstrings for test normalization
2021-12-08 16:26:24 +00:00
1156a81567
feat: Add pbjson serialization for storage rpc ( #3324 )
...
* feat: Add pbjson serialization for storage rpc
* chore: update pbjson-build to 0.1.1
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
2021-12-07 16:39:16 +00:00
7499eac067
fix: Disable uuid serde feature; we're not actually serializing any UUIDs
...
Connects to #3117 .
2021-12-06 09:37:31 -05:00
a23695d470
fix: Remove unused generated_types dependency in iox_data_generator
2021-12-06 09:37:16 -05:00
96afb9ffe9
fix: Remove unused tokio dev-dependency in mutable_buffers
2021-12-06 09:37:16 -05:00
e8d9e35321
fix: influxdb_iox tests need the json feature of reqwest
...
I think this didn't fail before because influxdb_iox used to use
influxdb2_client, which turned on the json feature. Then the workspace
hack crate also turned on the json feature, so when influxdb_iox stopped
using influxdb2_client, it *still* worked. But excluding the object
store crates from hakari analysis that also turned on the json feature
meant nothing turned it on anymore, so now influxdb_iox needs to turn it
on itself.
2021-12-06 09:37:15 -05:00
76909b07e4
fix: Exclude optional object-store implementation crates from cargo-hakari
...
Aha! cargo-hakari turns on all features, but usually in development all
the specific object storage implementation crates are off.
Excluding them in the hakari config reduces the workspace crate a bit.
Connects to #3117 .
2021-12-06 09:37:15 -05:00
1b0a2eb7c7
fix: Specify nom features in the same way cexpr does
...
This removes nom from the workspace-hack crate. I'm not sure what's
going on here as cexpr is setting default-features false but then
explicitly setting the features to be the same as nom's default
features, and cargo-hakari seems to be resolving that differently than
using nom's default features. I think this might be a bug in
cargo-hakari, going to investigate.
Connects to #3117 .
2021-12-06 09:37:14 -05:00
6b940166fe
fix: Remove a deprecated memchr feature from the workspace crate
...
This was made possible by upgrading nom, which was using the deprecated
feature name. This will save us from pain in the future when the
deprecated feature is removed.
Unfortunately, this adds nom to the workspace hack crate, but we'll see
if it needs to stay there :)
Connects to #3117 .
2021-12-06 09:37:14 -05:00
e4cc04612b
chore: Update bindgen, nom, cexpr, and related crates
...
Connects to #3117 .
This gets rid of some duplicate versions of crates in our dependency
tree.
2021-12-06 09:37:14 -05:00
bd0a2b36e3
fix: Remove futures-util proc-macro features and update to futures-* 0.3.18
...
The `futures-util` crate's `proc-macro-hack` and `proc-macro-nested`
features used to be part of `futures-util`'s default feature sets, so
the workspace-hack crate listed them.
`futures-util` 0.3.18 removed those features entirely.
Cargo won't update to versions of crates that don't have features that
have been specified, so the workspace-hack crate was blocking Cargo from
updating `futures-util` and a few other crates to 0.3.18.
Cargo's docs recommend against removing features in semver-compatible
releases, but nothing really prevents crate authors from doing this.
Connects to #3117 .
2021-12-06 09:37:14 -05:00
ddc1504135
fix: Exclude influxdb2_client from the workspace hack crate
...
This enables the removal of the futures and futures-io crates.
Connects to #3117 .
2021-12-06 09:37:14 -05:00
8ad2b5e9fd
fix: In end-to-end tests, use influxdb_iox_client instead of influxdb2_client
2021-12-06 09:37:14 -05:00
cf6286364e
fix: Exclude criterion from hakari analysis
...
Connects to #3117 .
Criterion builds clap without default features, which causes clap to be
listed in the workspace hack crate with its default features. In
general, actually building criterion will be rare (and we should move
all benches to separate crates that can be excluded) so it should be
fine to rebuild clap when building benchmarks anyway.
2021-12-06 09:36:50 -05:00
0fda9608d8
fix: Exclude optional pprof crate from workspace-hack
2021-12-06 09:36:32 -05:00
30bc8e5eba
Merge branch 'main' into dependabot/cargo/clap-2.34.0
2021-12-06 10:51:52 +00:00
0b87c1e170
chore(deps): bump clap from 2.33.3 to 2.34.0
...
Bumps [clap](https://github.com/clap-rs/clap ) from 2.33.3 to 2.34.0.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/commits )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-12-06 09:07:47 +00:00
e3aa97f04a
chore(deps): bump tracing-subscriber from 0.3.2 to 0.3.3
...
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing ) from 0.3.2 to 0.3.3.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.2...tracing-subscriber-0.3.3 )
---
updated-dependencies:
- dependency-name: tracing-subscriber
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-12-06 09:07:40 +00:00
472c91d520
chore(deps): bump reqwest from 0.11.6 to 0.11.7
...
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.6 to 0.11.7.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.6...v0.11.7 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-12-06 08:55:14 +00:00
99737ba379
chore(deps): bump libc from 0.2.108 to 0.2.109
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.108 to 0.2.109.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.108...0.2.109 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-12-06 08:42:04 +00:00
dd7d5cd7e3
chore(deps): bump handlebars from 4.1.5 to 4.1.6
...
Bumps [handlebars](https://github.com/sunng87/handlebars-rust ) from 4.1.5 to 4.1.6.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases )
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md )
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.1.5...v4.1.6 )
---
updated-dependencies:
- dependency-name: handlebars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-12-06 08:31:22 +00:00
573ee632b1
chore(deps): bump crc32fast from 1.2.2 to 1.3.0
...
Bumps [crc32fast](https://github.com/srijs/rust-crc32fast ) from 1.2.2 to 1.3.0.
- [Release notes](https://github.com/srijs/rust-crc32fast/releases )
- [Commits](https://github.com/srijs/rust-crc32fast/compare/v1.2.2...v1.3.0 )
---
updated-dependencies:
- dependency-name: crc32fast
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-12-06 01:22:09 +00:00
9e8639f230
chore: Update DataFusion pin ( #3279 )
...
* chore: Update DataFusion pin
* fix: Update for new DF API
* fix: update plan output
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-02 12:42:28 +00:00
c961454dcd
feat: `jaeger-debug-id` from data generator
2021-12-01 14:33:09 +01:00
20da62eb87
chore: Update `arrow`, `arrow-flight`, and `parquet` to 6.3.0 ( #3254 )
...
* chore: Update `arrow`, `arrow-flight`, and `parquet` to 6.3.0
* fix: update metadata size
2021-11-30 12:30:17 +00:00
068b44334d
Merge branch 'main' into crepererum/dml_shard_new_types
2021-11-29 10:33:49 +00:00
d7053cad6b
chore(deps): bump futures-io from 0.3.17 to 0.3.18 ( #3241 )
...
Bumps [futures-io](https://github.com/rust-lang/futures-rs ) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18 )
---
updated-dependencies:
- dependency-name: futures-io
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 10:33:26 +00:00
eba6ebd67a
chore(deps): bump futures-channel from 0.3.17 to 0.3.18
...
Bumps [futures-channel](https://github.com/rust-lang/futures-rs ) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18 )
---
updated-dependencies:
- dependency-name: futures-channel
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:46:01 +00:00
5d784ba866
chore(deps): bump serde_json from 1.0.71 to 1.0.72
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:28:07 +00:00
0ac4b46ebf
chore(deps): bump futures-task from 0.3.17 to 0.3.18
...
Bumps [futures-task](https://github.com/rust-lang/futures-rs ) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18 )
---
updated-dependencies:
- dependency-name: futures-task
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:13:12 +00:00
3bedf6e62a
chore(deps): bump crc32fast from 1.2.1 to 1.2.2
...
Bumps [crc32fast](https://github.com/srijs/rust-crc32fast ) from 1.2.1 to 1.2.2.
- [Release notes](https://github.com/srijs/rust-crc32fast/releases )
- [Commits](https://github.com/srijs/rust-crc32fast/compare/v1.2.1...v1.2.2 )
---
updated-dependencies:
- dependency-name: crc32fast
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:00:01 +00:00
b46d07b8a5
chore(deps): bump predicates from 2.0.3 to 2.1.0
...
Bumps [predicates](https://github.com/assert-rs/predicates-rs ) from 2.0.3 to 2.1.0.
- [Release notes](https://github.com/assert-rs/predicates-rs/releases )
- [Changelog](https://github.com/assert-rs/predicates-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/predicates-rs/compare/v2.0.3...v2.1.0 )
---
updated-dependencies:
- dependency-name: predicates
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 08:40:45 +00:00
b9d804c236
chore(deps): bump futures-core from 0.3.17 to 0.3.18
...
Bumps [futures-core](https://github.com/rust-lang/futures-rs ) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18 )
---
updated-dependencies:
- dependency-name: futures-core
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 08:29:32 +00:00
c729fc6a25
chore(deps): bump rdkafka from 0.27.0 to 0.28.0
...
Bumps [rdkafka](https://github.com/fede1024/rust-rdkafka ) from 0.27.0 to 0.28.0.
- [Release notes](https://github.com/fede1024/rust-rdkafka/releases )
- [Changelog](https://github.com/fede1024/rust-rdkafka/blob/master/changelog.md )
- [Commits](https://github.com/fede1024/rust-rdkafka/compare/v0.27.0...v0.28.0 )
---
updated-dependencies:
- dependency-name: rdkafka
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 08:18:02 +00:00
c0fe972b20
chore(deps): bump cc from 1.0.71 to 1.0.72
...
Bumps [cc](https://github.com/alexcrichton/cc-rs ) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/alexcrichton/cc-rs/releases )
- [Commits](https://github.com/alexcrichton/cc-rs/compare/1.0.71...1.0.72 )
---
updated-dependencies:
- dependency-name: cc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 01:29:47 +00:00
ad65687cba
refactor: consolidate sharding implementation
...
Move sharding implementation from router to `dml` (where the types can
be consumed w/o cloning the data) and only support the new sharding
configs.
The old sharding configs will be removed soon.
2021-11-26 13:00:36 +01:00
edf7becd20
fix: address review comments
2021-11-24 12:09:52 +01:00
f9110eecad
test: migrate `DistibutedTest` to use proper router
2021-11-23 15:39:53 +01:00
50e9e02ff7
chore: Update datafusion ( #3188 )
...
* chore: Update datafusion
* fix: Update for LogicalPlan changes
* fix: Update explain plan output
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-23 12:23:23 +00:00
d16a7759ca
Merge branch 'main' into cn/workspace-hack
2021-11-22 17:05:31 +00:00
f3c814e091
fix: Exclude crates conductor depends on from workspace hack
2021-11-22 11:55:38 -05:00
a24addc5d7
chore(deps): bump handlebars from 4.1.4 to 4.1.5 ( #3176 )
...
Bumps [handlebars](https://github.com/sunng87/handlebars-rust ) from 4.1.4 to 4.1.5.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases )
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md )
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.1.4...v4.1.5 )
---
updated-dependencies:
- dependency-name: handlebars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-22 09:56:58 +00:00
ca5a437be1
chore(deps): bump hyper from 0.14.14 to 0.14.15
...
Bumps [hyper](https://github.com/hyperium/hyper ) from 0.14.14 to 0.14.15.
- [Release notes](https://github.com/hyperium/hyper/releases )
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.14...v0.14.15 )
---
updated-dependencies:
- dependency-name: hyper
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-22 09:02:53 +00:00
bd974798d4
chore(deps): bump tracing-subscriber from 0.3.1 to 0.3.2
...
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing ) from 0.3.1 to 0.3.2.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.1...tracing-subscriber-0.3.2 )
---
updated-dependencies:
- dependency-name: tracing-subscriber
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-22 08:51:18 +00:00
72305ee446
chore(deps): bump tower from 0.4.10 to 0.4.11
...
Bumps [tower](https://github.com/tower-rs/tower ) from 0.4.10 to 0.4.11.
- [Release notes](https://github.com/tower-rs/tower/releases )
- [Commits](https://github.com/tower-rs/tower/compare/tower-0.4.10...tower-0.4.11 )
---
updated-dependencies:
- dependency-name: tower
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-22 08:40:20 +00:00
b83308183b
chore(deps): bump libc from 0.2.106 to 0.2.108
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.106 to 0.2.108.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.106...0.2.108 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-22 08:29:09 +00:00
5c6deb81db
chore(deps): bump serde_json from 1.0.70 to 1.0.71
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.70 to 1.0.71.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.70...v1.0.71 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-22 01:21:06 +00:00
fa36adc3e3
fix: Update the workspace hack crate that no longer needs cache_loader_async
2021-11-19 14:34:24 -05:00
a6908f1613
fix: Exclude tikv-jemalloc-sys from workspace hack as it's mutually exclusive with heappy
2021-11-19 13:46:17 -05:00
2f452268af
fix: Exclude some bench/test crates from the workspace hack
2021-11-19 13:38:27 -05:00
9fd4a560f5
feat: Results of running cargo hakari manage-deps
2021-11-19 09:21:57 -05:00
d1c6f76a0e
feat: Results of running cargo hakari generate
2021-11-19 09:21:15 -05:00
7783e4a7ff
fix: Make delete/restore aliases for release/claim; remove tombstone
...
Fixes #2680
2021-11-17 11:41:08 -05:00
47acd181c5
chore: Update datafusion + arrow/parquet/arrow-flight 6.2.0 ( #3136 )
...
* chore: Update datafusion and arrow
* chore: update arrow/parquet/arrow-flight to 6.2.0
* refactor: Add table_exists to SchemaProvider impl
* fix: clippy
* fix: clippy 2
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-17 14:04:49 +00:00
da61966858
build: remove proc-macro2 pin ( #3137 )
...
Seems unused, builds without it!
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-17 13:55:39 +00:00
04d8133227
chore: upgrade tokio to 1.14.0 to fix RUSTSEC-2021-0124
2021-11-17 09:44:52 +01:00
e6fdd79a0f
feat: emit Kafka stats as metrics instead of logs
...
This maps a subset of Kafka stats as metrics. The set can -- of course
-- be changed in the future depending on our needs.
Fixes #3100 .
2021-11-16 17:18:41 +01:00
4e71de508e
fix: ensure `ConsistenHasher` is consistent
...
The std `DefaultHasher` is NOT guaranteed to stay the same, so let's
directly use the `SipHasher13` which at the moment (2021-11-15) is used
by the standard lib.
Fixes #3063 .
2021-11-15 17:39:17 +01:00
58f3e2e559
refactor: move delete predicate proto serialization to generated_types ( #3108 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-15 12:02:14 +00:00
3d091208af
refactor: move delete predicate into data_types ( #2731 ) ( #3094 )
...
* refactor: move delete predicate into dml (#2731 )
* refactor: move DeletePredicate to data_types
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-15 10:28:58 +00:00
3485d70409
chore(deps): bump serde_json from 1.0.69 to 1.0.70
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.69 to 1.0.70.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.69...v1.0.70 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-15 08:48:44 +00:00
7fb967d088
chore(deps): bump nom from 7.0.0 to 7.1.0
...
Bumps [nom](https://github.com/Geal/nom ) from 7.0.0 to 7.1.0.
- [Release notes](https://github.com/Geal/nom/releases )
- [Changelog](https://github.com/Geal/nom/blob/main/CHANGELOG.md )
- [Commits](https://github.com/Geal/nom/commits )
---
updated-dependencies:
- dependency-name: nom
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-15 08:34:55 +00:00
d20b7ca390
fix: Remove unused dependencies
...
Thanks, `cargo udeps`!
2021-11-12 16:42:02 -05:00
da7ba511b2
feat: remove legacy per-line partitioning logic ( #3087 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-11 16:50:42 +00:00
6f268f8260
refactor: extract DML types ( #2731 ) ( #3084 )
...
* refactor: extract DML types (#2731 )
* chore: fmt
2021-11-11 12:34:07 +00:00
c80088bf12
feat: HTTP write interface for router
2021-11-10 17:55:46 +01:00
3dda694520
refactor: Extract a macro for matching errors in tests
2021-11-08 11:14:23 -05:00
abbfafcabd
chore: merge main to branch
2021-11-08 09:28:29 -05:00
f6bbf7b617
chore(deps): bump libc from 0.2.106 to 0.2.107
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.106 to 0.2.107.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.106...0.2.107 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-08 11:18:48 +00:00
9ce86f4f8e
chore(deps): bump nom from 7.0.0 to 7.1.0
...
Bumps [nom](https://github.com/Geal/nom ) from 7.0.0 to 7.1.0.
- [Release notes](https://github.com/Geal/nom/releases )
- [Changelog](https://github.com/Geal/nom/blob/main/CHANGELOG.md )
- [Commits](https://github.com/Geal/nom/commits )
---
updated-dependencies:
- dependency-name: nom
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-08 11:06:18 +00:00
e4d943d79c
chore(deps): bump comfy-table from 4.1.1 to 5.0.0
...
Bumps [comfy-table](https://github.com/nukesor/comfy-table ) from 4.1.1 to 5.0.0.
- [Release notes](https://github.com/nukesor/comfy-table/releases )
- [Changelog](https://github.com/Nukesor/comfy-table/blob/master/CHANGELOG.md )
- [Commits](https://github.com/nukesor/comfy-table/compare/v4.1.1...v5.0.0 )
---
updated-dependencies:
- dependency-name: comfy-table
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-08 10:55:57 +00:00
af6694299b
chore(deps): bump handlebars from 4.1.3 to 4.1.4
...
Bumps [handlebars](https://github.com/sunng87/handlebars-rust ) from 4.1.3 to 4.1.4.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases )
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/master/CHANGELOG.md )
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.1.3...v4.1.4 )
---
updated-dependencies:
- dependency-name: handlebars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-08 10:42:46 +00:00
7214fa9d6a
chore(deps): bump serde_json from 1.0.68 to 1.0.69
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.68 to 1.0.69.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.68...v1.0.69 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-08 10:31:33 +00:00
6e8517cb1f
Merge branch 'main' into dependabot/cargo/paste-1.0.6
2021-11-08 10:20:56 +00:00
8724f82c5c
chore(deps): bump paste from 1.0.5 to 1.0.6
...
Bumps [paste](https://github.com/dtolnay/paste ) from 1.0.5 to 1.0.6.
- [Release notes](https://github.com/dtolnay/paste/releases )
- [Commits](https://github.com/dtolnay/paste/compare/1.0.5...1.0.6 )
---
updated-dependencies:
- dependency-name: paste
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-08 10:16:07 +00:00
11f4a1dee8
feat: add connection management for router
2021-11-08 11:11:52 +01:00
60f0deaf1e
feat: remove flatbuffer entry ( #3045 )
2021-11-05 20:19:24 +00:00
88868e7496
feat: remove legacy write service from influxdb_iox_client ( #3043 )
...
* feat: remove legacy write service from influxdb_iox_client
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-05 17:26:18 +00:00
94016324da
Merge branch 'main' into ntran/min_max_string
2021-11-05 15:21:10 +00:00
8dcd2ae9d4
chore: latest datafusion to get the boolean(null) and/or expr folding bug fix
2021-11-05 11:20:16 -04:00
898567e221
feat: migrate server to DbWrite ( #2724 ) ( #3035 )
...
* feat: migrate server to DbWrite (#2724 )
* chore: print perf log output
* fix: don't suppress CI status code
* chore: review feedback
* fix: don't error on empty line protocol write payloads
* fix: test
* fix: test
2021-11-05 11:09:33 +00:00
92e6acaacd
feat: implement sharder for new router
2021-11-05 09:49:33 +01:00
89699cf0de
chore: use latest DataFusion to fix the min/max(dictionary string) bug
2021-11-04 11:04:03 -04:00
bd1679a903
fix: revert to previous Cargo.lock and remove rand seeder
2021-11-04 09:56:58 -04:00
7044b89453
feat: Refactor Data Generator
...
This is a huge commit that refactors the data generator. It removes many of the previous features that didn't quite make sense. The goal of this refactor was to make the data generator capable of representing complex tagsets that have values dependent on each other. It also significantly optimizes things to use far less memory and generate data much faster. Follow on work will update the generation of line protocol to support spaces in tags and their keys, double quotes in strings, and add more examples and documentation.
2021-11-04 09:56:58 -04:00
07ba629e2b
feat: migrate write buffer producer to MutableBatch and pbdata ( #2743 ) ( #3021 )
...
* feat: migrate write buffer producer to MutableBatch and pbdata (#2743 )
* fix: Kafka message content type header
* chore: fix doc
2021-11-04 10:20:40 +00:00
46213a5a28
chore: update dependencies
2021-11-03 17:00:20 -04:00
405ed6db1b
feat: add mutable_batch_tests ( #2724 ) ( #3011 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-03 18:15:35 +00:00
a91e572b43
feat: implement basic router server type
...
This only implements the deployment service and nothing else yet.
2021-11-03 16:34:51 +01:00
08fcd87337
feat: use protobuf encoding in write buffer ( #2724 ) ( #3018 )
2021-11-03 15:19:05 +00:00
e444fa4cb2
feat: pbdata encode ( #2724 ) ( #3009 )
2021-11-02 18:31:53 +00:00
1902c4f8a9
chore: Update DataFusion ( #3012 )
...
* chore: Update DataFusion
* fix: restore Cargo.log
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-02 18:06:21 +00:00
f1a6468e7b
feat: migrate write buffer consumer to use DbWrite ( #2724 ) ( #3003 )
...
* feat: migrate write buffer consumer to use DbWrite (#2724 )
* fix: doc
* chore: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-01 16:38:48 +00:00
bf4212e98d
chore(deps): bump pprof from 0.5.0 to 0.5.1
...
Bumps [pprof](https://github.com/tikv/pprof-rs ) from 0.5.0 to 0.5.1.
- [Release notes](https://github.com/tikv/pprof-rs/releases )
- [Commits](https://github.com/tikv/pprof-rs/commits )
---
updated-dependencies:
- dependency-name: pprof
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 11:48:31 +00:00
1b377f0377
chore(deps): bump tokio-stream from 0.1.7 to 0.1.8
...
Bumps [tokio-stream](https://github.com/tokio-rs/tokio ) from 0.1.7 to 0.1.8.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.7...tokio-stream-0.1.8 )
---
updated-dependencies:
- dependency-name: tokio-stream
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 11:38:59 +00:00
c540b40f05
chore(deps): bump tokio from 1.12.0 to 1.13.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.12.0...tokio-1.13.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 11:21:59 +00:00
9f04877556
Merge branch 'main' into dependabot/cargo/tokio-util-0.6.9
2021-11-01 10:26:00 +00:00
a1f978e280
chore(deps): bump tokio-util from 0.6.8 to 0.6.9
...
Bumps [tokio-util](https://github.com/tokio-rs/tokio ) from 0.6.8 to 0.6.9.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.6.8...tokio-util-0.6.9 )
---
updated-dependencies:
- dependency-name: tokio-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 01:28:41 +00:00
3ab81b7d7b
chore(deps): bump tracing-subscriber from 0.3.0 to 0.3.1
...
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing ) from 0.3.0 to 0.3.1.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.0...tracing-subscriber-0.3.1 )
---
updated-dependencies:
- dependency-name: tracing-subscriber
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 01:28:23 +00:00
6ceab054ab
refactor: move DbWrite to mutable_batch ( #2986 )
...
* refactor: move DbWrite to mutable_batch
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-29 15:13:05 +00:00
fc25e313c6
Merge branch 'main' into cn/iox-obj-sto-refactors
2021-10-29 13:53:26 +00:00
8a2410e161
feat: mutable batch write entry ( #2724 ) ( #2973 )
...
* feat: mutable batch write entry (#2724 )
* chore: lint
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-28 20:15:28 +00:00
95ba84edca
fix: Return UUID when a database is deleted
...
So that it can be immediately used to restore elsewhere.
2021-10-28 13:20:29 -04:00
c465d0af8f
refactor: abstract HTTP inteface for different run modes
...
First iteration of a common interface for the different server run
modes. This splits the existing HTTP server into a part that can be used
by all run modes (e.g. health checks, metrics endpoint, profiling) and a
part that is specific to the the database run mode.
Note that the interface is not final and might require more iterations,
but we have to start somewhere.
2021-10-28 11:16:41 +02:00
3010ff4176
feat: write entry to MutableBatch ( #2972 )
...
* feat: write entry to MutableBatch
* feat: error on empty TableBatch
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-27 14:05:22 +00:00
2833cefc12
test: remove Kafka from end2end tests
...
Kafka is now sufficiently tested via the `write_buffer` crate. The
end2end tests can now use the in-memory mock implementation or -- if
servers can only be controlled via CLI -- the file-based implementation.
2021-10-26 10:09:34 +02:00
9ceb5800c1
fix: adjust code to `tracing-subscriber` 0.3
2021-10-25 17:01:50 +02:00
b7186d0bec
chore(deps): bump tracing-subscriber from 0.2.25 to 0.3.0
...
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing ) from 0.2.25 to 0.3.0.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.25...tracing-subscriber-0.3.0 )
---
updated-dependencies:
- dependency-name: tracing-subscriber
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-25 17:01:50 +02:00
820e0d56bb
feat: support multiple tables in lp -> MutableBatch conversion ( #2959 )
2021-10-25 11:42:29 +00:00
bd707fb7ca
chore(deps): bump backtrace from 0.3.61 to 0.3.62
...
Bumps [backtrace](https://github.com/rust-lang/backtrace-rs ) from 0.3.61 to 0.3.62.
- [Release notes](https://github.com/rust-lang/backtrace-rs/releases )
- [Commits](https://github.com/rust-lang/backtrace-rs/compare/0.3.61...0.3.62 )
---
updated-dependencies:
- dependency-name: backtrace
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-25 07:49:24 +00:00
80d6ec6fe9
chore(deps): bump hyper from 0.14.13 to 0.14.14
...
Bumps [hyper](https://github.com/hyperium/hyper ) from 0.14.13 to 0.14.14.
- [Release notes](https://github.com/hyperium/hyper/releases )
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.13...v0.14.14 )
---
updated-dependencies:
- dependency-name: hyper
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-25 07:39:20 +00:00
2b832fcc5c
chore(deps): bump libc from 0.2.104 to 0.2.105
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.104 to 0.2.105.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.104...0.2.105 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-25 07:28:31 +00:00
fae12cee54
chore(deps): bump tower from 0.4.9 to 0.4.10
...
Bumps [tower](https://github.com/tower-rs/tower ) from 0.4.9 to 0.4.10.
- [Release notes](https://github.com/tower-rs/tower/releases )
- [Commits](https://github.com/tower-rs/tower/compare/tower-0.4.9...tower-0.4.10 )
---
updated-dependencies:
- dependency-name: tower
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-25 07:15:51 +00:00
5bf2389486
chore(deps): bump http-body from 0.4.3 to 0.4.4
...
Bumps [http-body](https://github.com/hyperium/http-body ) from 0.4.3 to 0.4.4.
- [Release notes](https://github.com/hyperium/http-body/releases )
- [Changelog](https://github.com/hyperium/http-body/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/http-body/commits )
---
updated-dependencies:
- dependency-name: http-body
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-25 01:40:25 +00:00
d2b41c5a13
feat: write pbdata to MutableBatch ( #2724 ) ( #2927 )
...
* feat: write pbdata to MutableBatch (#2724 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-21 14:32:35 +00:00
297f995a44
feat: partition MutableBatch rows ( #2926 )
...
* feat: partition MutableBatch rows
* chore: review feedback
* chore: review feedback
* chore: fmt
2021-10-21 12:30:13 +00:00
272e43492a
feat: mutable batch filtering ( #2920 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-21 12:06:26 +00:00
f64e749eb9
feat: write line protocol to MutableBatch ( #2914 )
...
* feat: write line protocol to MutableBatch
* chore: fix lint
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-21 08:12:54 +00:00
9974a5364c
chore(security): Replace prettytable with comfy-table ( #2905 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-20 10:44:36 +00:00
ce0127a1f7
feat: MutableBatch write API ( #2090 ) ( #2724 ) ( #2882 )
...
* feat: MutableBatch write API (#2090 ) (#2724 )
* chore: fix lint
* fix: handle dictionaries with unused mappings
* chore: review feedback
* chore: further review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-20 08:44:14 +00:00
a82dc6f5f0
chore: Update datafusion + arrow ( #2903 )
...
* chore: Update datafusion to latest, arrow to 6.0.0
* fix: Update tests
* fix: bubble internal error
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-19 17:14:08 +00:00
6ec0bd5bab
feat: file-based write write_buffer
...
Closes #2849 .
2021-10-19 15:26:43 +02:00
c466f16803
chore(deps): bump reqwest from 0.11.5 to 0.11.6
...
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.5...v0.11.6 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-19 13:05:58 +00:00
0be89715e2
Merge branch 'main' into dependabot/cargo/rdkafka-0.27.0
2021-10-19 12:55:21 +00:00
9c9a31a402
chore: remove unused `iox_data_generator` => `tracing-futures` dep
2021-10-19 14:46:26 +02:00
93987ff107
chore: remove unused `iox_data_generator` => `packers` dep
2021-10-19 14:46:26 +02:00
7dafb0ff74
chore: remove unused `influxdb_iox` => `synchronized-writer` dep
2021-10-19 14:46:26 +02:00
da85f04b3f
chore: remove unused `schema` => `arrow_util` dep
2021-10-19 14:46:25 +02:00
99c019bf33
chore: remove unused `server_benchmarks` => `packers` dep
2021-10-19 14:45:56 +02:00
a364bdfb5f
chore: remove unused `query_tests` => `chrono` dep
2021-10-19 14:45:56 +02:00
d8f35d8ee9
chore: remove unused `parquet_file` => `chrono` dep
2021-10-19 14:45:56 +02:00
586d6b6f28
chore: remove unused `parquet_catalog` => `chrono` dep
2021-10-19 14:45:56 +02:00
2079fb689e
chore: remove unused `generated_types` => `thiserror` dep
2021-10-19 14:45:56 +02:00
32e18b6436
chore(deps): bump rdkafka from 0.26.0 to 0.27.0
...
Bumps [rdkafka](https://github.com/fede1024/rust-rdkafka ) from 0.26.0 to 0.27.0.
- [Release notes](https://github.com/fede1024/rust-rdkafka/releases )
- [Changelog](https://github.com/fede1024/rust-rdkafka/blob/master/changelog.md )
- [Commits](https://github.com/fede1024/rust-rdkafka/compare/v0.26.0...v0.27.0 )
---
updated-dependencies:
- dependency-name: rdkafka
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-19 11:49:34 +00:00
4cc87fb184
chore: misc updates ( #2886 )
...
* chore: misc updates
* fix: don't downgrade nix
* chore: remove orphaned comment
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-19 11:47:27 +00:00
157b556d4c
feat: make flight responses streaming ( #2876 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-18 15:01:50 +00:00
70555ab33d
chore(deps): bump structopt from 0.3.23 to 0.3.25 ( #2877 )
...
Bumps [structopt](https://github.com/TeXitoi/structopt ) from 0.3.23 to 0.3.25.
- [Release notes](https://github.com/TeXitoi/structopt/releases )
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md )
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.23...v0.3.25 )
---
updated-dependencies:
- dependency-name: structopt
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-10-18 14:49:15 +00:00
c5405fbd5f
chore(deps): bump sqlparser from 0.11.0 to 0.12.0 ( #2867 )
...
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.11.0 to 0.12.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases )
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.11.0...v0.12.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-18 14:11:02 +00:00
691a43789b
chore(deps): bump libc from 0.2.103 to 0.2.104
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.103 to 0.2.104.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.103...0.2.104 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-18 09:16:38 +00:00
bdd6d67e7a
refactor: split out mutable_batch crate ( #2841 )
...
* refactor: split out mutable_batch crate
* refactor: restore chunk module for better diffs
* chore: fmt
* chore: review feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-14 16:46:54 +00:00
28195b9c0c
chore: new `parquet_catalog` crate
2021-10-14 14:34:59 +02:00
61ec559eee
Merge branch 'main' into crepererum/write_buffer_span_ctx
2021-10-14 11:50:07 +00:00
3457257ef3
Merge branch 'main' into crepererum/internal_types_snafu
2021-10-14 11:09:14 +00:00
074ae40382
feat: migrate entry to use bytes::Bytes ( #2842 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-14 10:55:51 +00:00
027483cb83
chore: remove unused `snafu` dep from `internal_types`
2021-10-14 12:54:55 +02:00
5e06519afb
feat: propagate trace information through write buffer
2021-10-14 11:07:41 +02:00
0568452a0c
chore: Update datafusion ( #2838 )
...
* chore: update datafusion version
* refactor: Update to use new datafusion apis
* fix: do not upgrade other packages
2021-10-13 20:51:19 +00:00
8a82f92c5d
refactor: add TimeProvider abstraction ( #2722 ) ( #2815 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-12 21:19:03 +00:00
3dfe400e6b
feat: migrate write path to TimeProvider ( #2722 ) ( #2807 )
2021-10-12 12:09:08 +00:00
0554173684
feat: migrate write buffer to TimeProvider ( #2722 ) ( #2804 )
...
* feat: migrate write buffer to TimeProvider (#2722 )
* chore: review feedback
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: Marco Neumann <marco@crepererum.net>
2021-10-12 10:32:34 +00:00
b39e01f7ba
feat: migrate PersistenceWindows to TimeProvider ( #2722 ) ( #2798 )
2021-10-11 20:40:00 +00:00
49c63d35b1
chore(deps): bump cache_loader_async from 0.1.1 to 0.1.2
...
Bumps [cache_loader_async](https://github.com/ZeroTwo-Bot/cache-loader-async-rs ) from 0.1.1 to 0.1.2.
- [Release notes](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/releases )
- [Changelog](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/commits )
---
updated-dependencies:
- dependency-name: cache_loader_async
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 15:02:12 +00:00
7bd32a4916
feat: add time provider abstraction ( #2722 ) ( #2792 )
...
* feat: add time provider abstraction (#2722 )
* chore: tests
* chore: more tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 15:00:29 +00:00
61190a9d08
chore(deps): bump predicates from 2.0.2 to 2.0.3
...
Bumps [predicates](https://github.com/assert-rs/predicates-rs ) from 2.0.2 to 2.0.3.
- [Release notes](https://github.com/assert-rs/predicates-rs/releases )
- [Changelog](https://github.com/assert-rs/predicates-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/predicates-rs/compare/v2.0.2...v2.0.3 )
---
updated-dependencies:
- dependency-name: predicates
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 14:32:08 +00:00
24ae269b3a
refactor: cancel executor jobs on drop
...
Our executor is not meant as a fire-and-forget system. Instead the
submitter should always poll the result. Dropping the receiver side (aka
the job handle) should cancel the job.
2021-10-11 16:13:04 +02:00
afe34751e7
refactor: split out schema crate ( #2781 )
...
* refactor: split out schema crate
* chore: fix doc
2021-10-11 09:45:08 +00:00
f35a49edd0
refactor: move Sequence to data_types ( #2780 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 09:23:00 +00:00
435a8aec91
chore(deps): bump reqwest from 0.11.4 to 0.11.5 ( #2789 )
...
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.4...v0.11.5 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 09:14:15 +00:00
7cf3cf252a
chore(deps): bump tracing from 0.1.28 to 0.1.29 ( #2788 )
...
Bumps [tracing](https://github.com/tokio-rs/tracing ) from 0.1.28 to 0.1.29.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.28...tracing-0.1.29 )
---
updated-dependencies:
- dependency-name: tracing
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-11 09:05:33 +00:00
42d4ad61e1
chore(deps): bump ahash from 0.7.4 to 0.7.5
...
Bumps [ahash](https://github.com/tkaitchuck/ahash ) from 0.7.4 to 0.7.5.
- [Release notes](https://github.com/tkaitchuck/ahash/releases )
- [Commits](https://github.com/tkaitchuck/ahash/commits )
---
updated-dependencies:
- dependency-name: ahash
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:36:50 +00:00
71a9aa2120
chore(deps): bump ouroboros from 0.12.0 to 0.13.0
...
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/commits )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:24:27 +00:00
3f921d43e6
chore(deps): bump tracing-subscriber from 0.2.24 to 0.2.25
...
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing ) from 0.2.24 to 0.2.25.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.24...tracing-subscriber-0.2.25 )
---
updated-dependencies:
- dependency-name: tracing-subscriber
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:14:11 +00:00
1327735537
chore(deps): bump thiserror from 1.0.29 to 1.0.30
...
Bumps [thiserror](https://github.com/dtolnay/thiserror ) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases )
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.29...1.0.30 )
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:01:21 +00:00
01a44899fd
chore(deps): bump assert_cmd from 2.0.1 to 2.0.2
...
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd ) from 2.0.1 to 2.0.2.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases )
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.1...v2.0.2 )
---
updated-dependencies:
- dependency-name: assert_cmd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 01:45:35 +00:00
c7727f1b5b
chore: Update datafusion + other deps ( #2760 )
...
* chore: Update datafusion and other deps
* fix: fmt
* fix: cleanup workaround
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-07 16:48:11 +00:00
d4bc33b921
refactor: use RWLock instead of TaskTracker for query exec shutdown ( #2761 )
2021-10-07 14:18:40 +00:00
fb9119fa4c
feat: remove serde from data types ( #2652 )
2021-10-07 08:58:06 +00:00
63d74be490
refactor: make `ChunkId` a UUID
2021-10-07 10:23:27 +02:00
39157828b1
feat: remove remaining usages of Instant ( #2722 ) ( #2749 )
...
* feat: remove remaining usages of Instant (#2722 )
* chore: review feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-06 16:44:02 +00:00
ce5b24e65d
refactor: use DateTime<Utc> in PersistenceWindows ( #2722 ) ( #2743 )
...
* refactor: use DateTime<Utc> in PersistenceWindows (#2722 )
* chore: fix benchmark
* chore: fmt
* chore: review feedback
2021-10-06 09:39:32 +00:00
7a6f381e69
chore(deps): bump arrow-flight from 5.4.0 to 5.5.0
...
Bumps [arrow-flight](https://github.com/apache/arrow-rs ) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases )
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md )
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0 )
---
updated-dependencies:
- dependency-name: arrow-flight
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 13:27:26 +00:00
6d9205d87c
chore(deps): bump parquet from 5.4.0 to 5.5.0
...
Bumps [parquet](https://github.com/apache/arrow-rs ) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases )
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md )
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0 )
---
updated-dependencies:
- dependency-name: parquet
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 12:15:11 +01:00
5060bf71fe
Merge branch 'main' into crepererum/in_mem_expr_part1
2021-10-04 08:45:50 +00:00
425ecebf9d
chore(deps): bump ouroboros from 0.11.1 to 0.12.0
...
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.11.1 to 0.12.0.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/commits )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 08:25:32 +00:00
b33e1386e0
chore(deps): bump prometheus from 0.12.0 to 0.13.0
...
Bumps [prometheus](https://github.com/tikv/rust-prometheus ) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/tikv/rust-prometheus/releases )
- [Changelog](https://github.com/tikv/rust-prometheus/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tikv/rust-prometheus/compare/v0.12.0...v0.13.0 )
---
updated-dependencies:
- dependency-name: prometheus
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 08:10:07 +00:00
a0bbbdb197
refactor: use dedicated in-memory `Expr` type during IO
...
Place a dedicated `Expr` type between datafusion and protobuf. While
this type is currently only used during serialization, it will be used
within `Predicate` in a follow-up change to enable de-duplication of
predicates and avoid the double parsing of datafusion expressions (we
are already somewhat parsing them when we check for valid delete
predicates).
This change looks larger than it is. In practice it just separates the
conversion "datafusion => protobuf" into "datafusion => IOx => protobuf"
and "protobuf => datafusion" into "protobuf => IOx => datafusion". So
(apart from the error types) this is functionally the same.
2021-10-04 09:59:14 +02:00
8893b483a3
chore(deps): bump smallvec from 1.6.1 to 1.7.0
...
Bumps [smallvec](https://github.com/servo/rust-smallvec ) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/servo/rust-smallvec/releases )
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.6.1...v1.7.0 )
---
updated-dependencies:
- dependency-name: smallvec
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 07:57:00 +00:00
44d7479db1
chore(deps): bump cloud-storage from 0.10.2 to 0.10.3
...
Bumps [cloud-storage](https://github.com/ThouCheese/cloud-storage-rs ) from 0.10.2 to 0.10.3.
- [Release notes](https://github.com/ThouCheese/cloud-storage-rs/releases )
- [Changelog](https://github.com/ThouCheese/cloud-storage-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/ThouCheese/cloud-storage-rs/commits )
---
updated-dependencies:
- dependency-name: cloud-storage
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 01:43:09 +00:00
154dd4460e
refactor: address review comments
2021-10-01 16:37:22 -04:00
bd22c73b8a
chore: Merge branch 'main' into ntran/delete_endpoint
2021-10-01 13:33:39 -04:00
a1f91207b3
fix: Fix RUSTSEC-2021-0119
...
Update `nix` dependency which includes a fix for RUSTSEC-2021-0119
2021-10-01 09:08:47 +02:00
53a45d89b7
feat: support HTTP Delete Endpoints
2021-09-30 18:09:17 -04:00
edd6c12e93
fix: Add logging when retrying AWS requests
2021-09-29 08:19:32 -04:00
9cf343db08
refactor: Use loop instead of futures_retry
2021-09-29 08:19:32 -04:00
5c78b7d5ae
refactor: Extract an s3_request method that handles retries
2021-09-29 08:19:32 -04:00
86cee568d5
feat: use upstream pbjson ( #2650 )
...
* feat: use upstream pbjson
* chore: fmt
2021-09-28 16:29:26 +00:00
daa2ec2f4c
revert: "fix: Revert "feat: remove routerify ( #2586 )" ( #2623 )" ( #2649 )
...
This reverts commit 18941fcbff
2021-09-28 14:42:17 +00:00
82de0af79b
chore(deps): bump libc from 0.2.102 to 0.2.103
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.102 to 0.2.103.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.102...0.2.103 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-27 19:46:32 +00:00
a55a21c644
chore: Update datafusion ( #2635 )
...
* chore: Update datafusion and sqlparser
* fix: remove STACK_SIZE workaround
* chore: update datafusion_util
* chore: update predicate
* chore: update query_tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-27 14:13:19 +00:00
8813801e67
chore(deps): bump ouroboros from 0.10.1 to 0.11.1
...
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.10.1 to 0.11.1.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/commits )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-27 01:27:24 +00:00
c427f84848
feat: correct pbjson serialization of Timestamp and Duration ( #2620 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-24 13:20:50 +00:00
18941fcbff
fix: Revert "feat: remove routerify ( #2586 )" ( #2623 )
...
This reverts commit 9fba7b0ba0
2021-09-24 09:50:24 +00:00
b46541406a
chore: update itertools ( #2615 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-22 18:25:35 +00:00
92e6173f49
feat: implement jaeger-agent protocol directly ( #2607 )
...
* feat: implement jaeger-agent protocol directly
* chore: review feedback
* fix: remove jaeger feature flag
2021-09-22 17:30:37 +00:00
e80d8aa391
chore: update latest datafusion and use its lit_timestamp_nano
2021-09-22 11:30:42 -04:00
d38648952c
chore: Update datafusion ( #2602 )
...
* chore: Update datafusion + other deps
* refactor: update query crate for new async interfaces
* refactor: update server crate for new async interface
* refactor: update query_tests crate for new async interfaces
* refactor: update ioxd and server to use new async interface
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-22 10:33:25 +00:00
24c1563535
chore: remove unused `generated_types` => `futures` dep
2021-09-22 11:34:36 +02:00
16a257e7b9
chore: remove unused `generated_types` => `serde_json` dep
2021-09-22 11:19:19 +02:00
d7b697dfe9
chore: remove unused `object_store` => `tracker` dep
2021-09-22 11:13:40 +02:00
5a2ee5c385
chore: remove unused `internal_types` => `observability_deps` dep
2021-09-22 11:09:06 +02:00
a9662b0edb
chore: remove unused `mutable_buffer` => `influxdb_line_protocol` dep
2021-09-22 10:56:15 +02:00
ef5ab67c77
chore: upgrade azure dependencies
2021-09-21 13:35:11 +02:00
cf39825770
chore: update DataFusion ( #2592 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-21 10:25:24 +00:00
46088a7ff1
feat: disable rustyline dirs-next dependency ( #2579 )
2021-09-20 14:42:07 +00:00
a51e9d8c19
chore(deps): bump tracing-subscriber from 0.2.22 to 0.2.24
...
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing ) from 0.2.22 to 0.2.24.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.22...tracing-subscriber-0.2.24 )
---
updated-dependencies:
- dependency-name: tracing-subscriber
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 09:27:21 +00:00
140c71eaf0
Merge branch 'main' into crepererum/issue2518a
2021-09-20 09:16:39 +00:00
e96aa49390
feat: disable prometheus protobuf ( #2578 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 09:16:04 +00:00
f62d0eab3c
feat: disable bytes serde ( #2580 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 09:07:12 +00:00
9fba7b0ba0
feat: remove routerify ( #2586 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 08:57:19 +00:00
c900704e58
chore: upgrade datafusion
2021-09-20 10:48:32 +02:00
876bb10cf8
chore(deps): bump rand_distr from 0.4.1 to 0.4.2
...
Bumps [rand_distr](https://github.com/rust-random/rand ) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/rust-random/rand/releases )
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/rand/compare/rand_distr-0.4.1...rand_distr-0.4.2 )
---
updated-dependencies:
- dependency-name: rand_distr
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:39:39 +00:00
1112597c44
chore(deps): bump dirs from 3.0.2 to 4.0.0
...
Bumps [dirs](https://github.com/soc/dirs-rs ) from 3.0.2 to 4.0.0.
- [Release notes](https://github.com/soc/dirs-rs/releases )
- [Commits](https://github.com/soc/dirs-rs/commits )
---
updated-dependencies:
- dependency-name: dirs
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:28:04 +00:00
c926994491
chore(deps): bump tracing from 0.1.27 to 0.1.28
...
Bumps [tracing](https://github.com/tokio-rs/tracing ) from 0.1.27 to 0.1.28.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.27...tracing-0.1.28 )
---
updated-dependencies:
- dependency-name: tracing
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:17:24 +00:00
82c3fb35e7
chore(deps): bump hyper from 0.14.12 to 0.14.13
...
Bumps [hyper](https://github.com/hyperium/hyper ) from 0.14.12 to 0.14.13.
- [Release notes](https://github.com/hyperium/hyper/releases )
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/hyper/compare/v0.14.12...v0.14.13 )
---
updated-dependencies:
- dependency-name: hyper
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 07:53:18 +00:00
9c80d32af5
refactor: use normal google timestamps in parquet metadata again
...
We changed from Google timestamp (which use variable-sized integers) to
our own fixed-sized integer timestamps so that the size of the parquet
metadata does not depend on the timestamp. However with the introduction
of compression this is the case anyways (since slightly different
timestamps lead to different compression results) and we need now
derministic timestamps for tests. So there is now point in using our own
timestamp type. Switching back to the variable-sized type also shrinks
the post-compression results a bit.
2021-09-20 09:34:03 +02:00
afc507ae14
feat: compress encoded parquet metadata
...
Depending on the number of columns, this should safe between 60% and
75%.
2021-09-20 09:33:18 +02:00
5298543321
feat: don't use opentelemetry async trace exporter ( #2576 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-19 13:20:13 +00:00
4382bc0b71
feat: disable Snafu futures feature ( #2573 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-19 11:36:24 +00:00
3ffb16daa6
feat: remove parse_duration ( #2574 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-19 11:27:24 +00:00
62f302b31a
feat: remove human format ( #2577 )
2021-09-19 11:14:26 +00:00
f34eab70b3
feat: add pbjson bytes support ( #2560 )
...
* feat: add pbjson bytes support
* chore: fix lint
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-16 16:46:12 +00:00
1d55d9a1b5
feat: add pbjson support ( #2468 )
...
* feat: add pbjson support
* chore: fix test
2021-09-16 07:33:27 +00:00
185f45f56b
chore: Update datafusion + other deps ( #2545 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-15 20:56:44 +00:00
c66095cad1
feat: remove metrics crate ( #2552 )
2021-09-15 19:43:33 +00:00
6e7fa3e574
feat: migrate http ingest metrics ( #2542 )
...
* feat: migrate http ingest metrics
* chore: review feedback
* refactor: RAII entry ingest recorder
2021-09-15 19:01:10 +00:00
72021e010f
test: test unsupported cases of predicate serialization
2021-09-15 18:18:05 +02:00
44eb3b994d
feat: `Predicate` serialization
...
Closes #2493 .
2021-09-15 16:37:25 +02:00
509c07330d
refactor: decouple `parquet_file` from `query`
2021-09-14 18:26:16 +02:00
bfaba78dc3
refactor: move `predicate` into its own crate
...
Two reasons:
1. I wanna decouple `parquet_file` from `query` (nearly done, needs a
small follow-up PR).
2. `predicate` will have more and more features (like serialization)
which justifies a new home
2021-09-14 17:13:02 +02:00
4795bd5c9c
refactor: stricter delete predicate TS parsing
...
As a a nice side effect, the parser no longer depends on the line
protocol parser.
2021-09-14 13:14:19 +02:00
3f2e46c397
feat: prune old transactions from preserved catalog
2021-09-14 12:08:17 +02:00
7f915ba9d4
feat: Add pre-generated values and tag sets to data generator
...
This adds the ability to pre-generate values and tag sets in the data generator. This makes it easy to have tags that depend on other tag values (like buckets in an org) and have tag values that have one associated tag (like if something is in production or staging environment). Follow on work will add the actual generation to the agent spec. An added bonus of these pre-generated values is that generating samples won't require any sort of template evaluation for all of the tags in the tag sets. Only unique values (like trace_id or span_id) would need to be generated during sampling generation.
2021-09-13 17:45:53 -04:00
5eef76c868
chore: Update dependencies (including datafusion) ( #2521 )
...
* chore: Update datafusion deps to pre-release
* refactor: Update IOx to use new datafusion Statistics
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-13 21:30:44 +00:00
f3bcafcfea
feat: migrate http metrics to metric crate ( #2508 )
...
* feat: migrate http metrics
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-13 18:56:20 +00:00
Marco Neumann
Edd Robinson
kodiakhq[bot]
Andrew Lamb
Carol (Nichols || Goulding)
Raphael Taylor-Davies
Nga Tran
Dom
Luke Bond
Carol (Nichols || Goulding)
NGA-TRAN
Paul Dix
Marko Mikulicic
dependabot[bot]
Jake Goulding
Andrew Lamb
Carol (Nichols || Goulding)