9bf7dee628
chore(deps): Bump libc from 0.2.144 to 0.2.145 ( #7921 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.144 to 0.2.145.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.144...0.2.145 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-06-05 09:34:15 +00:00
d8b06c59c4
chore(deps): Bump once_cell from 1.17.2 to 1.18.0
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.17.2 to 1.18.0.
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.17.2...v1.18.0 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-05 02:03:15 +00:00
00f579086d
chore(deps): Bump clap from 4.3.0 to 4.3.1
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.3.0 to 4.3.1.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.3.0...clap_complete-v4.3.1 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 15:24:33 +00:00
1d6fd83a9a
Merge branch 'main' into savage/wal-regenerate-lp-catalog-support
2023-06-02 14:23:55 +00:00
e9b5708c70
refactor(cli): Perform `regenerate-lp` using a sorted output comparison
...
Query the ingester directly through the test cluster to allow for less
brittle assertion of results.
2023-06-02 13:43:44 +01:00
50797b6967
test(cli): Assert writing `regenerate-lp` output produces same query results
...
This changes the e2e test to delete the WAL segment file, restart the
ingester and ensure the results returned by an ingester query after
feeding the regenerated line proto in are the same as those before.
2023-06-02 12:45:52 +01:00
9d445cc935
Merge branch 'main' into dom/strftime-panic
2023-06-02 10:03:39 +01:00
d743f071b5
Merge branch 'main' into sgc/issue/7829_time_bounds_2
2023-06-02 14:00:05 +10:00
d5719f9be2
refactor: Moved simplification of time range expressions to parser
2023-06-02 09:50:01 +10:00
47214ec9a0
fix: prevent panics in partitioning logic
...
Changes the partitioning logic to be fallible. This prevents an invalid
partition template from causing a panic, previously possible through two
known code paths:
* TagValue formatter referencing a non-tag column
* Time formatter using an invalid strftime format string
If either occurs, the write attempt is now aborted and an error returned
to the user with a HTTP 500 status code.
Additionally unexpected partitioner errors now map to a catch-all error
instead of panicking.
2023-06-01 17:44:44 +02:00
4b0753a800
Merge branch 'main' into dom/reversible-partition-key
2023-05-31 16:04:16 +01:00
a48f681e56
feat(parquet): reduce and limit buffering when writing parquet files ( #7880 )
...
* feat: limit buffering when writing parquet files ("combined solution")
* chore: Run cargo hakari tasks
---------
Co-authored-by: Raphael Taylor-Davies <r.taylordavies@googlemail.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-31 13:27:32 +00:00
e1c1908a0b
refactor: add `parquet_file` PG index for querier ( #7842 )
...
* refactor: add `parquet_file` PG index for querier
Currently the `list_by_table_not_to_delete` catalog query is somewhat
expensive:
```text
iox_catalog_prod=> select table_id, sum((to_delete is NULL)::int) as n from parquet_file group by table_id order by n desc limit 5;
table_id | n
----------+------
1489038 | 7221
1489037 | 7019
1491534 | 5793
1491951 | 5522
1513377 | 5339
(5 rows)
iox_catalog_prod=> EXPLAIN ANALYZE SELECT id, namespace_id, table_id, partition_id, object_store_id,
min_time, max_time, to_delete, file_size_bytes,
row_count, compaction_level, created_at, column_set, max_l0_created_at
FROM parquet_file
WHERE table_id = 1489038 AND to_delete IS NULL;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------
Bitmap Heap Scan on parquet_file (cost=46050.91..47179.26 rows=283 width=200) (actual time=464.368..472.514 rows=7221 loops=1)
Recheck Cond: ((table_id = 1489038) AND (to_delete IS NULL))
Heap Blocks: exact=7152
-> BitmapAnd (cost=46050.91..46050.91 rows=283 width=0) (actual time=463.341..463.343 rows=0 loops=1)
-> Bitmap Index Scan on parquet_file_table_idx (cost=0.00..321.65 rows=22545 width=0) (actual time=1.674..1.674 rows=7221 loops=1)
Index Cond: (table_id = 1489038)
-> Bitmap Index Scan on parquet_file_deleted_at_idx (cost=0.00..45728.86 rows=1525373 width=0) (actual time=460.717..460.717 rows=4772117 loops=1)
Index Cond: (to_delete IS NULL)
Planning Time: 0.092 ms
Execution Time: 472.907 ms
(10 rows)
```
I think this may also be because PostgreSQL kinda chooses the wrong
strategy, because it could just look at the existing index and filter
from there:
```text
iox_catalog_prod=> EXPLAIN ANALYZE SELECT id, namespace_id, table_id, partition_id, object_store_id,
min_time, max_time, to_delete, file_size_bytes,
row_count, compaction_level, created_at, column_set, max_l0_created_at
FROM parquet_file
WHERE table_id = 1489038;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------
Index Scan using parquet_file_table_idx on parquet_file (cost=0.57..86237.78 rows=22545 width=200) (actual time=0.057..6.994 rows=7221 loops=1)
Index Cond: (table_id = 1489038)
Planning Time: 0.094 ms
Execution Time: 7.297 ms
(4 rows)
```
However PostgreSQL doesn't know the cardinalities well enough. So
let's add a dedicated index to make the querier faster.
* feat: new migration system
* docs: explain dirty migrations
2023-05-31 10:56:32 +00:00
24f0dca838
test(cli): Add test to ensure `regenerate-lp` continues on minor errors
...
This adds a test to the `wal regenerate-lp` command to ensure that
non-fatal errors do not block regeneration of any other recoverable
entries.
2023-05-30 18:01:28 +01:00
37bb5e0585
test: arbitrary reversible partition keys
...
This test constructs a partition key from an arbitrary selection of
pre-defined parts, and uses the resulting template to partition a write
containing an arbitrary selection of pre-defined tag columns.
Once a partition key is derived, the test asserts build_column_values()
reverses it into the original set of tag (column_name, value) tuples
present in the write.
2023-05-30 15:58:26 +02:00
27bef292a3
feat: unambiguously reversible partition keys
...
This commit changes the format of partition keys when generated with
non-default partition key templates ONLY. A prior fixture test is
unchanged by this commit, ensuring the default partition keys remain
the same.
When a custom partition key template is provided, it may specify one or
more parts, with the TagValue template causing values extracted from tag
columns to appear in the derived partition key.
This commit changes the generated partition key in the following ways:
* The delimiter of multi-part partition keys; the character used to
delimit partition key parts is changed from "/" to "|" (the pipe
character) as it is less likely to occur in user-provided input,
reducing the encoding overhead.
* The format of the extracted TagValue values (see below).
Building on the work of custom partition key overrides, where an
immutable partition template is resolved and set at table creation time,
the changes in this PR enable the derived partition key to be
unambiguously reversed into the set of tag (column_name, column_value)
tuples it was generated from for use in query pruning logic. This is
implemented by the build_column_values() method in this commit, which
requires both the template, and the derived partition key.
Prior to this commit, a partition key value extracted from a tag column
was in the form "tagname_x" where "x" is the value and "tagname" is the
name of the tag column it was extracted from. After this commit, the
partition key value is in the form "x"; the column name is removed from
the derived string to reduce the catalog storage overhead (a key driver
of COGS). In the case of a NULL tag value, the sentinel value "!" is
inserted instead of the prior "tagname_" marker. In the case of an empty
string tag value (""), the sentinel "^" value is inserted instead of the
"tagname_-" marker, ensuring the distinction between an empty value and
a not-present tag is preserved.
Additionally tag values utilise percent encoding to encode reserved
characters (part delimiter, empty sentinel character, % itself) to
eliminate deserialisation ambiguity.
Examples of how this has changed derived partition keys, for a template
of [Time(YYYY-MM-DD), TagValue(region), TagValue(bananas)]:
Write: time=1970-01-01,region=west,other=ignored
Old: "1970-01-01-region_west-bananas"
New: "1970-01-01|west|!"
Write: time=1970-01-01,other=ignored
Old: "1970-01-01-region-bananas"
New: "1970-01-01|!|!"
2023-05-30 15:58:25 +02:00
840692e1f3
chore(deps): Bump once_cell from 1.17.1 to 1.17.2
...
Bumps [once_cell](https://github.com/matklad/once_cell ) from 1.17.1 to 1.17.2.
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md )
- [Commits](https://github.com/matklad/once_cell/compare/v1.17.1...v1.17.2 )
---
updated-dependencies:
- dependency-name: once_cell
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-30 02:58:12 +00:00
bfc395a30d
chore(deps): Bump comfy-table from 6.1.4 to 6.2.0 ( #7883 )
...
Bumps [comfy-table](https://github.com/nukesor/comfy-table ) from 6.1.4 to 6.2.0.
- [Release notes](https://github.com/nukesor/comfy-table/releases )
- [Changelog](https://github.com/Nukesor/comfy-table/blob/main/CHANGELOG.md )
- [Commits](https://github.com/nukesor/comfy-table/compare/v6.1.4...v6.2.0 )
---
updated-dependencies:
- dependency-name: comfy-table
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-29 09:22:56 +00:00
edbecd53af
chore(deps): Bump log from 0.4.17 to 0.4.18 ( #7884 )
...
Bumps [log](https://github.com/rust-lang/log ) from 0.4.17 to 0.4.18.
- [Release notes](https://github.com/rust-lang/log/releases )
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/log/compare/0.4.17...0.4.18 )
---
updated-dependencies:
- dependency-name: log
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-29 09:17:08 +00:00
e0720db138
chore(deps): Bump tokio from 1.28.1 to 1.28.2 ( #7885 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.28.1 to 1.28.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.28.1...tokio-1.28.2 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-29 09:11:24 +00:00
e2b9beffad
chore(deps): Bump criterion from 0.5.0 to 0.5.1
...
Bumps [criterion](https://github.com/bheisler/criterion.rs ) from 0.5.0 to 0.5.1.
- [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/bheisler/criterion.rs/compare/0.5.0...0.5.1 )
---
updated-dependencies:
- dependency-name: criterion
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-29 02:02:12 +00:00
2a07b53879
feat: add more tag predicate rewrite logic for InfluxQL ( #7869 )
...
* feat: add more tag predicate rewrite logic for InfluxQL
* chore: cargo fmt
* chore: fmt
* test: add more tests
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-26 21:53:52 +00:00
51d59f8216
refactor(`wal_inspect`): Make `LineProtoWriter` namespace unaware
...
Instead, the type responsible for initialising it handles namespaced
`Write` initialisation and management, as well as the failure paths that
may need handling. This commit introduces a `NamespaceDemultiplexer`
type with a generic implementation allowing fallible `async` lazy init
of any type from a given `NamespaceId`. This paves the way for catalog-aware
initialisation of `LineProtoWriter`s.
2023-05-26 17:12:35 +01:00
eb0d77f354
chore(deps): Bump regex from 1.8.2 to 1.8.3 ( #7873 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.8.2 to 1.8.3.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.8.2...1.8.3 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-26 09:34:25 +00:00
ececd0ada7
chore(deps): Bump base64 from 0.21.1 to 0.21.2 ( #7874 )
...
Bumps [base64](https://github.com/marshallpierce/rust-base64 ) from 0.21.1 to 0.21.2.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md )
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.21.1...v0.21.2 )
---
updated-dependencies:
- dependency-name: base64
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-05-26 09:28:43 +00:00
c4eca5fecf
Merge branch 'main' into cn/table-create-grpc-api
2023-05-25 18:53:12 +00:00
138b14e0db
chore: Update DataFusion and arrow to `40.0.0` ( #7864 )
...
* chore: Update DataFusion and arrow to `40.0.0`
* chore: Run cargo hakari tasks
* fix: update for API
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-25 17:34:51 +00:00
17219d71fe
feat: Use the table service in the router
2023-05-25 10:44:57 -04:00
e1a93252c5
feat: Add a new table service crate
2023-05-25 10:44:57 -04:00
89d8207784
chore(deps): Bump io-lifetimes from 1.0.10 to 1.0.11 ( #7865 )
...
Bumps [io-lifetimes](https://github.com/sunfishcode/io-lifetimes ) from 1.0.10 to 1.0.11.
- [Commits](https://github.com/sunfishcode/io-lifetimes/compare/v1.0.10...v1.0.11 )
---
updated-dependencies:
- dependency-name: io-lifetimes
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-25 07:49:53 +00:00
c8712bbc90
fix: Add a fixture test encoding and documenting default partition template assumptions
2023-05-24 10:36:52 -04:00
604bab9508
fix: Make Table create_or_get be only create
2023-05-24 10:34:30 -04:00
afb3838437
feat: Optionally supply the namespace partition template when creating a namespace
2023-05-24 10:10:34 -04:00
24a4f36d24
chore(deps): Bump proptest from 1.1.0 to 1.2.0 ( #7857 )
...
Bumps [proptest](https://github.com/proptest-rs/proptest ) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/proptest-rs/proptest/releases )
- [Changelog](https://github.com/proptest-rs/proptest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/proptest-rs/proptest/compare/v1.1.0...v1.2.0 )
---
updated-dependencies:
- dependency-name: proptest
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-24 09:21:32 +00:00
b7fbfa6fb2
chore(deps): Bump criterion from 0.4.0 to 0.5.0 ( #7856 )
...
Bumps [criterion](https://github.com/bheisler/criterion.rs ) from 0.4.0 to 0.5.0.
- [Changelog](https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/bheisler/criterion.rs/compare/0.4.0...0.5.0 )
---
updated-dependencies:
- dependency-name: criterion
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-24 09:08:37 +00:00
928a4d163e
build: remove unused dependencies from crates
...
This commit fixes loads of crates (47!) had unused dependencies, or
mis-configured dependencies (test deps as normal deps).
I added the "unused_crate_dependencies" to all crates to help prevent
this mess from growing again!
https://doc.rust-lang.org/beta/nightly-rustc/rustc_lint_defs/builtin/static.UNUSED_CRATE_DEPENDENCIES.html
This has the minor downside of false-positives when specifying
dev-dependencies for test/bench binaries - these are files in /test or
/benches (not normal tests). This commit includes a workaround,
importing them in lib.rs (gated by a feature flag). I think the
trade-off of better dependency management is worth it!
2023-05-23 14:55:43 +02:00
ef0bfb6186
chore(deps): Bump regex from 1.8.1 to 1.8.2 ( #7845 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.8.1...1.8.2 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-23 12:37:03 +00:00
c5ee3250f7
fix: flaky `service::tests::test_log_on_panic` ( #7850 )
...
* refactor: remove line number from test
* refactor: clean up code
* fix: flaky `service::tests::test_log_on_panic`
This seems to be a really rare issue. So provoking it requires a lot of
tests. This can be done (using `fish`):
```console
$ while cargo with "parallel -n0 {bin} {args} ::: $(seq 10)" -- test -p service_grpc_influxrpc --lib -- --test-threads 20; echo; end
```
Fixes #7838 .
* fix: panic handler construction
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-23 12:17:49 +00:00
b9bcaf1aa0
Merge branch 'main' into savage/wal-regenerate-lp-cli-command
2023-05-22 16:18:44 +00:00
0719928800
chore: remove unused deps
...
The wal crate imports a bunch of stuff it never uses!
2023-05-22 13:38:49 +02:00
f3f1a5edef
chore(deps): Bump sqlparser from 0.33.0 to 0.34.0 ( #7834 )
...
* chore(deps): Bump sqlparser from 0.33.0 to 0.34.0
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.33.0 to 0.34.0.
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.33.0...v0.34.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
* fix: sqlparser compat
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
2023-05-22 10:16:38 +00:00
6cb7619d83
chore(deps): Bump base64 from 0.21.0 to 0.21.1 ( #7832 )
...
Bumps [base64](https://github.com/marshallpierce/rust-base64 ) from 0.21.0 to 0.21.1.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md )
- [Commits](https://github.com/marshallpierce/rust-base64/commits )
---
updated-dependencies:
- dependency-name: base64
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-22 09:50:06 +00:00
63b3279e33
chore(deps): Bump digest from 0.10.6 to 0.10.7 ( #7835 )
...
Bumps [digest](https://github.com/RustCrypto/traits ) from 0.10.6 to 0.10.7.
- [Commits](https://github.com/RustCrypto/traits/compare/digest-v0.10.6...digest-v0.10.7 )
---
updated-dependencies:
- dependency-name: digest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-22 08:48:17 +00:00
bf847584d4
chore(deps): Bump clap from 4.2.7 to 4.3.0 ( #7833 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.2.7 to 4.3.0.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.2.7...clap_complete-v4.3.0 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-22 08:33:05 +00:00
8cd96f7238
chore(deps): Bump toml from 0.7.3 to 0.7.4 ( #7827 )
...
Bumps [toml](https://github.com/toml-rs/toml ) from 0.7.3 to 0.7.4.
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.3...toml-v0.7.4 )
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-05-19 09:25:49 +00:00
90a25a3ff0
chore: update DataFusion ( #7825 )
...
* chore: update DataFusion
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-05-18 17:51:16 +00:00
e97c48b388
feat: download catalog objects as part of `remote store get-table` ( #7779 )
...
* feat: download catalog objects as part of remote store get-table
* fix: fix future_not_send
* fix: reorder import order
* chore: reduce repetition of `Error` in naming
* fix: improve the partition filtering API
* fix: clarify export comment
* fix: make index printing consistent
* fix: improve skip message
* chore: add extra clippy lints to inmport_export crate
* fix: comments
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-18 16:52:43 +00:00
c263585d94
feat(cli): Add table ID `debug wal regenerate-lp` command implementation
...
This adds a command to `influxdb_iox` that can take a WAL segment file
and regenerate all write operation entries, writing to stdout or namespaced
files within a target directory, using table ID as the measurement name
in the case where there is no catalog access at point of regeneration.
2023-05-18 17:20:41 +01:00
43a1f7c0b2
chore(deps): Bump rustls from 0.21.0 to 0.21.1 ( #7819 )
...
Bumps [rustls](https://github.com/rustls/rustls ) from 0.21.0 to 0.21.1.
- [Changelog](https://github.com/rustls/rustls/blob/main/RELEASE_NOTES.md )
- [Commits](https://github.com/rustls/rustls/compare/v/0.21.0...v/0.21.1 )
---
updated-dependencies:
- dependency-name: rustls
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-05-18 09:29:26 +00:00
62fed73bcd
refactor: upgrade DataFusion to `19b03240920ad63cac916b42951754c0337bdac8#19b03240920ad63cac916b42951754c0337bdac8` ( #7813 )
...
I need:
- https://github.com/apache/arrow-datafusion/pull/6226 .
Changes in code due to:
- https://github.com/apache/arrow-datafusion/pull/6332
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-17 13:57:12 +00:00
e51ca3bc2d
chore(deps): Bump reqwest from 0.11.17 to 0.11.18 ( #7806 )
...
* chore(deps): Bump reqwest from 0.11.17 to 0.11.18
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.17 to 0.11.18.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.17...v0.11.18 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-17 06:15:24 +00:00
506bd80f6f
Merge branch 'main' into chore/router-metrics-for-auth-v2
2023-05-16 10:24:42 -07:00
c077dec0e0
refactor: rename `import` to `iox_import_export` ( #7799 )
...
* refactor: rename `import` crate to `import_export`
* chore: Consolidate tsm import/export
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-16 17:10:17 +00:00
2bb6445668
chore: update DataFusion and arrow / arrow-flight / parquet to `39.0.0` ( #7793 )
...
* chore: update DataFusion and arrow/parquet/arrow-flight to 39.0.0
* chore: update DataFusion and arrow/parquet/arrow-flight to 39.0.0 in workspace-hack/Cargo.toml
* chore: Run cargo hakari tasks
* chore: fix CI test and lint
* chore: update csv schema
* refactor: remove type-annotate for `Arc`
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-16 13:42:26 +00:00
6bfa269896
Merge branch 'main' into savage/wal-inspect-library-refactor
2023-05-16 11:07:47 +00:00
3462e29859
chore(deps): Bump uuid from 1.3.2 to 1.3.3
...
Bumps [uuid](https://github.com/uuid-rs/uuid ) from 1.3.2 to 1.3.3.
- [Release notes](https://github.com/uuid-rs/uuid/releases )
- [Commits](https://github.com/uuid-rs/uuid/compare/1.3.2...1.3.3 )
---
updated-dependencies:
- dependency-name: uuid
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-16 02:00:24 +00:00
de3ff7aa2d
chore: update MockAuthorizer to use return call pattern
2023-05-15 09:26:09 -07:00
db76a85a4d
chore: create duration metric decorator for Authorizer
2023-05-15 08:36:24 -07:00
867fd39dbf
Merge branch 'main' into authz/refactor-interface
2023-05-15 08:03:10 -07:00
e14bc925db
Merge branch 'main' into cn/move-peas
2023-05-15 14:36:18 +00:00
7735e7c95b
chore: Update DataFusion again ( #7777 )
...
* chore: Update datafusion again
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-15 12:38:45 +00:00
6aa634c1b9
Merge branch 'main' into cn/move-peas
2023-05-15 13:29:42 +01:00
fba9836f2a
chore(deps): Bump pin-project from 1.0.12 to 1.1.0
...
Bumps [pin-project](https://github.com/taiki-e/pin-project ) from 1.0.12 to 1.1.0.
- [Release notes](https://github.com/taiki-e/pin-project/releases )
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md )
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.0.12...v1.1.0 )
---
updated-dependencies:
- dependency-name: pin-project
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-15 02:02:32 +00:00
62d83b9219
test: add tests for IoxAuthorizer contract
2023-05-12 15:31:12 -05:00
ea3029ee3f
refactor: break out Authorizer trait versus IoxAuthorizer struct.
...
* make the boundaries more evident
* also make explicit what actions are tied to the IoxAuthorizer client (a.k.a. the client connection & request)
2023-05-12 15:28:58 -05:00
14007808bd
fix: Move remaining conversions between data types and proto into data_types
...
And have data_types depend on generated_types rather than vice versa.
2023-05-12 13:31:04 -04:00
1770d0f4d8
fix: Move ingester-querier gRPC communication to its own crate
2023-05-12 13:28:30 -04:00
ac0ea99cde
chore(deps): Bump tracing-core from 0.1.30 to 0.1.31 ( #7781 )
...
Bumps [tracing-core](https://github.com/tokio-rs/tracing ) from 0.1.30 to 0.1.31.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-core-0.1.30...tracing-core-0.1.31 )
---
updated-dependencies:
- dependency-name: tracing-core
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-05-12 13:40:04 +00:00
6c185bdec8
chore(deps): Bump handlebars from 4.3.6 to 4.3.7
...
Bumps [handlebars](https://github.com/sunng87/handlebars-rust ) from 4.3.6 to 4.3.7.
- [Release notes](https://github.com/sunng87/handlebars-rust/releases )
- [Changelog](https://github.com/sunng87/handlebars-rust/blob/v4.3.7/CHANGELOG.md )
- [Commits](https://github.com/sunng87/handlebars-rust/compare/v4.3.6...v4.3.7 )
---
updated-dependencies:
- dependency-name: handlebars
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-12 13:11:56 +00:00
bd1d1f648c
chore(deps): Bump serde from 1.0.162 to 1.0.163 ( #7780 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.162 to 1.0.163.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.162...v1.0.163 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-12 13:07:56 +00:00
36c1d23e19
chore(deps): Bump tokio from 1.28.0 to 1.28.1 ( #7776 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.28.0 to 1.28.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.28.0...tokio-1.28.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-11 19:46:58 +00:00
d99444fb2e
chore(deps): Bump console-subscriber from 0.1.8 to 0.1.9
...
Bumps [console-subscriber](https://github.com/tokio-rs/console ) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/tokio-rs/console/releases )
- [Commits](https://github.com/tokio-rs/console/compare/console-subscriber-v0.1.8...console-subscriber-v0.1.9 )
---
updated-dependencies:
- dependency-name: console-subscriber
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-10 01:59:08 +00:00
23c0110b32
feat: Create newtypes for different partition templates
...
So that the different kinds aren't mixed up. Also extracts the logic
having to do with which template takes precedence onto the
PartitionTemplate type itself.
2023-05-09 14:55:02 +02:00
372ec8ef96
Merge branch 'main' into cn/delete-experiments
2023-05-09 10:17:30 +01:00
9229ce5668
fix: Rename compactor2_test_utils to compactor_test_utils
2023-05-09 11:02:11 +02:00
45493b91e5
fix: Rename ioxd_compactor2 to ioxd_compactor
2023-05-09 11:02:11 +02:00
dd9c5d1b13
fix: Rename compactor2 to compactor
2023-05-09 10:58:55 +02:00
6000d1e8d4
fix: Remove unused crates
2023-05-08 20:24:56 -04:00
3d5df5574a
fix: Remove vestiges of shards
2023-05-08 20:24:36 -04:00
0849ce6f2b
fix: Rename ingester2_test_ctx to ingester_test_ctx
2023-05-08 20:23:02 -04:00
19a56bdf4b
fix: Rename ioxd_ingester2 to ioxd_ingester
2023-05-08 20:20:03 -04:00
68117aee0f
chore(deps): Bump libc from 0.2.142 to 0.2.144 ( #7766 )
...
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.142 to 0.2.144.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.142...0.2.144 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-08 19:12:58 +00:00
56916cf942
fix: Rename ingester2 to ingester
2023-05-08 12:03:05 -04:00
2860d87fe1
chore: Update DataFusion ( #7756 )
...
* chore: Update DataFusion pin
* chore: Update explain plans
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-05-05 18:58:18 +00:00
aa70b14fe4
chore(deps): Bump rustix from 0.37.18 to 0.37.19 ( #7752 )
...
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.37.18 to 0.37.19.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.37.18...v0.37.19 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-05 15:47:26 +00:00
da6e123e9d
chore(deps): Bump serde from 1.0.160 to 1.0.162 ( #7751 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.160 to 1.0.162.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.160...1.0.162 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-05 13:29:50 +00:00
667da5eea4
chore: Add test for `dsn-file://` catalog urls ( #7735 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-04 10:48:18 +00:00
abe5d26d2f
chore: update DataFusion to `2787e7a36a6be83d91201df20827d3695f933300` ( #7732 )
...
Required to get:
- https://github.com/apache/arrow-datafusion/pull/6199
2023-05-03 16:40:00 +00:00
b2e5ea2266
refactor(wal): Add test & docs for WriteOpEntryDecoder
...
Adds some documentation for the WriteOpEntryDecoder and
a unit test that asserts it skips over non write entries
and can continue to be consumed from.
2023-05-03 11:51:00 +01:00
f6dea224e8
feat(wal): Add wal_inspect crate & a write op entry decoder
...
This adds a new crate with a type capable of converting
decoded WAL Write Op entries to line protocol and writing
the result to a namespaced destination. The wal crate
now exports a type which reads the sequenced wal ops and
decodes them as namespaced table batch writes.
2023-05-03 11:50:59 +01:00
bd07add78d
chore(deps): Bump clap from 4.2.5 to 4.2.7 ( #7726 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.2.5 to 4.2.7.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.2.5...v4.2.7 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-03 09:39:10 +00:00
65ca52ce3b
chore: Update DataFusion again ( #7722 )
...
* chore: Update DF again
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-05-02 18:34:45 +00:00
2b1f8b56e2
chore: Update DataFusion ( #7719 )
...
* chore: Update DataFusion
* chore: update for API change
* chore: update some tests
* fix: Update plans in optimizer
* chore: Update plans
* chore: Update error messages
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-05-02 17:55:04 +00:00
18b02a03cf
chore: move to the smaller, cargo-team maintained `home` crate ( #7716 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-02 16:10:22 +00:00
0c4a86cbaa
fix: Use humantime rather than chrono-english
2023-05-01 11:31:42 -04:00
cd4d961db9
fix: garbage_collector no longer uses chrono-english so it can use the workspace hack crate
2023-05-01 11:31:42 -04:00
67957d6fb5
chore(deps): Bump rustix from 0.37.15 to 0.37.18 ( #7701 )
...
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.37.15 to 0.37.18.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.37.15...v0.37.18 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-01 15:11:37 +00:00
b7c3711282
chore(deps): Bump uuid from 1.3.1 to 1.3.2 ( #7702 )
...
Bumps [uuid](https://github.com/uuid-rs/uuid ) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/uuid-rs/uuid/releases )
- [Commits](https://github.com/uuid-rs/uuid/compare/1.3.1...1.3.2 )
---
updated-dependencies:
- dependency-name: uuid
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-01 14:54:28 +00:00
9146306015
chore(deps): Bump flate2 from 1.0.25 to 1.0.26 ( #7703 )
...
Bumps [flate2](https://github.com/rust-lang/flate2-rs ) from 1.0.25 to 1.0.26.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases )
- [Commits](https://github.com/rust-lang/flate2-rs/commits )
---
updated-dependencies:
- dependency-name: flate2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-01 14:44:06 +00:00
3bbeb99c9f
fix: Fix cargo_audit using tracing-subscriber ( #7710 )
2023-05-01 14:09:21 +00:00
dependabot[bot]
kodiakhq[bot]
Fraser Savage
Dom
Stuart Carnie
Andrew Lamb
Marco Neumann
Christopher M. Wolff
Carol (Nichols || Goulding)
wiedld
Chunchun Ye
wiedld
Utkarsh Gupta