008270b346
chore(deps): Bump serde from 1.0.156 to 1.0.159 ( #7372 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.156 to 1.0.159.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.156...v1.0.159 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-30 09:23:03 +00:00
97dd61bfb1
chore(deps): Bump is-terminal from 0.4.5 to 0.4.6 ( #7371 )
...
* chore(deps): Bump is-terminal from 0.4.5 to 0.4.6
Bumps [is-terminal](https://github.com/sunfishcode/is-terminal ) from 0.4.5 to 0.4.6.
- [Release notes](https://github.com/sunfishcode/is-terminal/releases )
- [Commits](https://github.com/sunfishcode/is-terminal/compare/v0.4.5...v0.4.6 )
---
updated-dependencies:
- dependency-name: is-terminal
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-30 08:18:16 +00:00
066c3280eb
fix: phys. optimizers must respect sort partitioning ( #7362 )
...
* fix: sort pushdown must preserve partioning
* fix: projection pushdown must preserve sort partitioning
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-30 08:10:21 +00:00
19a0c7fe9c
feat: Teach InfluxQL how to process `FILL(null|previous|<value>)` ( #7359 )
...
* chore: Publicise gap-filling APIs
Helps #6916
* feat: IOx learns `FILL(null|previous|<value>)`
Helps #6916
* chore: More test cases
* chore: Revert change to TreeNodeVisitor
* chore: Update snapshot with expected gap-filling changes
2023-03-29 23:11:20 +00:00
f41c1a7945
feat: update gap fill planner rule to use LOCF ( #7358 )
...
* feat: update gap fill planner rule to use LOCF
* chore: cargo fmt
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-29 15:45:34 +00:00
d8b0139ea9
chore: Update datafusion + arrow/parquet/arrow-flight to 36 ( #7354 )
...
* chore: Update datafusion + arrow/parquet/arrow-flight to 36
* refactor: update optimize for new API
* refactor: update parquet for new API
* chore: Update more dependencies
* chore: Update to use the new buffer creation APIs
* chore: Run cargo hakari tasks
* fix: bad len
* fix: update for API change
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-29 13:41:59 +00:00
208d83d982
feat(flight): Support influxql queries via JSON encoded tickets ( #7357 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-29 13:34:05 +00:00
581d402368
Merge pull request #7343 from influxdata/savage/add-tests-for-hot-partition-persistence
...
feat(ingester2): Add test coverage for hot partition persistence
2023-03-29 11:06:34 +00:00
2e0d7fa4a9
Merge branch 'main' into savage/add-tests-for-hot-partition-persistence
2023-03-29 11:59:48 +01:00
d2a561b177
chore(deps): Bump rustix from 0.36.11 to 0.37.4 ( #7360 )
...
* chore(deps): Bump rustix from 0.36.11 to 0.37.4
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.11 to 0.37.4.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.11...v0.37.4 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-03-29 10:48:35 +00:00
39856ad432
fix: projection pushdown should project `ParquetExec` ordering ( #7356 )
...
* fix: projection pushdown should project `ParquetExec` ordering
Bug found while working on the final steps for #6098 .
* fix: Update expected output
* test: make test even harder
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2023-03-29 09:05:19 +00:00
52e54e0f8d
feat: more aggressive `CombineChunks` ( #7355 )
...
Try to combine chunks even when not all Union-arms/inputs are
combinable. This will later help to transform
```yaml
---
union:
- parquet:
files: [f1]
- parquet:
files: [f2]
- dedup:
parquet:
files: [f3]
```
into
```yaml
---
union:
- parquet:
files: [f1, f2]
- dedup:
parquet:
files: [f3]
```
Helps #6098 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-29 06:24:17 +00:00
75dba43ced
test: extend retention policy query test ( #7352 )
...
Add an ingester chunk to the parquet chunks.
Helpful for #6098 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-29 05:32:22 +00:00
a26cb6032a
feat: InfluxQL learns LOG function (arity 2) ( #7333 )
2023-03-28 20:22:27 +00:00
43e236e040
chore: Update datafusion again ( #7353 )
...
* chore: Update DataFusion
* refactor: Update predicate crate for new transform API
* refactor: Update iox_query crate for new APIs
* refactor: Update influxql for new API
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-28 16:21:49 +00:00
b277b620fa
Merge pull request #7326 from influxdata/jrb_21_backfill_handling
...
feat: vertical split files when L0s are highly overlapped
2023-03-28 10:57:53 -05:00
041f14ae50
Merge remote-tracking branch 'origin/main' into jrb_21_backfill_handling
2023-03-28 10:34:05 -05:00
f834df004b
chore: insta churn
2023-03-28 10:23:57 -05:00
a241ba1596
chore: sort files before selecting split candidate
2023-03-28 10:23:26 -05:00
dbf6493312
feat: add scalar function LOCF ( #7347 )
...
* feat: add scalar function LOCF
* chore: cargo update spin@0.9.6
Apparently this version was yanked
2023-03-28 14:35:27 +00:00
f16622fdc1
docs: add compactor invariants and rules ( #7346 )
...
* docs: add compactor invariants and rules
* chore: address review commnents
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-28 13:35:04 +00:00
95732fe221
docs: Update gRPC example in README ( #7345 )
...
Given that `ManagementService` no longer exists, the example is misleading and returns an error.
- Replace `ManagementService` example with `NamespaceService` examples.
- Revise instruction for providing `.proto` files and using the wrapper script.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-28 11:26:40 +00:00
4466e16018
chore(deps): Bump reqwest from 0.11.15 to 0.11.16 ( #7349 )
...
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.15 to 0.11.16.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.15...v0.11.16 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-28 11:19:51 +00:00
1ad0160d30
chore(deps): Bump serde_json from 1.0.94 to 1.0.95 ( #7348 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.94 to 1.0.95.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.94...v1.0.95 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-28 11:09:59 +00:00
5a965e2481
Merge branch 'main' into savage/add-tests-for-hot-partition-persistence
2023-03-28 11:37:20 +01:00
c7b3565990
refactor: Compare data inside the partition as persisted
...
Use tokio::task::yield_now to allow the persist task to
be enqueued instead of tokio::time::advance.
2023-03-28 11:28:53 +01:00
621672b38f
test: Add an e2e test for write replication and fix it because it didn't work 😜 ( #7344 )
...
* test: Add an e2e test for write replication
* fix: Pass through rpc_write_replicas configuration to RpcWrite handler
---------
Co-authored-by: Dom <dom@itsallbroken.com>
2023-03-28 09:31:16 +00:00
f588bd5e8d
chore: update `spin` to 0.9.7 ( #7350 )
...
0.9.6 was yanked.
2023-03-28 08:47:25 +00:00
0db347bc1a
Merge remote-tracking branch 'origin/main' into jrb_21_backfill_handling
...
# Conflicts:
# compactor2/tests/layouts/knobs.rs
# compactor2/tests/layouts/large_files.rs
# compactor2/tests/layouts/many_files.rs
2023-03-27 14:20:16 -05:00
638a8ea4a8
chore: move random backfill tests to their own file
2023-03-27 13:48:16 -05:00
ed282ed738
chore: add case to cause non-overlapped file to be processed by high_l0_overlap_split
2023-03-27 13:40:06 -05:00
6a60914f95
chore: insta updates
2023-03-27 11:15:24 -05:00
af71e3295c
Merge pull request #7342 from influxdata/dom/cleanup-mock-iter
...
refactor(test): generic return iterator
2023-03-27 15:01:25 +01:00
de043f93e5
feat(ingester2): Add test coverage for hot partition persistence
...
This commit adds a test for the hot partition persistence
post-write observer functionality.
2023-03-27 12:46:54 +01:00
874325d9ce
refactor(test): generic return iterator
...
This change allows the MockWriteClient to accept more input types and
internalises the box dyn bits; I got tired of writing "Box::new()"
everywhere.
2023-03-27 13:31:31 +02:00
71b88b22b9
fix: ensure we don't loose predicates in chunk roundtrips ( #7340 )
...
`extract_chunks` never runs after predicate pushdown. However IF this
should ever happen, we would potentially forget the predicates attached
to `ParquetExec`. So let's make sure we refuse chunk extraction in this
case. This is similar to the existing behavior, i.e. we don't support
chunk extraction after filter pushdown (i.e. if there is a filter around
an `RecordBatchesExec`).
For #6098 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-27 11:18:56 +00:00
44551e7519
chore(deps): Bump regex from 1.7.2 to 1.7.3 ( #7338 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.7.2 to 1.7.3.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.7.2...1.7.3 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-27 09:20:35 +00:00
e067cfd3d7
Merge pull request #7316 from influxdata/dom/replicate
...
feat: write replication
2023-03-27 10:13:11 +01:00
48333f66a4
Merge branch 'main' into dom/replicate
2023-03-27 10:06:07 +01:00
e24fc9d2d8
Merge pull request #7339 from influxdata/dependabot/cargo/mockito-1.0.1
...
chore(deps): Bump mockito from 1.0.0 to 1.0.1
2023-03-27 10:05:18 +01:00
b62099ea4a
Merge branch 'main' into dom/replicate
2023-03-27 10:04:54 +01:00
6105fd5bb7
refactor: compute n_copies once
...
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2023-03-27 10:04:17 +01:00
6a4d55e905
chore(deps): Bump mockito from 1.0.0 to 1.0.1
...
Bumps [mockito](https://github.com/lipanski/mockito ) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/lipanski/mockito/releases )
- [Commits](https://github.com/lipanski/mockito/compare/1.0.0...1.0.1 )
---
updated-dependencies:
- dependency-name: mockito
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-27 08:35:31 +00:00
00a6676475
chore(deps): Bump predicates from 3.0.1 to 3.0.2 ( #7337 )
...
Bumps [predicates](https://github.com/assert-rs/predicates-rs ) from 3.0.1 to 3.0.2.
- [Release notes](https://github.com/assert-rs/predicates-rs/releases )
- [Changelog](https://github.com/assert-rs/predicates-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/predicates-rs/compare/v3.0.1...v3.0.2 )
---
updated-dependencies:
- dependency-name: predicates
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-27 08:32:27 +00:00
010d35c2b5
chore(deps): Bump indexmap from 1.9.2 to 1.9.3 ( #7336 )
...
Bumps [indexmap](https://github.com/bluss/indexmap ) from 1.9.2 to 1.9.3.
- [Release notes](https://github.com/bluss/indexmap/releases )
- [Changelog](https://github.com/bluss/indexmap/blob/master/RELEASES.md )
- [Commits](https://github.com/bluss/indexmap/compare/1.9.2...1.9.3 )
---
updated-dependencies:
- dependency-name: indexmap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-27 08:19:43 +00:00
976dadb4d4
chore: split out the time range selection for splitting into separate function
2023-03-24 16:59:02 -05:00
0d66b04f68
refactor(flightsql): Build `GetTables` directly from DataFusion catalog ( #7325 )
...
* refactor: Build GetTables response directly from SessionContext
* fix: fmt
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-24 19:15:24 +00:00
f73187ff7e
feat: add interpolation fill strategy to GapFillExec ( #7317 )
...
* feat: add interpolation fill strategy to GapFillExec
* chore: clippy
* chore: code review feedback
* chore: fix doc comments
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-24 18:53:14 +00:00
1817104236
chore: Update DataFusion again ( #7329 )
...
* chore: update datafusion
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-24 18:45:53 +00:00
35865e1ab3
refactor(flightsql): Build `GetCatalogs` directly from DataFusion catalog ( #7327 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-24 18:34:56 +00:00
dependabot[bot]
Marco Neumann
Stuart Carnie
Christopher M. Wolff
Andrew Lamb
kodiakhq[bot]
Fraser Savage
Joe-Blount
Joe-Blount
Nga Tran
Jason Stirnaman
Carol (Nichols || Goulding)
Dom