a49a0aaa21
chore(deps): Bump zstd from 0.12.3+zstd.1.5.2 to 0.12.4 ( #8280 )
...
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.12.3+zstd.1.5.2 to 0.12.4.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-20 14:14:08 +00:00
0173c50ba1
fix: use correct error code when querier is shutting down ( #8282 )
...
When a long running query is in process and the querier is shutting
down, it might happen that the executor (= thread pool and tokio
executor responsible for the CPU-bound DataFusion execution) is shut
down while the query is running. From a "systems interaction" PoV I
think this is totally fine and I would like to avoid some weird
ref-counting. Or in other words: if the system is shutting down, shut it
down.
However the error was treated as "internal" which is not useful. The
client should rather be informed that its server was gone and that it is
OK (and desired) to retry. So as per
<https://grpc.github.io/grpc/core/md_doc_statuscodes.html > I think this
should signal "unavailable".
This change wires the error code in such a way that the gRPC service
layer can properly inspect it and then changes the error mapping.
Ref https://github.com/influxdata/idpe/issues/17917 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-20 12:08:22 +00:00
9263b42c35
chore(deps): Bump clap from 4.3.16 to 4.3.17 ( #8277 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.3.16 to 4.3.17.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.3.16...v4.3.17 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-20 12:03:06 +00:00
6ae9143742
fix: `end_to_end_cases::cli::query_ingester` flakyness ( #8281 )
...
While I cannot reproduce the CI flakyness locally (probably because the
local system is fast enough), looking at the test convinced me that the
ingester should not persist.
Closes #8245 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-20 11:48:59 +00:00
d3432198b6
revert: batch partition catalog requests in querier ( #8269 ) ( #8283 )
...
Panics in prod.
This reverts commit 0c347e8e64
2023-07-20 09:42:40 +00:00
0994f54af4
chore(deps): Bump sqlparser from 0.35.0 to 0.36.0 ( #8279 )
...
* chore(deps): Bump sqlparser from 0.35.0 to 0.36.0
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.35.0 to 0.36.0.
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.35.0...v0.36.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-20 09:30:14 +00:00
0f2cbcaef5
chore(deps): Bump zstd-safe from 6.0.5+zstd.1.5.4 to 6.0.6 ( #8275 )
...
Bumps [zstd-safe](https://github.com/gyscos/zstd-rs ) from 6.0.5+zstd.1.5.4 to 6.0.6.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd-safe
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-20 09:05:54 +00:00
4eff0fb359
chore(deps): Bump tower-http from 0.4.1 to 0.4.2 ( #8278 )
...
Bumps [tower-http](https://github.com/tower-rs/tower-http ) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/tower-rs/tower-http/releases )
- [Commits](https://github.com/tower-rs/tower-http/compare/tower-http-0.4.1...tower-http-0.4.2 )
---
updated-dependencies:
- dependency-name: tower-http
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-20 08:36:49 +00:00
0c347e8e64
feat: batch partition catalog requests in querier ( #8269 )
...
This is mostly wiring that builds on top of the other PRs linked to #8089 .
I think we eventually could make the batching code nicer by adding
better wrappers / helpers, but lets do that if we have other batched
caches and this patterns proofs to be useful.
Closes #8089 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-20 08:31:49 +00:00
59c532ee56
chore(deps): Bump serde from 1.0.171 to 1.0.173 ( #8276 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.171 to 1.0.173.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.171...v1.0.173 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-20 08:05:38 +00:00
2f2fcb6f05
refactor(influxql): make DERIVATIVE a user-defined window function ( #8265 )
...
Now that user-defined window functions are available, change the
DERIVATIVE and NON_NEGATIVE_DERIVATIVE function implementations to
use user-defined windows functions. This should improve performance
by allowing the entire window to be processed in one go, rather
than processing one row at a time.
The implementation is also moved out of the planner module alongside
the other user-defined window functions.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-20 06:19:02 +00:00
b3fca844d5
chore: update doc to `remote store get-table` ( #8261 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-19 20:23:41 +00:00
2519c9c6d2
Merge pull request #8262 from influxdata/jrb_65_move_progress_reporting
...
fix(compactor timeout): chunk up high plan counts in the compactor
2023-07-19 14:16:48 -05:00
ccb950445c
Merge remote-tracking branch 'origin/main' into jrb_65_move_progress_reporting
2023-07-19 14:05:48 -05:00
4c5c2473e6
chore: move scratchpad cleanup earlier
2023-07-19 10:16:28 -05:00
4e88571142
feat: add batch partition getters ( #8268 )
2023-07-19 15:05:41 +00:00
004b401a05
chore: upgrade to sqlx 0.7.1 ( #8266 )
...
There are a bunch of dependencies in `Cargo.lock` that are related to
mysql. These are NOT compiled at all, and are also not part of `cargo
tree`. The reason for the inclusion is a bug in cargo:
https://github.com/rust-lang/cargo/issues/10801
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-19 12:18:57 +00:00
d3b3805e5f
chore(deps): Bump clap from 4.3.15 to 4.3.16 ( #8263 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.3.15 to 4.3.16.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.3.15...v4.3.16 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-19 09:26:04 +00:00
fbbaf155ec
feat: ability to batch cache loader requests ( #8259 )
...
Core functionality for #8089 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-19 09:20:40 +00:00
69e4f998dd
fix: chunk up high plan counts in the compactor
2023-07-18 15:16:04 -05:00
b79009b128
refactor: refactor `Schemas` type in InfluxQL planner ( #8237 )
...
* refactor: refactor `Schemas` type in InfluxQL planner
* chore: update after merging main
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-18 17:14:39 +00:00
efae0f108a
feat(idpe-17887): enable `/` in db name for v1 write. ( #8235 )
...
* test case for proposed new behavior in v1 write endpoint.
* autogen and default are equivalent reserved words for rp
* have write endpoint match query endpoint, in that db and rp are always concated
2023-07-18 09:36:25 -07:00
1723aa46a6
docs: adjust querier cache docs ( #8260 )
...
`parquet_file` cache has a TTL as of #7975 .
2023-07-18 15:13:15 +00:00
67d1d41b11
chore(deps): Bump scopeguard from 1.1.0 to 1.2.0 ( #8255 )
...
Bumps [scopeguard](https://github.com/bluss/scopeguard ) from 1.1.0 to 1.2.0.
- [Commits](https://github.com/bluss/scopeguard/compare/v1.1.0...v1.2.0 )
---
updated-dependencies:
- dependency-name: scopeguard
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-18 12:49:34 +00:00
4c4c9f731c
chore(deps): Bump uuid from 1.4.0 to 1.4.1 ( #8256 )
...
Bumps [uuid](https://github.com/uuid-rs/uuid ) from 1.4.0 to 1.4.1.
- [Release notes](https://github.com/uuid-rs/uuid/releases )
- [Commits](https://github.com/uuid-rs/uuid/compare/1.4.0...1.4.1 )
---
updated-dependencies:
- dependency-name: uuid
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-18 12:30:42 +00:00
39450c54da
refactor: cache system code moves ( #8258 )
...
* refactor: fix `TestLoader` arg order
* refactor: extract test utils
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-18 12:20:27 +00:00
14f99f219b
chore(deps): Bump clap from 4.3.12 to 4.3.15 ( #8254 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.3.12 to 4.3.15.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.3.12...v4.3.15 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-18 09:18:58 +00:00
0bb9e059e5
refactor(influxql): make difference a user-defined window function ( #8257 )
...
Change the implementation of DIFFERENCE, and NON_NEGATIVE_DIFFERENCE,
to use a user-defined window function, rather than a user-defined
aggregator function. This slightly simplifies the implementation.
The window function calculates difference operation on the full
window in one operation rather than a row at a time. Where ther are
no NULL values in the input this function uses arrow compute kernels.
As part of this refactor the window function's implementation is
moved out of the planner module.
2023-07-18 09:03:58 +00:00
d1640bb926
feat(influxql): CUMULATIVE_SUM window function ( #8248 )
...
* feat(influxql): CUMULATIVE_SUM window function
Implement the InfluxQL CUMULATIVE_SUM window function. This is
implemented as described in
https://docs.influxdata.com/influxdb/v1.8/query_language/functions/#cumulative_sum .
* chore: Add a test demonstrating NULL handling of CUMULATIVE_SUM
---------
Co-authored-by: Stuart Carnie <stuart.carnie@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-18 06:13:58 +00:00
33e41fc5cb
fix: improve error for malformed gap fill query ( #8252 )
...
* fix: improve error for malformed gap fill query
* fix: code review feedback
2023-07-17 21:20:34 +00:00
b916a89159
fix: recurse through SubqueryAlias when finding gap fill time range ( #8249 )
2023-07-17 19:39:30 +00:00
e33a078128
chore(deps): Bump paste from 1.0.13 to 1.0.14 ( #8244 )
...
Bumps [paste](https://github.com/dtolnay/paste ) from 1.0.13 to 1.0.14.
- [Release notes](https://github.com/dtolnay/paste/releases )
- [Commits](https://github.com/dtolnay/paste/compare/1.0.13...1.0.14 )
---
updated-dependencies:
- dependency-name: paste
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-17 16:10:02 +00:00
2b164c0037
chore: Update DataFusion pin ( #8246 )
2023-07-17 15:53:54 +00:00
9fb689e50d
Merge pull request #8204 from influxdata/jrb_63_compactor_spans
...
chore: add tracing to compactor
2023-07-17 09:58:21 -05:00
85a9e13262
Merge branch 'main' into jrb_63_compactor_spans
2023-07-17 09:52:27 -05:00
7216a40271
Merge pull request #8230 from influxdata/cn/all-over-again
...
feat: Use PartitionHashId, when present, on QueryChunks
2023-07-17 14:52:11 +00:00
ebba032399
Merge branch 'main' into cn/all-over-again
2023-07-17 14:46:48 +00:00
cf046d0b3e
refactor: Extract a from implementation for creating TransitionPartitionId
2023-07-17 10:34:01 -04:00
a9b788b58f
feat: Collate chunks based on their partition hash id if they have it
2023-07-17 10:34:01 -04:00
ae4e7e43c4
Merge pull request #8231 from influxdata/jrb_64_add_test_case
...
fix: compactor loop in highly backlogged case
2023-07-17 09:29:28 -05:00
bba0685c59
Merge remote-tracking branch 'origin/main' into jrb_64_add_test_case
2023-07-17 09:16:13 -05:00
190bc41ca4
chore: comment spelling
2023-07-17 09:14:55 -05:00
70d104b12d
chore(deps): Bump sysinfo from 0.29.4 to 0.29.5 ( #8243 )
...
Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo ) from 0.29.4 to 0.29.5.
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md )
- [Commits](https://github.com/GuillaumeGomez/sysinfo/commits )
---
updated-dependencies:
- dependency-name: sysinfo
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-17 14:08:55 +00:00
4c0e5db3a5
chore(deps): Bump insta from 1.30.0 to 1.31.0 ( #8242 )
...
Bumps [insta](https://github.com/mitsuhiko/insta ) from 1.30.0 to 1.31.0.
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/insta/compare/1.30.0...1.31.0 )
---
updated-dependencies:
- dependency-name: insta
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-17 14:01:21 +00:00
c2606ff3ac
test: Add and use methods creating arbitrary TransitionPartitionId and PartitionHashIds
2023-07-17 09:56:55 -04:00
158c5119d1
fix: Make TransitionPartitionId and PartitionHashId sortable
2023-07-17 09:56:55 -04:00
313baca8b6
fix: Use sort_by rather than sort_by_key to use references
...
These places are sorting by `PartitionId` currently, which implements
`Copy`, but are about to be changed to be sorted on `PartitionHashId`,
which does not implement `Copy`.
2023-07-17 09:56:55 -04:00
745895c643
test: Ensure ingester only sends partition hash id if the partition provider gives it one
...
Also set a partition hash ID on all partitions built in tests with the
`PartitionDataBuilder` by default.
2023-07-17 09:56:55 -04:00
54adecca58
test: Send partition_hash_id to the querier iff PartitionResponse gets a partition hash ID
...
Test to make sure I don't break this behavior before I intend to
2023-07-17 09:54:44 -04:00
9317b36066
refactor: some `cache_system` clean-ups ( #8186 )
...
* refactor: pull out type
* feat: impl `Loader` for `Arc`
* refactor: have a single `TestLoader`
* refactor: simplify code
* refactor: encapsulate `CancellationSafeFutureReceiver`
2023-07-17 12:12:33 +00:00
dependabot[bot]
Marco Neumann
Martin Hilton
Chunchun Ye
Joe-Blount
Joe-Blount
Christopher M. Wolff
wiedld
Andrew Lamb
kodiakhq[bot]
Carol (Nichols || Goulding)