f3b6c8bf15
chore(deps): Bump rustls from 0.21.3 to 0.21.5 ( #8207 )
...
Bumps [rustls](https://github.com/rustls/rustls ) from 0.21.3 to 0.21.5.
- [Release notes](https://github.com/rustls/rustls/releases )
- [Commits](https://github.com/rustls/rustls/compare/v/0.21.3...v/0.21.5 )
---
updated-dependencies:
- dependency-name: rustls
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-07-11 09:21:38 +00:00
9111cd517f
feat(influxql): PERCENTILE function ( #8187 )
...
* feat(influxql): support TOP and BOTTOM functions
Add support for the TOP and BOTTOM functions which return the first
n rows in some ordered data set.
* fix: clippy
* refactor(influxql): use window aggregates for selectors
Change the implentation of ProjectionType::Selector to use a window
aggregate, rather than an aggregate with a custom selector function.
This is in preparation for implementing PERCENTILE.
* feat(influxql): PERCENTILE selector
Add a selector for the row containing the nth percentile of a
partition. This is the behaviour used when a single selector function
is used in an influxql query.
* feat(influxql): PERCENTILE aggregator
Add the PERCENTILE aggregation function for when the PERCENTILE
function is used in an aggregating projection. This implementation
buffers all non-null field values in memory in order to perform the
operation and therefore could be an expensive operation. This is
necessary for compatibility with earlier influxdb versions.
* refactor(influxql): move PERCENTILE implementation out of plan
The plan module is getting rather full of user-defined function
implementations. This breaks the new functions used to implement
percentile into some new top-level modules for aggregate and window
UDFs.
* fix: doc-lint
* chore: refactor `find_enumerated`
* chore: use `s` in format string
* chore: include the unexpected selector function in the error
* chore(influxql): review suggestions
Added some addition comments to help understanding.
Changed the handling os slector functions such that FIRST, LAST,
MAX & MIN behave the same as they did before PERCENTILE was added.
* chore(influxql): make percent_row_number a window UDF
Now that user-defined window functions are available make the
percent_row_number function be one of those. this allows the values
to be calculated for the entire window partition in one go.
For some reason the user-defined window function cannot return NULL
values. This function uses 0 where it would otherwise use NULL, as
row numbering starts at 1.
---------
Co-authored-by: Stuart Carnie <stuart.carnie@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-11 05:33:16 +00:00
5521310005
Merge pull request #8094 from influxdata/savage/individually-sequence-partitions-within-writes
...
feat(ingester): Assign individual sequence numbers for writes per partition
2023-07-10 14:39:39 +00:00
dec0244bff
refactor(e2e): Wait 100ms between queries in debug::build_catalog test
2023-07-10 15:27:30 +01:00
7e17b54f2a
Merge branch 'main' into savage/individually-sequence-partitions-within-writes
2023-07-10 15:19:45 +01:00
0978aa0551
fix(e2e): Add small busy-loop to debug::build_catalog test to assert only on non-empty results
2023-07-10 15:13:37 +01:00
83febf3eef
Merge pull request #8192 from influxdata/jrb_62_index_partition_tbl_on_createdat
...
chore: create index on created_at in partition table
2023-07-10 08:44:57 -05:00
fdecf96103
Merge branch 'main' into jrb_62_index_partition_tbl_on_createdat
2023-07-10 08:38:12 -05:00
5fa861abab
Merge branch 'main' into savage/individually-sequence-partitions-within-writes
2023-07-10 12:48:37 +00:00
3ce11d8d66
chore: Update DataFusion ( #8190 )
...
* chore: Update DataFusion
* chore: Run cargo hakari tasks
* fix: Update for API changes
* fix: use display format
* chore: Update explain plan output
* fix: update plans
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-10 09:54:50 +00:00
1f1b17f712
Merge pull request #8188 from influxdata/dom/partition-query-concurrency
...
test(bench): concurrent ingester partition queries
2023-07-10 09:41:03 +00:00
19b59d9de5
Merge branch 'main' into dom/partition-query-concurrency
2023-07-10 09:35:22 +00:00
1f82f6b059
chore(deps): Bump snafu from 0.7.4 to 0.7.5 ( #8193 )
...
Bumps [snafu](https://github.com/shepmaster/snafu ) from 0.7.4 to 0.7.5.
- [Changelog](https://github.com/shepmaster/snafu/blob/main/CHANGELOG.md )
- [Commits](https://github.com/shepmaster/snafu/compare/0.7.4...0.7.5 )
---
updated-dependencies:
- dependency-name: snafu
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-10 09:35:10 +00:00
d395a4907a
Merge branch 'main' into dom/partition-query-concurrency
2023-07-10 10:34:43 +01:00
f0789b74ce
chore(deps): Bump regex from 1.9.0 to 1.9.1 ( #8195 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.9.0 to 1.9.1.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.9.0...1.9.1 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-10 09:29:12 +00:00
341dcf2124
Merge branch 'main' into dom/partition-query-concurrency
2023-07-10 10:24:09 +01:00
ab16180f15
chore(deps): Bump serde from 1.0.167 to 1.0.168 ( #8194 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.167 to 1.0.168.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.167...v1.0.168 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-10 09:24:01 +00:00
d87f69a76c
Merge pull request #8197 from influxdata/dependabot/cargo/async-channel-1.9.0
...
chore(deps): Bump async-channel from 1.8.0 to 1.9.0
2023-07-10 10:17:46 +01:00
12317fee23
chore(deps): Bump async-channel from 1.8.0 to 1.9.0
...
Bumps [async-channel](https://github.com/smol-rs/async-channel ) from 1.8.0 to 1.9.0.
- [Release notes](https://github.com/smol-rs/async-channel/releases )
- [Changelog](https://github.com/smol-rs/async-channel/blob/master/CHANGELOG.md )
- [Commits](https://github.com/smol-rs/async-channel/compare/v1.8.0...v1.9.0 )
---
updated-dependencies:
- dependency-name: async-channel
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-10 01:42:26 +00:00
c2442c31f3
chore: create partition table index for created_at
2023-07-07 16:27:05 -05:00
b5faa37152
fix: Plumb tracing header name env/flag to client ( #8189 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-07 21:07:29 +00:00
048fc32bd5
feat: add `influxdb_iox debug build-catalog` command ( #8067 )
...
* feat: add `influxdb_iox debug build-catalog` command
* fix: tests
* fix: Use info! logs instead of println for status
* fix: Set partition_hash_id as well
* fix: remove leftover code
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-07 18:32:27 +00:00
ea38e93511
test(bench): concurrent partition queries
...
Benchmark the performance of concurrent queries against a single
partition, varying the number of concurrent queries and size of buffered
data in the partition.
2023-07-07 16:27:44 +02:00
1736333e32
Merge pull request #8173 from influxdata/jrb_58_throttle_partition_processing
...
feat: throttle partition evaluation in the compactor
2023-07-07 08:52:30 -05:00
ec6a609f63
Merge branch 'main' into jrb_58_throttle_partition_processing
2023-07-07 08:46:22 -05:00
5239540b1d
refactor: replace `println!` w/ `info!` ( #8185 )
...
`println!` is:
- confusing: like why is everything colored but this few lines are
special?!
- annoying: you cannot filter them which esp. for debugging is a pain
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-07 13:05:39 +00:00
115eda07d6
Merge pull request #8070 from influxdata/savage/remove-op-level-sequence-number-for-writes
...
refactor(wal): Remove op-level `sequence_number`, use per table map
2023-07-07 10:17:13 +00:00
e06b6987f0
Merge branch 'main' into savage/remove-op-level-sequence-number-for-writes
2023-07-07 10:12:04 +00:00
18d5b51a78
chore(deps): Bump sysinfo from 0.29.2 to 0.29.4 ( #8183 )
...
Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo ) from 0.29.2 to 0.29.4.
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md )
- [Commits](https://github.com/GuillaumeGomez/sysinfo/commits )
---
updated-dependencies:
- dependency-name: sysinfo
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-07 09:43:37 +00:00
8b000862e1
chore(deps): Bump pretty_assertions from 1.3.0 to 1.4.0 ( #8182 )
...
Bumps [pretty_assertions](https://github.com/rust-pretty-assertions/rust-pretty-assertions ) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/rust-pretty-assertions/rust-pretty-assertions/releases )
- [Changelog](https://github.com/rust-pretty-assertions/rust-pretty-assertions/blob/main/CHANGELOG.md )
- [Commits](https://github.com/rust-pretty-assertions/rust-pretty-assertions/compare/v1.3.0...v1.4.0 )
---
updated-dependencies:
- dependency-name: pretty_assertions
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-07 09:35:18 +00:00
057ee40cb9
chore(deps): Bump thiserror from 1.0.41 to 1.0.43 ( #8181 )
...
Bumps [thiserror](https://github.com/dtolnay/thiserror ) from 1.0.41 to 1.0.43.
- [Release notes](https://github.com/dtolnay/thiserror/releases )
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.41...1.0.43 )
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-07 09:25:12 +00:00
9df1186472
chore(deps): Bump is-terminal from 0.4.8 to 0.4.9 ( #8180 )
...
Bumps [is-terminal](https://github.com/sunfishcode/is-terminal ) from 0.4.8 to 0.4.9.
- [Commits](https://github.com/sunfishcode/is-terminal/compare/v0.4.8...v0.4.9 )
---
updated-dependencies:
- dependency-name: is-terminal
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-07 09:16:43 +00:00
778299112b
chore(deps): Bump serde from 1.0.166 to 1.0.167 ( #8179 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.166 to 1.0.167.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.166...v1.0.167 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-07 09:05:08 +00:00
51a70f0ae9
Merge pull request #8184 from influxdata/sgc/issue/8168_integer_division
...
fix: InfluxQL planner promotes integer division to floats
2023-07-07 04:40:51 +00:00
1ca547b313
fix: Teach planner to rewrite binary expressions for div operator
...
Specifically when the operands are integers, to match InfluxQL OG
2023-07-07 11:22:03 +10:00
05dacf7fe7
fix: Teach type evaluator about division when operands are integers
2023-07-07 11:20:54 +10:00
65f928f227
chore: unwrap option in log ( #8178 )
2023-07-06 23:01:39 +00:00
afb899b204
chore: add logging when compactor fetches partitions ( #8176 )
2023-07-06 21:51:04 +00:00
eb162c775b
feat: log query and namespace on querier erorr ( #8174 )
2023-07-06 19:05:30 +00:00
9f522bfd30
Revert "feat(idpe-17789): scheduler job_status() ( #8121 )" ( #8175 )
...
This reverts commit 5d19fa3635
2023-07-06 18:52:25 +00:00
28eb3dcd92
feat: throttle partition evaluation in the compactor
2023-07-06 11:54:18 -05:00
5d19fa3635
feat(idpe-17789): scheduler job_status() ( #8121 )
...
This block of work moves into the scheduler some of the specific downstream actions affiliated with compaction outcomes. Which responsibilities stay in the compactor, versus moved to the scheduler, roughly followed the heuristic of whether the action (a) had an impact on global catalog state (a.k.a. commits and partition skipping), (b) whether it's logging affiliated with compactor health (e.g. ParitionDoneSink logging outcomes) versus system health (e.g. logging commits), and (c) reporting to the scheduler on any errors encountered during compaction. This boundary is subject to change as we move forward.
Also, a noted caveat (TODO) on this commit. We have a CompactionJob which is used to track work handed off to each compactor. Currently it still uses the partition_id for tracking, but the followup PR will start moving the compactor to have more CompactionJob uuid awareness.
2023-07-06 09:15:59 -07:00
ecc5072197
Merge pull request #8006 from influxdata/7899/wal-disk-metrics
...
feat(7899): wal disk metric
2023-07-06 15:04:06 +01:00
a005f344d8
Merge branch 'main' into 7899/wal-disk-metrics
2023-07-06 14:44:11 +01:00
d46a5f0c51
refactor: update metrics out of phase
...
Emit the metrics out-of-phase with the metric scraper.
2023-07-06 15:36:30 +02:00
d979739576
perf: read disks once, resolve mount point once
...
Instead of refreshing every metric in the System every 10 seconds,
refresh only the disk statistics for the disk we're interested in.
Additionally resolve the parent disk for the directory path once,
instead of each loop.
2023-07-06 15:33:35 +02:00
b2c1532d3a
chore(deps): Bump regex from 1.8.4 to 1.9.0 ( #8160 )
...
* chore(deps): Bump regex from 1.8.4 to 1.9.0
Bumps [regex](https://github.com/rust-lang/regex ) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.8.4...1.9.0 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-07-06 12:48:27 +00:00
bbcf3f0556
chore(deps): Bump toml from 0.7.5 to 0.7.6 ( #8161 )
...
Bumps [toml](https://github.com/toml-rs/toml ) from 0.7.5 to 0.7.6.
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.7.5...toml-v0.7.6 )
---
updated-dependencies:
- dependency-name: toml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
2023-07-06 12:35:47 +00:00
c55367366f
fix: return error rather than panic on unsupported predicate ( #8158 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-07-06 11:29:22 +00:00
a6df3a4b36
chore(deps): Bump clap from 4.3.10 to 4.3.11 ( #8162 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.3.10 to 4.3.11.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.3.10...v4.3.11 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-06 10:15:25 +00:00
dependabot[bot]
Martin Hilton
kodiakhq[bot]
Fraser Savage
Joe-Blount
Andrew Lamb
Dom
Joe-Blount
Marko Mikulicic
Marco Neumann
Stuart Carnie
wiedld