0b9240cbbe
chore(deps): Bump tokio-util from 0.7.7 to 0.7.8 ( #7665 )
...
Bumps [tokio-util](https://github.com/tokio-rs/tokio ) from 0.7.7 to 0.7.8.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.7.7...tokio-util-0.7.8 )
---
updated-dependencies:
- dependency-name: tokio-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-26 09:24:39 +00:00
108e72e3a0
Merge pull request #7668 from influxdata/dependabot/cargo/tracing-0.1.38
...
chore(deps): Bump tracing from 0.1.37 to 0.1.38
2023-04-26 10:17:15 +01:00
d72179fbfc
chore: Run cargo hakari tasks
2023-04-26 02:03:57 +00:00
d36c9dced5
chore(deps): Bump tracing from 0.1.37 to 0.1.38
...
Bumps [tracing](https://github.com/tokio-rs/tracing ) from 0.1.37 to 0.1.38.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.37...tracing-0.1.38 )
---
updated-dependencies:
- dependency-name: tracing
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-04-26 02:02:42 +00:00
eca86a0f06
feat(flightsql): Support `CommandGetXdbcTypeInfo` metadata endpoint with an empty RecordBatch ( #7576 )
...
* feat: WIP support CommandGetXdbcTypeInfo metadata endpoint with tests
* chore: update test case and add jdbc test
* chore: uncomment jdbc getCoumns test
* chore: lint
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-25 21:06:27 +00:00
2341306c26
refactor: Use upstream copy of FlightDataEncoder ( #7656 )
2023-04-25 20:16:47 +00:00
bc324c4ac2
chore: Update DataFusion and arrow/parquet/arrow-flight to `38.0.0` ( #7662 )
...
* chore: Update DataFusion and arrow/parquet/arrow-flight to `38.0.0`
* chore: Run cargo hakari tasks
---------
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2023-04-25 14:29:23 -04:00
523fd0cabf
Merge pull request #7621 from influxdata/idpe-17434/namespace-validation-alignment
...
chore(idpe-17434): remove utf8-percent encoding from onWrite path
2023-04-25 09:52:50 -07:00
3990dcd519
Merge branch 'main' into idpe-17434/namespace-validation-alignment
2023-04-25 09:41:45 -07:00
06fbab2649
fix(garbage collector): limit catalog update for files to delete ( #7623 )
...
* fix(garbage collector): limit catalog update for files to delete
Impose a 1000 LIMIT on flag_for_delete_by_retention so the garbage
collector's load on the catalog is limited. 1000 is used as the fixed
limit in another catalog DML.
* follow up to requests in #7562
* chore: add test for limit on update
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-25 08:57:23 -07:00
159a254ada
fix: Lower message about unsupported predicates on the ingester to debug level ( #7655 )
2023-04-25 10:46:03 -04:00
068da64c65
Merge pull request #7651 from influxdata/cn/partition-data
...
refactor: Extract commonly duplicated test code into helpers
2023-04-25 15:20:26 +01:00
ba5de6b21d
Merge branch 'main' into cn/partition-data
2023-04-25 14:37:22 +01:00
b74a81e792
test: add tests on month and year date_bin ( #7648 )
...
* test: add tests on month and year date_bin
* fix: add IOX_COMPARE: uuid to get deterministics name for output parquet_file in the explain
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-25 10:16:21 +00:00
740269839d
fix: Extract test helper functions
2023-04-24 20:14:59 -04:00
148c103c70
refactor: push authz into specific configs ( #7607 )
...
* refactor: move authz-addr flag into router-specific config
* refactor: move authz-addr flag into querier-specific config
* refactor: remove global AuthzConfig which is now redundant with the pushdown to individual configs. Keep constant the env vars used universally.
* chore: make errors lowercase, and use the required bool for the authz-addr flag
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-25 00:14:28 +00:00
2aaa431d5a
fix: You can't just call everything bananas, Dom
...
Remove the MockNamespaceNameProvider::default implementation that
hardcodes all namespace names to "bananas"; I just tore some of my hair
out trying to figure out why I was getting bananas WHEN THERE SHOULD
HAVE BEEN NO BANANAS
2023-04-24 20:13:54 -04:00
38262f37cb
refactor: Take an Arc as an argument to be more consistent with similar types
...
NamespaceData::new doesn't take an Arc of the namespace name loader, but
PartitionData::new does. If they both take Arcs, then making test
constants and helpers they can share is easier. Same deal for
TableData::new.
2023-04-24 20:13:54 -04:00
b3cdea25ba
chore: update term conventions to allowlist
2023-04-24 17:07:25 -07:00
06bc2d6272
feat: Add `pre` and `post` visit events for `Literal` ( #7629 )
2023-04-24 21:15:44 +00:00
ebf0b34523
feat: add compactor concurrency within single partition's plans ( #7642 )
...
* feat: trying to get concurrency within partition compaction
* feat: Create a stream then map and try collect
* feat: set compactor concurrency within single partition's plans
* chore: add log about plan concurrency
---------
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
2023-04-24 20:48:35 +00:00
446b55d5b5
Merge pull request #7639 from influxdata/cn/dml-meta
...
fix: In DmlMeta, only use SequenceNumber rather than Sequence (shard index plus seq num)
2023-04-24 19:12:53 +00:00
77c1311c4d
Merge branch 'main' into cn/dml-meta
2023-04-24 19:05:46 +00:00
fe5fa3b3e8
fix: don't skip failed rules during logical planning ( #7641 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-24 18:15:08 +00:00
2d592a0295
feat: teach parser to allow arithmetic functions ( #7620 )
...
* feat: teach parser to allow arithmetic functions
* refactor: remove unnecessary lowercasing of fn names
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-24 17:09:22 +00:00
8e7e22d167
chore(garbage collector): info logging for deleting parquet file ( #7638 )
...
* for influxdata/idpe#17451
* follow up to #7562
2023-04-24 16:57:18 +00:00
7b7101b53f
fix: In DmlMeta, only use SequenceNumber rather than Sequence (shard index plus seq num)
2023-04-24 12:23:16 -04:00
8b0f7b6c64
Merge pull request #7635 from influxdata/dom/wal-spawn-timeout
...
feat(ingester): blocked write load shedding
2023-04-24 14:55:25 +00:00
b4b299f88d
Merge branch 'main' into dom/wal-spawn-timeout
2023-04-24 15:49:04 +01:00
96916d121c
Merge pull request #7543 from influxdata/cn/remove-rpc-mode-env-var
...
fix: Remove uses and tests for RPC mode env var
2023-04-24 14:23:53 +00:00
a8214a4474
Merge branch 'main' into dom/wal-spawn-timeout
2023-04-24 15:16:07 +01:00
53196870a5
test: Remove shard variation from more tests
2023-04-24 10:08:01 -04:00
cf051e8091
fix: Remove iox catalog shard tests
2023-04-24 10:08:00 -04:00
8f17d217fb
fix: Move an error condition earlier, before we try to do catalog work
...
If the command is going to fail anyway, may as well stop earlier
2023-04-24 10:08:00 -04:00
24e6248706
fix: Update a compactor snapshot
2023-04-24 10:08:00 -04:00
8d4c2bfabb
fix: Only ever create the transition shard in the in-memory catalog
...
Tests that use the in-memory catalog are creating different shards that
then creates old-style Parquet file paths, but in production, everything
uses the transition shard now. To make the tests more like production,
only ever create and use the transition shard, and stop checking for
different shard IDs.
2023-04-24 10:08:00 -04:00
584d3ab0e7
fix: Set up the transition shard in the in-mem catalog
2023-04-24 10:08:00 -04:00
5e01261a87
fix: Change the normalization regex to new path format
2023-04-24 10:08:00 -04:00
a9081fc8a8
fix: Remove uses and tests for RPC write env var
2023-04-24 10:00:09 -04:00
9730c52c8a
fix: InfluxQL output sorting ( #7633 )
...
* test: remove #7513 workarounds from `issue_6112.influxql`
* fix: `SHOW TAG VALUES` output sorting
* fix: `SHOW MEASUREMENTS` output sorting
* fix: `SHOW TAG KEYS` output sorting
2023-04-24 13:37:25 +00:00
2769b52321
feat(ingester): blocked write load shedding
...
This commit bounds the duration of time a write may take to be processed
after being added to the WAL for increased reliability.
In ideal cases, anything that is added to the WAL is buffered /
completed to prevent WAL replay materialising writes that never
completed in the first place (#7111 ), but in some cases (i.e. catalog
outage, etc) the write becomes blocked, stuck in a retry loop or
otherwise never making progress.
When a write is blocked, the data within it remains in RAM, and the
overhead of the spawned task retrying also consumes CPU resources. If
all writes are blocked for sufficient time, these limitless writes keep
building up until a resource is exhausted (i.e. RAM -> OOM) causing an
outage of the ingester.
Instead, this change allocates a write at most 15 seconds to complete,
before it is cancelled and an error returned to the user (if they're
still connected) to shed load. The error message "buffer apply request
timeout" was chosen to be uniquely identifying - systems have lots of
timeouts!
Normally the inner.apply() call at this point completes in microseconds,
so 15 seconds is more than long enough to avoid shedding legitimate
load.
2023-04-24 13:53:40 +02:00
f411f5d6d5
chore: update DF to `181e5ccf2816ccaa05d8aaef0b375d4b7bbceece` ( #7630 )
...
I need the following two PRs:
- https://github.com/apache/arrow-datafusion/pull/6045
- https://github.com/apache/arrow-datafusion/pull/6085
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-04-24 09:46:54 +00:00
b7bb833af2
Merge pull request #7631 from influxdata/dependabot/cargo/regex-syntax-0.7.1
...
chore(deps): Bump regex-syntax from 0.6.29 to 0.7.1
2023-04-24 10:35:46 +01:00
342795c006
chore(deps): Bump regex-syntax from 0.6.29 to 0.7.1
...
Bumps [regex-syntax](https://github.com/rust-lang/regex ) from 0.6.29 to 0.7.1.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/commits )
---
updated-dependencies:
- dependency-name: regex-syntax
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-04-24 09:28:49 +00:00
526a517b71
Merge pull request #7626 from influxdata/dependabot/cargo/tracing-subscriber-0.3.17
...
chore(deps): Bump tracing-subscriber from 0.3.16 to 0.3.17
2023-04-24 10:25:58 +01:00
2d1902610f
chore(deps): Bump tracing-subscriber from 0.3.16 to 0.3.17
...
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing ) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.16...tracing-subscriber-0.3.17 )
---
updated-dependencies:
- dependency-name: tracing-subscriber
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-04-24 08:43:22 +00:00
d8ffb8a4df
chore(deps): Bump rustix from 0.37.13 to 0.37.14 ( #7624 )
...
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.37.13 to 0.37.14.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.37.13...v0.37.14 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 08:40:38 +00:00
850d7f7011
chore(deps): Bump regex from 1.8.0 to 1.8.1 ( #7627 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.8.0 to 1.8.1.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/commits/1.8.1 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 08:25:26 +00:00
5c2aaeeef9
chore(deps): Bump ordered-float from 3.6.0 to 3.7.0 ( #7625 )
...
Bumps [ordered-float](https://github.com/reem/rust-ordered-float ) from 3.6.0 to 3.7.0.
- [Release notes](https://github.com/reem/rust-ordered-float/releases )
- [Commits](https://github.com/reem/rust-ordered-float/compare/v3.6.0...v3.7.0 )
---
updated-dependencies:
- dependency-name: ordered-float
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 08:13:09 +00:00
daabe9663c
chore(idpe-17434): make restrictive whitelist of chars accepted, for any NamespaceName
2023-04-21 16:36:00 -07:00
dependabot[bot]
Dom
CircleCI[bot]
Chunchun Ye
Andrew Lamb
wiedld
Phil Bracikowski
Nga Tran
Carol (Nichols || Goulding)
wiedld
Stuart Carnie
Joe-Blount
kodiakhq[bot]
Christopher M. Wolff
Marco Neumann