5735aae998
feat(cli): Add namespace delete command
...
This adds the CLI action to call the namespace soft-deletion
gRPC endpoint and an associated e2e CLI test.
2023-03-23 11:23:00 +00:00
1c83b746c2
chore(deps): Bump rustix from 0.36.11 to 0.37.3 ( #7308 )
...
* chore(deps): Bump rustix from 0.36.11 to 0.37.3
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.11 to 0.37.3.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.11...v0.37.3 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-23 10:43:11 +00:00
3955b89e5f
Merge branch 'main' into savage/add-hot-partition-persistence-metric
2023-03-23 10:36:51 +00:00
4a074d48f2
Merge pull request #7309 from influxdata/dependabot/cargo/dotenvy-0.15.7
...
chore(deps): Bump dotenvy from 0.15.6 to 0.15.7
2023-03-23 10:33:22 +00:00
1ff3d8f8e4
Merge pull request #7310 from influxdata/dependabot/cargo/bitflags-2.0.2
...
chore(deps): Bump bitflags from 1.3.2 to 2.0.2
2023-03-23 10:32:59 +00:00
19021ec3f3
refactor: disallow empty UpstreamSnapshot
...
This simplifies the semantics of upstream RPC endpoint snapshots; an
empty snapshot will never be returned, and therefore if the caller has a
snapshot, it contains at least one endpoint to attempt an RPC call
against.
This moves the "no upstream" error handling from a write-time error, to
a pre-write error.
2023-03-23 11:31:20 +01:00
d4486bef3e
refactor: access upstream count in UpstreamSnapshot
...
Adds a method to return the number of upstreams in an UpstreamSnapshot.
2023-03-23 11:31:20 +01:00
2fea22587a
Merge branch 'main' into savage/add-hot-partition-persistence-metric
2023-03-23 10:29:58 +00:00
76f23bd768
chore: Run cargo hakari tasks
2023-03-23 02:04:56 +00:00
793dc28cf8
chore(deps): Bump bitflags from 1.3.2 to 2.0.2
...
Bumps [bitflags](https://github.com/bitflags/bitflags ) from 1.3.2 to 2.0.2.
- [Release notes](https://github.com/bitflags/bitflags/releases )
- [Changelog](https://github.com/bitflags/bitflags/blob/main/CHANGELOG.md )
- [Commits](https://github.com/bitflags/bitflags/compare/1.3.2...2.0.2 )
---
updated-dependencies:
- dependency-name: bitflags
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-23 02:03:10 +00:00
3a473d2374
chore(deps): Bump dotenvy from 0.15.6 to 0.15.7
...
Bumps [dotenvy](https://github.com/allan2/dotenvy ) from 0.15.6 to 0.15.7.
- [Release notes](https://github.com/allan2/dotenvy/releases )
- [Changelog](https://github.com/allan2/dotenvy/blob/master/CHANGELOG.md )
- [Commits](https://github.com/allan2/dotenvy/compare/v0.15.6...v0.15.7 )
---
updated-dependencies:
- dependency-name: dotenvy
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-23 02:02:33 +00:00
08ef689d21
feat: Teach InfluxQL how to plan an aggregate query ( #7230 )
...
* feat: Display failed query
Allows a user to immediately identify the failed query.
* feat: API improvements to InfluxQL parser
* feat: Extend `SchemaProvider` trait to query for UDFs
* fix: We don't want the parser to panic on overflows
* fix: ensure `map_type` maps the timestamp data type
* feat: API to map a InfluxQL duration expression to a DataFusion interval
* chore: Copied APIs from DataFusion SQL planner
These APIs are private but useful for InfluxQL planning.
* feat: Initial aggregate query support
* feat: Add an API to fetch a field by name
* chore: Fixes to handling NULLs in aggregates
* chore: Add ability to test expected failures for InfluxQL
* chore: appease rustfmt and clippy 😬
* chore: produce same error as InfluxQL
* chore: appease clippy
* chore: Improve docs
* chore: Simplify aggregate and raw planning
* feat: Add support for GROUP BY TIME(stride, offset)
* chore: Update docs
* chore: remove redundant `is_empty` check
Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>
* chore: PR feedback to clarify purpose of function
* chore: The series_sort can't be empty, as `time` is always added
This was originally intended as an optimisation when executing an
aggregate query that did not group by time or tags, as it will produce
N rows, where N is the number of measurements queried.
* chore: update comment for clarity
---------
Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>
2023-03-23 01:13:15 +00:00
77948e3341
chore(iox/compactor): add test cases tracking max_l0_created_at ( #7181 )
...
* chore(iox/compactor): add test cases tracking max_l0_created_at
* chore(iox/compactor): add invariant check to simulator for misordered max_l0_created_at
* chore(iox/compactor): test updates for max_l0_created_at
This commit updates the newly added tests to comply with the new invariant check.
Several test cases are deleted because the changes to comply with the invariant make the permutations pointless.
* chore(iox/simulator): Add test cases hitting new invariant check for illegal max_l0_created_at
* chore: update previous tests to comply with new invariant
* chore: address comments
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 22:14:06 +00:00
2b8db8d645
refactor: Move decision about what plan to create into IRPlanner
2023-03-22 17:03:04 -04:00
4c551a3ce9
refactor: Move uuid merging further into plan building
2023-03-22 16:53:28 -04:00
626dd236f9
refactor: Have run_plans take ownership of split_or_compact
...
Making this clone more explicit, higher up. Deleting the old files from
the catalog might only need the IDs, and the compaction levels for the
metrics, but that can be later.
2023-03-22 16:45:04 -04:00
a58ef38bb0
refactor: Move target_level into PlanIR
...
Because this info needs to stay with the rest of the plan info
2023-03-22 16:15:16 -04:00
87708dc64a
feat(flightsql): Add filters to `GetTables` metadata endpoint ( #7289 )
...
* feat: add optional param to GetTables
* chore: add the third param to query plan
* feat: add table_types param
* chore: clippy
* test: add test cases with filters
* chore: update query to avoid SQL injection
* refactor: update where clause and cleanup
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 20:14:01 +00:00
a20c5dcd8b
refactor: Move the logic for getting the input paths into FilesToSplitOrCompact
2023-03-22 15:51:43 -04:00
e2dcf0b5e9
refactor: Share execute_plan calls between compact and split paths
2023-03-22 15:17:40 -04:00
3235be572d
refactor: Inline run_split_plan in anticipation of combining more code
2023-03-22 15:08:16 -04:00
91ac9627fa
refactor: Inline run_compaction_plan in anticipation of combining code with run_split_plan
2023-03-22 14:44:01 -04:00
7100be4103
refactor: Inline run_split_plans in anticipation of combining with run_compaction_plan
...
Also set a capacity on the Vec to avoid some reallocations.
2023-03-22 14:18:44 -04:00
45a32f80f9
refactor: Move the decision for making progress into the filter
...
And define the conditions for that decision on a type responsible for
the data that goes into that condition.
Also start moving away from methods that only work on one variant of
FilesToSplitOrCompact, and add a state for when FilesToSplitOrCompact
doesn't actually contain any files.
2023-03-22 13:24:09 -04:00
c0b99011c8
refactor: Make some names more consistent and less redundant
...
I can't ever remember whether it's "compact or split" or "split or
compact", so now I think it's always "split or compact".
Also remove "FilesTo" from the enum variants because "FilesTo" is in the
overall enum name.
2023-03-22 13:06:11 -04:00
a3fa986f6e
refactor: Give post-classification partition filter a separate trait
...
I want to change the types, but for now, this is mostly a copy of the
partition filter trait and setting up the associated
logging/metrics/mock.
2023-03-22 13:06:11 -04:00
63bb4db6e9
refactor: Rename unable_to_compact to possible_progress
2023-03-22 13:06:11 -04:00
b78b3d3ee2
refactor: Rename UnableToCompactPartitionFilter to PossibleProgressFilter
...
I'm going to move all the decisions about whether there's progress to be
made or not into here.
2023-03-22 13:06:11 -04:00
353eda377e
refactor: Reduce complexity where there's only one filter remaining
...
The vec and the AndFilter aren't needed because there's only one filter
happening in this spot now.
2023-03-22 13:06:11 -04:00
50d9d40322
refactor: Rename a filter slot based on when it's applied
...
The filter in this spot could potentially do whatever, but the important
part of what should go in this field is that it will be called on the
files after they're classified by the file classifier.
2023-03-22 13:06:11 -04:00
3681003f44
refactor: Destructure FileClassification so the parts can be moved different places
2023-03-22 13:06:10 -04:00
6172e6c513
chore(deps): Bump regex from 1.7.1 to 1.7.2 ( #7299 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.7.1 to 1.7.2.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.7.1...1.7.2 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-22 14:45:20 +00:00
9a2fdb7bf6
chore(deps): Bump bitflags from 1.3.2 to 2.0.2 ( #7296 )
...
* chore(deps): Bump bitflags from 1.3.2 to 2.0.2
Bumps [bitflags](https://github.com/bitflags/bitflags ) from 1.3.2 to 2.0.2.
- [Release notes](https://github.com/bitflags/bitflags/releases )
- [Changelog](https://github.com/bitflags/bitflags/blob/main/CHANGELOG.md )
- [Commits](https://github.com/bitflags/bitflags/compare/1.3.2...2.0.2 )
---
updated-dependencies:
- dependency-name: bitflags
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 14:44:57 +00:00
c35e5973d3
chore(deps): Bump rustix from 0.36.11 to 0.37.3 ( #7298 )
...
* chore(deps): Bump rustix from 0.36.11 to 0.37.3
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.36.11 to 0.37.3.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.36.11...v0.37.3 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 14:34:42 +00:00
c8f90b1cdc
chore(deps): Bump regex-syntax from 0.6.28 to 0.6.29 ( #7300 )
...
Bumps [regex-syntax](https://github.com/rust-lang/regex ) from 0.6.28 to 0.6.29.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/commits )
---
updated-dependencies:
- dependency-name: regex-syntax
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 14:26:10 +00:00
2ba3d13ad8
chore(deps): Bump sqlx-core from 0.6.2 to 0.6.3 ( #7295 )
...
Bumps [sqlx-core](https://github.com/launchbadge/sqlx ) from 0.6.2 to 0.6.3.
- [Release notes](https://github.com/launchbadge/sqlx/releases )
- [Changelog](https://github.com/launchbadge/sqlx/blob/v0.6.3/CHANGELOG.md )
- [Commits](https://github.com/launchbadge/sqlx/compare/v0.6.2...v0.6.3 )
---
updated-dependencies:
- dependency-name: sqlx-core
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 14:18:26 +00:00
187fe85dcd
chore(deps): Bump sqlx from 0.6.2 to 0.6.3 ( #7294 )
...
Bumps [sqlx](https://github.com/launchbadge/sqlx ) from 0.6.2 to 0.6.3.
- [Release notes](https://github.com/launchbadge/sqlx/releases )
- [Changelog](https://github.com/launchbadge/sqlx/blob/v0.6.3/CHANGELOG.md )
- [Commits](https://github.com/launchbadge/sqlx/compare/v0.6.2...v0.6.3 )
---
updated-dependencies:
- dependency-name: sqlx
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 14:06:13 +00:00
7f7d0b8d52
fix(flightsql): Send correct schema in GetFlightInfo for Queries ( #7283 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 13:58:44 +00:00
222d765323
chore(compactor/simulator): add data backfill test cases ( #7293 )
...
* chore(compactor/simulator): add data backfill test cases
* chore: remove extra comment (left from copy & paste)
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-22 13:51:55 +00:00
90b52d2332
Merge pull request #7270 from influxdata/cn/shard-tests
...
test: Add compactor shard e2e tests
2023-03-22 13:44:34 +00:00
18405a5919
Merge branch 'main' into cn/shard-tests
2023-03-22 13:36:27 +00:00
3fbab1a335
feat(persist): Add hot partition persistence counter metric
...
This counter tracks how many times each partition has been
enqueued for persistence as a result of hitting the pre-configured
limit.
2023-03-22 11:41:31 +00:00
8518ab502b
Merge pull request #7301 from influxdata/auth-header
...
fix: Put authorization header in request extension
2023-03-22 04:27:09 +01:00
6660b86f6c
fix: Put authorization header in request extension
2023-03-22 04:12:07 +01:00
7ee40acb22
test: set max_l0_created_at appropriately in the core tests ( #7292 )
...
* test: make max_l0_created_at more sense in the core tests
* chore: fix typo
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-21 20:40:09 +00:00
777128f2b7
feat(flightsql): Avoid planning Flight queries twice ( #7281 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-21 20:15:53 +00:00
f780aba353
test: set max_l0_created_at to reasonable values for the tests and al… ( #7286 )
...
* test: set max_l0_created_at to reasonable values for the tests and also verify it using both test layout and catalog function
* fix: typo
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-21 18:57:10 +00:00
dd63b5c52d
chore: expose partition_timeout in the compactor simulator ( #7291 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-21 18:48:43 +00:00
dd1906a99e
Merge pull request #7284 from influxdata/dom/graceful-shutdown
...
test(ingester): graceful shutdown
2023-03-21 18:41:21 +00:00
901743b48e
Merge branch 'main' into dom/graceful-shutdown
2023-03-21 18:33:46 +00:00
Fraser Savage
dependabot[bot]
Dom
CircleCI[bot]
Stuart Carnie
Joe-Blount
Carol (Nichols || Goulding)
Chunchun Ye
Andrew Lamb
kodiakhq[bot]
Marko Mikulicic
Nga Tran