0e478af470
chore(deps): Bump tempfile from 3.7.0 to 3.7.1 ( #8429 )
...
Bumps [tempfile](https://github.com/Stebalien/tempfile ) from 3.7.0 to 3.7.1.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.7.0...v3.7.1 )
---
updated-dependencies:
- dependency-name: tempfile
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-07 09:21:11 +00:00
b62e9bb0c8
chore(deps): Bump pin-project from 1.1.2 to 1.1.3 ( #8427 )
...
Bumps [pin-project](https://github.com/taiki-e/pin-project ) from 1.1.2 to 1.1.3.
- [Release notes](https://github.com/taiki-e/pin-project/releases )
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md )
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.1.2...v1.1.3 )
---
updated-dependencies:
- dependency-name: pin-project
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-07 09:09:23 +00:00
e69b2c40c6
chore(deps): Bump regex from 1.9.1 to 1.9.3 ( #8426 )
...
Bumps [regex](https://github.com/rust-lang/regex ) from 1.9.1 to 1.9.3.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.9.1...1.9.3 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-07 09:01:07 +00:00
ae6d416ad4
chore(deps): Bump filetime from 0.2.21 to 0.2.22 ( #8425 )
...
Bumps [filetime](https://github.com/alexcrichton/filetime ) from 0.2.21 to 0.2.22.
- [Commits](https://github.com/alexcrichton/filetime/compare/0.2.21...0.2.22 )
---
updated-dependencies:
- dependency-name: filetime
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-07 08:18:54 +00:00
758c42305d
chore(deps): Bump regex-automata from 0.3.4 to 0.3.6 ( #8424 )
...
Bumps [regex-automata](https://github.com/rust-lang/regex ) from 0.3.4 to 0.3.6.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/regex-automata-0.3.4...regex-automata-0.3.6 )
---
updated-dependencies:
- dependency-name: regex-automata
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-07 08:05:01 +00:00
922a3cd292
feat: harden migration version checks ( #8415 )
...
* refactor: clean up
* feat: harden migration version checks
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-07 07:59:24 +00:00
36276b962e
fix: change max files per plan to something reasonable ( #8418 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-04 15:14:57 +00:00
614b64d3e4
feat: retry failed migrations ( #8414 )
2023-08-04 11:15:05 +00:00
9803623543
feat: post-migration sanity checks ( #8407 )
2023-08-04 08:56:22 +00:00
975d9a1ae5
refactor: simplify code, harden PG locking ( #8406 )
...
* test: simplify code
* feat: `IOxMigration::single_transaction`
* refactor: unlock faster
* refactor: harden PG locking
Check that `unlock` is actually successful and add a bunch of tests.
2023-08-04 08:39:01 +00:00
5530d90074
chore(deps): Bump serde from 1.0.180 to 1.0.181 ( #8413 )
...
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.180 to 1.0.181.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.180...v1.0.181 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-04 08:11:16 +00:00
0c9570889b
feat: allow fixing migrations ( #8394 )
...
We need to fix our non-idempotent `CREATE INDEX CONCURRENTLY` migrations
so they are self-healing. This is an essential part of #7897 .
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-04 08:05:45 +00:00
b7c0bcb61f
chore(compactor): move divide & branches to RoundInfo calculation ( #8410 )
2023-08-03 22:02:06 +00:00
81b5d80a91
feat(idpe-17935): move filtering of skipped partitions to the scheduler ( #8358 )
...
* catalog.get_in_skipped_compaction() should handle for multiple partitions
* add the ability to perform transformation on sets of partitions (rather than filtering one by one). Start with the transformation to remove skipped partitions, in the scheduler.
* move the env var and cli flag setting, for when to ignore skipped partitions, to the scheduler config.
2023-08-03 11:43:09 -07:00
1138a5907c
Merge pull request #8409 from influxdata/dom/gossip
...
feat(router): intra-cluster schema diff gossiping
2023-08-03 17:34:32 +01:00
986e46e526
Merge branch 'main' into dom/gossip
2023-08-03 17:04:02 +01:00
a017d1d7f9
test: simplify integration test setup
...
Remove the redundant async mutex (previously required, but I refactored
the code to make it unnecessary) and DRY the node setup.
2023-08-03 18:02:33 +02:00
08deb59c62
Merge pull request #8408 from influxdata/dom/partition-bloom
...
perf(ingester): catalog bloom filter for partition IDs
2023-08-03 16:48:23 +01:00
4e97eef7a9
docs: fix typos
...
I see why Carol edits THE BOOK.
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2023-08-03 16:40:09 +01:00
0bc4f5c7b8
Merge branch 'main' into dom/partition-bloom
2023-08-03 16:23:44 +01:00
b02e1941e1
chore: remove prop failure
...
This was a bad test, so isn't meaningful.
2023-08-03 17:20:20 +02:00
757ecc1d03
perf(router): schema gossip between peers
...
This commit allows schema gossiping to be enabled on router nodes.
Enabling gossiping allows any schema changes made on router A to be sent
to the N-1 other routers, populating their internal caches in
anticipation of handling a similar request.
By populating their cache, they avoid incurring a catalog lookup to
populate their local state upon a cache miss, therefore reducing request
latency, and reducing catalog load.
Enabling gossip on the routers automatically enables schema gossiping -
enabling gossip remains optional, and off by default.
2023-08-03 17:10:17 +02:00
8928c838a8
test: schema gossip w/ default partition keys
...
Ensure gossiping namespace & tables with empty partition keys is
correct.
2023-08-03 17:10:16 +02:00
00542f7041
test: schema gossip integration
...
Adds an integration test ensuring the schema gossip layer added to one
instance ("node A") propagates schema diffs to another ("node B").
2023-08-03 17:10:16 +02:00
16c115d5cb
docs(router): gossip subsystem types / topology
...
Describes the router's schema gossiping types and how they fit together.
2023-08-03 17:10:15 +02:00
3133318e16
refactor: remove redundant NamespaceCache impl
...
The NamespaceCache does not need to be a decorator itself - it can
operate using a reference to the cache without needing access to cache
requests.
2023-08-03 17:10:14 +02:00
01cc1c7dfa
Merge pull request #8405 from influxdata/dom/gossip-tx
...
feat: send-side schema gossip implementation
2023-08-03 16:05:55 +01:00
a23d17f98d
test: fix cache hit test
...
The filter shouldn't be given a new-style partition.
2023-08-03 17:04:38 +02:00
b1cdb928f6
refactor: always log error message
...
Always log the actual error as it may change.
2023-08-03 16:59:06 +02:00
fc903b8102
test: preserve duplicates in column set assertions
...
Don't collect into a BTreeSet for sorting as it drops duplicates.
2023-08-03 16:56:56 +02:00
5e00650413
chore: Run cargo hakari tasks
2023-08-03 14:44:06 +00:00
c77299d619
build: disable default proptest features
2023-08-03 16:42:57 +02:00
5c14781264
chore: regenerate workspace-hack
2023-08-03 16:40:55 +02:00
03f7025211
perf: minimise partition catalog queries
...
This commit implements a PartitionProvider decorator that
probabilistically determines if a partition is going to be a an
"old-style" row-addressed partition created prior to #7963 , or a
"new-style" hash-addressed partition created after using a fast,
space-efficient, compressed bloom filter.
If a partition is identified as a new-style, hash-addressed partition,
the PartitionData is immediately initialised using the deterministic
hash ID without performing a catalog query at all.
If a partition is identified as an old-style, row-addressed partition, a
catalog query is performed to resolve the row ID as it would without
this filter.
A new-style, hash-addressed partition may sometimes be incorrectly
identified as a row-addressed partition, causing a spurious catalog
query, which is then correctly identified as a hash-addressed partition.
This is tuned to happen ~1-0.1% of the time, eliminating 99% to 99.9% of
unnecessary catalog queries.
2023-08-03 16:40:38 +02:00
7a4ed257a2
feat: send-side schema gossip implementation
...
This commit adds the SchemaChangeObserver, the delegate which is handed
a schema diff, and is responsible for computing the gossip message and
handing it off to the gossip system.
This sits between the cache layer, and the gossip layer, converting
schema things into gossip things.
This isn't connected up, so no messages will be sent.
2023-08-03 12:42:16 +02:00
b386f68c11
chore(deps): Bump rustls from 0.21.5 to 0.21.6 ( #8401 )
...
Bumps [rustls](https://github.com/rustls/rustls ) from 0.21.5 to 0.21.6.
- [Release notes](https://github.com/rustls/rustls/releases )
- [Commits](https://github.com/rustls/rustls/compare/v/0.21.5...v/0.21.6 )
---
updated-dependencies:
- dependency-name: rustls
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-03 09:40:49 +00:00
c159e99144
chore(deps): Bump regex-automata from 0.3.3 to 0.3.4 ( #8403 )
...
Bumps [regex-automata](https://github.com/rust-lang/regex ) from 0.3.3 to 0.3.4.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/regex-automata-0.3.3...regex-automata-0.3.4 )
---
updated-dependencies:
- dependency-name: regex-automata
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-03 09:35:19 +00:00
4469b16d1f
chore(deps): Bump rustix from 0.38.4 to 0.38.6 ( #8402 )
...
Bumps [rustix](https://github.com/bytecodealliance/rustix ) from 0.38.4 to 0.38.6.
- [Release notes](https://github.com/bytecodealliance/rustix/releases )
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.38.4...v0.38.6 )
---
updated-dependencies:
- dependency-name: rustix
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-03 09:30:06 +00:00
58f1d2431c
Merge pull request #8404 from influxdata/dependabot/cargo/cc-1.0.81
...
chore(deps): Bump cc from 1.0.79 to 1.0.81
2023-08-03 10:24:24 +01:00
f674832a32
chore(deps): Bump cc from 1.0.79 to 1.0.81
...
Bumps [cc](https://github.com/rust-lang/cc-rs ) from 1.0.79 to 1.0.81.
- [Release notes](https://github.com/rust-lang/cc-rs/releases )
- [Commits](https://github.com/rust-lang/cc-rs/compare/1.0.79...1.0.81 )
---
updated-dependencies:
- dependency-name: cc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-08-03 01:48:36 +00:00
a79cddf942
chore: remove dead code compute_sort_key_for_chunks ( #8399 )
2023-08-02 20:45:38 +00:00
28816dfabd
chore(doc): fix typo ( #8398 )
2023-08-02 20:33:32 +00:00
3d6264e21a
Merge pull request #8369 from influxdata/idpe-17789/rename-mod
...
refactor(idpe-17789): rename mod to compaction_job_done_sink
2023-08-02 11:31:04 -07:00
09ee60e35f
refactor(idpe-17789): rename mod to compaction_job_done_sink
2023-08-02 11:21:22 -07:00
68ab2c97c1
feat(idpe 17789): provide job to CompactionJobDoneSink (formerly known as PartitionDoneSink) ( #8368 )
...
* rename PartitionDoneSink to CompactionJobSink. and change signature in trait
* update all trait implementations, including local variables and comments
* rename partition_done_sink in the components and driver, to be compaction_job_done_sink
2023-08-02 11:19:50 -07:00
e45c9b6f61
fix(idpe-17943): spans in futures stream need parent ( #8364 )
2023-08-02 09:49:53 -07:00
21a619651f
Merge pull request #8393 from influxdata/dom/gossip-namespace-cache
...
feat: NamespaceSchemaGossip cache decorator
2023-08-02 16:58:54 +01:00
a32c3d0fa8
Merge branch 'main' into dom/gossip-namespace-cache
2023-08-02 16:39:32 +01:00
ed392cfa37
Merge pull request #8295 from influxdata/cn/new-new
...
feat: Only send the catalog partition ID if the partition doesn't have a hash ID in the database
2023-08-02 15:20:24 +00:00
daeee55416
Merge branch 'main' into cn/new-new
2023-08-02 15:15:25 +00:00
dependabot[bot]
Marco Neumann
Andrew Lamb
Joe-Blount
wiedld
Dom
CircleCI[bot]
Nga Tran
Chunchun Ye
wiedld
kodiakhq[bot]