Commit Graph

402 Commits (load-generator/add-new-sequential-data-types)

Author SHA1 Message Date
Jackson Newhouse 956924fa1f Merge remote-tracking branch 'core/main' into jnewhouse/oss-20250320 2025-03-20 09:56:18 -07:00
Jackson Newhouse 7c02593fef
chore: Overhaul CLI tests for better ergonomics. (#26168)
* chore: Overhaul CLI tests for better ergonomics.

* chore: switch with_query() to with_sql().
2025-03-20 09:19:12 -07:00
Praveen Kumar 7c58a38fc1
Merge remote-tracking branch 'OSS/main' into chore/core-sync-20250319 2025-03-19 15:48:57 +00:00
Trevor Hilton d2da058ded
chore: cleanup token create CLI message (#26158) 2025-03-18 10:08:16 -04:00
Trevor Hilton 4cf03bc133 Merge remote-tracking branch 'core/main' into tjh/core-sync-863a6d0b4a 2025-03-17 20:30:52 -04:00
Trevor Hilton 863a6d0b4a
feat: ack catalog update broadcast (#26118)
This creates a CatalogUpdateMessage type that is used to send
CatalogUpdates; this type performs the send on the oneshot Sender so
that the consumer of the message does not need to do so.

Subscribers to the catalog get a CatalogSubscription, which uses the
CatalogUpdateMessage type to ACK the message broadcast from the catalog.

This means that catalog message broadcast can fail, but this commit does
not provide any means of rolling back a catalog update.

A test was added to check that it works.
2025-03-17 20:20:07 -04:00
wayne 96bd6d2b26
Merge pull request #597 from influxdata/catalog/make-trigger-spec-consistent-with-node-spec
chore: clean up TriggerSpecification usage a little
2025-03-17 11:37:22 -06:00
Trevor Hilton 3ad0670452
test: small change to distinct cache cli test (#26152)
Added one check to the distinct cache create test that checks duplicate
creation behaviour while investigating influxdb_pro#601
2025-03-17 11:05:25 -04:00
Jamie Strandboge 95765bf80f
Merge remote-tracking branch 'core/main' into jdstrand/oss-sync 2025-03-16 14:22:26 -05:00
wayne warren 19f7055cda chore: clean up TriggerSpecification usage a little 2025-03-15 17:26:27 -06:00
wayne warren 85d99e90fd chore: be consistent in use of ApiNodeSpec in LVC, DVC, & triggers 2025-03-15 16:50:29 -06:00
Trevor Hilton bae81dd1ee feat: use node spec for processing engine triggers 2025-03-15 13:52:24 -04:00
Trevor Hilton c13b9d34dd
Merge pull request #591 from influxdata/catalog/enable-node-spec-on-distinct-cache-create-and-delete
feat: specify NodeSpec for DistinctCache creation/deletion
2025-03-14 22:17:05 -04:00
Trevor Hilton ca65e0844c
Merge pull request #586 from influxdata/catalog/enable-node-spec-on-last-cache-create-and-delete
feat: specify NodeSpec for LastCache creation/deletion
2025-03-14 22:16:15 -04:00
wayne warren 4acec322c9 fix: don't offer NodeSpec parameter for cache deletion 2025-03-14 19:46:54 -06:00
wayne warren f7da679fa2 fix: incorrect example in docstrings 2025-03-14 19:45:07 -06:00
wayne warren 5fbd1f980f feat: enable NodeSpec to be provided during DistinctCache creation/deletion 2025-03-14 19:45:07 -06:00
wayne warren 08ef6e7c70 fix: don't offer NodeSpec parameter for cache deletion 2025-03-14 19:44:28 -06:00
wayne warren acc0623e7f fix: incorrect example in docstrings 2025-03-14 17:30:55 -06:00
wayne warren 343eeef7b7 refactor: move ApiNodeSpec to influxdb3_catalog crate 2025-03-14 15:56:26 -06:00
wayne warren c0eb48075f refactor: use NodeId instead of Arc<str> for NodeSpec::Nodes 2025-03-14 15:06:36 -06:00
Trevor Hilton bfcacb8c7d feat: make --cluster-id mandatory
This makes the --cluster-id mandatory on Enterprise and requires that
the --cluster-id be different than the --node-id.

This adds a test to check that the two args can't match.
2025-03-14 15:23:47 -04:00
wayne warren ac0d5256a7 chore: call out application to query and process nodes specifically in docstrings 2025-03-14 13:21:25 -06:00
wayne warren c380ef0f08 chore: address rust doc format issue 2025-03-14 13:21:25 -06:00
wayne warren a2614e2d34 feat: allow LastCache nodespec to be provided during deletion 2025-03-14 13:21:23 -06:00
wayne warren 621ca8b73a feat: allow LastCache "NodeSpec" to be specified during creation 2025-03-14 13:14:43 -06:00
Jackson Newhouse e49f0e89fa
fix(processing_engine): Change how temporary plugin files are managed. (#26141) 2025-03-14 10:52:38 -07:00
Trevor Hilton 2886b8dd52 Merge remote-tracking branch 'core/main' into tjh/core-0c8d17fb89 2025-03-14 13:50:22 -04:00
Jackson Newhouse e1015f15f7
feat(processing_engine): proper escaping of LineBuilder arguments. (#26134) 2025-03-14 09:31:58 -07:00
Jackson Newhouse 331f88533c
feat(processing_engine): Attach a node id to the processing engine and triggers for multi-node support compatibility. (#26136) 2025-03-13 16:10:02 -07:00
wayne warren d19333f10a test: add multi-compactor integration test and warning log 2025-03-13 13:47:42 -06:00
wayne warren 82310a7646 refactor: replace --compactor-id with catalog 2025-03-13 10:35:11 -06:00
wayne warren bfe9301216 refactor: replace --compact-from-node-ids with catalog node list 2025-03-12 18:37:28 -06:00
wayne warren 9b3f71748b refactor: get node IDs for replication directly from catalog
fix: need to register ingest nodes in replica test cases
2025-03-12 17:55:24 -06:00
wayne warren e395c3fb07 refactor: replace --read-from-node-ids with catalog node list 2025-03-12 14:02:09 -06:00
Michael Gattozzi 97b8c471f8
fix: Make sure JSON fully closes on empty records (#26130) 2025-03-12 14:01:40 -04:00
Trevor Hilton b6cb6dd51e
chore: back-port catalog debug log cleanup from enterprise (#26128)
* chore: back-port debug log cleanup for catalog

* chore: back-port debug log cleanup for wal

* chore: back-port debug log cleanup for write
2025-03-12 13:20:21 -04:00
Trevor Hilton db4f779554 Merge remote-tracking branch 'core/main' into tjh/core-sync-a455598be6 2025-03-11 19:50:38 -04:00
Jackson Newhouse a455598be6
feat(processing_engine): Implement Processing Engine Cache. (#26111) 2025-03-11 15:48:47 -07:00
Trevor Hilton 503819468e
feat: catalog checkpoints (#26126) 2025-03-11 18:20:36 -04:00
wayne warren 17f4a0a5e3 fix: BufferMode::Persist should imply the node is a querier for now 2025-03-11 13:57:45 -06:00
wayne warren f2e18514a3 chore: cleanup/improvements for BufferModes 2025-03-11 13:41:28 -06:00
wayne warren c36b586ad8 fix: logic bug in setup of compacted data producer 2025-03-11 13:02:03 -06:00
wayne warren b4790ad3da chore: get rid of unnecessary closure 2025-03-11 12:30:29 -06:00
wayne warren a04381257e fix: BufferMode::All causes node to be considered compactor, ingester, or querier 2025-03-11 12:30:29 -06:00
wayne warren 69644ee890 test: fix tests after write buffer mode refactor 2025-03-11 12:29:35 -06:00
wayne warren a6481ca410 refactor: use Vec<NodeMode> in catalog/logs 2025-03-11 12:06:27 -06:00
wayne warren 329fb882b6 refactor: toward simplifying clustered setup by supporting multiple BufferModes 2025-03-11 12:06:27 -06:00
Trevor Hilton 4bdc1c4dd7 Merge remote-tracking branch 'core/main' into tjh/core-sync-72dc4458fd 2025-03-11 12:35:44 -04:00
Trevor Hilton 72dc4458fd
chore: backport changes to catalog from enterprise (#26116)
* chore: backport changes to influxdb3_catalog crate

* chore: backport changes to influxdb3_cache crate

* chore: backport changes to influxdb3_write crate

* chore: backport changes to influxdb3_proc_eng crate

* chore: backport influxdb3 crate changes for catalog

* chore: backport changes to influxdb3_id crate

* chore: backport changes to influxdb3_wal crate

* chore: backport changes to influxdb3_clap_blocks crate

* chore: backport changes to influxdb3_client crate

* chore: backport influxdb3_server crate changes

* chore: fix after full backport

* fix: ordering of catalog broadcast
2025-03-11 12:11:51 -04:00