Commit Graph

6376 Commits (3567fa164daa0652dcc9435ab4d4b71a1cc2c441)

Author SHA1 Message Date
kodiakhq[bot] 3567fa164d
Merge pull request #3352 from influxdata/pd/add-data-generator-schemas
chore: add two test schemas in data generator examples
2021-12-13 15:17:30 +00:00
kodiakhq[bot] 967a41f7a4
Merge branch 'main' into pd/add-data-generator-schemas 2021-12-13 15:07:06 +00:00
kodiakhq[bot] 7747eb6dd0
Merge pull request #3360 from influxdata/pd/add-total-throughput
feat: add total throughput to data generator output
2021-12-13 15:02:10 +00:00
Paul Dix 8efd02930e feat: add total throughput to data generator output
On log level info, data generator will now output what the total throughput is in rows per second after every agent's successful write.
2021-12-11 17:12:28 -05:00
kodiakhq[bot] fe0bc21807
Merge pull request #3357 from influxdata/ntran/doc_talks_update
docs: update tech talks
2021-12-10 22:47:28 +00:00
Nga Tran 6fce12079a docs: update tech talks 2021-12-10 16:07:45 -05:00
kodiakhq[bot] c1ed2e95ab
Merge pull request #3353 from influxdata/ntran/replay_compact_os
test: replay test for compact OS chunks
2021-12-10 19:07:01 +00:00
Nga Tran e76157f0e4 chore: cleanup 2021-12-10 12:08:04 -05:00
Nga Tran 5253c71e03 test: replay test for compact OS chunks 2021-12-10 12:04:00 -05:00
Paul Dix e3d9454f8a chore: add two test schemas in data generator examples
One schema is a test for many small databases like a bunch of free tier users. The other is for a database with 2k measurments getting moderate ingest load (4.1 MB/sec).
2021-12-10 11:26:42 -05:00
kodiakhq[bot] add33e084f
Merge pull request #3350 from influxdata/ntran/test_compact_os_delete_propagation
test: propogate delete tests for compact OS chunks
2021-12-10 15:47:48 +00:00
kodiakhq[bot] 2e3aa2e1ab
Merge branch 'main' into ntran/test_compact_os_delete_propagation 2021-12-10 15:11:11 +00:00
kodiakhq[bot] 328294c4a6
Merge pull request #3346 from influxdata/pd/add-regex-to-db-writers
feat: add regex to bucket writers assignment in data generator
2021-12-10 10:11:26 +00:00
kodiakhq[bot] f477def27a
Merge branch 'main' into pd/add-regex-to-db-writers 2021-12-10 10:02:10 +00:00
kodiakhq[bot] 306c2dcba7
Merge pull request #3349 from influxdata/alamb/update_state_machine
docs: Update database startup machine diagram
2021-12-10 10:01:44 +00:00
kodiakhq[bot] dc152abb55
Merge branch 'main' into alamb/update_state_machine 2021-12-10 09:51:40 +00:00
Nga Tran 7d53fad427
Merge branch 'main' into ntran/test_compact_os_delete_propagation 2021-12-09 17:27:22 -05:00
Nga Tran 0d6fefdd09 test: test concurrent deletes and OS compaction 2021-12-09 17:13:18 -05:00
kodiakhq[bot] 7a56b4847a
Merge pull request #3345 from influxdata/ntran/compact_os_partition
feat: compact all object store chunks of a given partition
2021-12-09 20:50:17 +00:00
Nga Tran c0ba69f09e chore: marge main to branch and resolve conflict 2021-12-09 15:40:33 -05:00
Nga Tran 35370922f3 refactor: make a setup for 2 persisted chunks that can be used in for different places 2021-12-09 15:21:56 -05:00
Andrew Lamb 191e743ce0
fix: Update server/src/database.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-12-09 15:04:39 -05:00
Andrew Lamb 3cda6b6c0f
refactor: Remove collect_query and replication (#3348)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-09 19:58:19 +00:00
Andrew Lamb b2f7306d5a docs: Update database startup machine diagram 2021-12-09 14:22:29 -05:00
Nga Tran 099e2d4056
chore: Apply suggestions from code review
Co-authored-by: Marco Neumann <marco@crepererum.net>
2021-12-09 13:52:42 -05:00
Paul Dix 8c88e1e52c refactor: change orgbucket to database in data generator 2021-12-09 13:39:33 -05:00
Paul Dix 01e86a031e feat: add regex to bucket writers assignment in data generator
This adds the ability to specify a regex to match against database names when specifying what agents should write to which buckets in the data generator.

A default has also been added for ratio so that it doesn't need to be specified if only a single database writer is defined.
2021-12-09 13:39:33 -05:00
Nga Tran e46708354e test: add management cli tests 2021-12-09 12:53:45 -05:00
kodiakhq[bot] bef1024fbb
Merge pull request #3347 from influxdata/cn/optional-kafka
feat: Optionally build Kafka
2021-12-09 16:31:45 +00:00
Carol (Nichols || Goulding) 87d8f4a85f
fix: Return error instead of panicking if Kafka support is requested but not included
Also add some tests around this behavior.
2021-12-09 10:04:27 -05:00
Carol (Nichols || Goulding) bd919b52d6
fix: Add the kafka feature to the perf build 2021-12-09 08:49:34 -05:00
Carol (Nichols || Goulding) beede9bd4e
feat: Test and build releases with the Kafka feature enabled 2021-12-09 08:49:34 -05:00
Carol (Nichols || Goulding) 365917c2aa
fix: Organize imports 2021-12-09 08:49:34 -05:00
Carol (Nichols || Goulding) 471c3181bb
feat: Make a top-level influxdb_iox feature for kafka functionality
Thread the feature through router and server to the write buffer crate.
Move an end-to-end test that uses Kafka behind the feature flag.
2021-12-09 08:49:34 -05:00
Carol (Nichols || Goulding) 8c7b3966de
fix: Organize imports 2021-12-09 08:49:34 -05:00
Carol (Nichols || Goulding) 403dcae93c
feat: Put kafka write_buffer code behind a feature flag
Which is off by default. This makes rdkafka optional to minimize
build-time dependencies for users that don't plan on using a Kafka write
buffer.
2021-12-09 08:49:34 -05:00
kodiakhq[bot] 46b43d3c30
Merge pull request #3342 from influxdata/crepererum/issue3336
fix: limit number of S3 connections
2021-12-09 09:53:50 +00:00
kodiakhq[bot] 38ce54c1be
Merge branch 'main' into crepererum/issue3336 2021-12-09 09:44:55 +00:00
kodiakhq[bot] 95f907efd8
Merge pull request #3340 from influxdata/crepererum/set_server_id_twice
fix: allow setting the same server ID twice
2021-12-09 08:21:08 +00:00
kodiakhq[bot] 12c3c6b12c
Merge branch 'main' into crepererum/set_server_id_twice 2021-12-09 08:08:01 +00:00
Nga Tran 4e86d0ef30 test: propogate delete tests for compact OS chunks 2021-12-08 17:50:51 -05:00
kodiakhq[bot] 8f81120d54
Merge pull request #3319 from influxdata/cn/quick-release
feat: Create a build-time-optimized release build profile
2021-12-08 21:18:56 +00:00
kodiakhq[bot] 60de5b7c5a
Merge branch 'main' into cn/quick-release 2021-12-08 21:10:10 +00:00
Nga Tran efbfbb1a0b feat: compact all object store chunks of a given partition 2021-12-08 16:06:03 -05:00
Marco Neumann 876a9af35a fix: limit number of S3 connections
Otherwise the whole thing blows up when starting a server that has many
DBs registerd, because we potentially create 1 connection per DB (e.g.
to read out the preserved catalog).

Fixes #3336.
2021-12-08 19:06:02 +01:00
Marco Neumann a8bc5b3b6d fix: allow setting the same server ID twice
This is important for idempotence and simplifies clients and helper
scripts a lot.
2021-12-08 19:03:30 +01:00
kodiakhq[bot] b0209137e6
Merge pull request #3330 from influxdata/pd/data-generator-many-dbs
feat: add ability for data generator to write to many buckets
2021-12-08 17:26:21 +00:00
kodiakhq[bot] 90a6b255a9
Merge branch 'main' into pd/data-generator-many-dbs 2021-12-08 17:17:58 +00:00
kodiakhq[bot] 12c40b0f0f
Merge pull request #3329 from influxdata/ntran/compact_os_sql_tests
test: sql tests and checkpoint test for compact OS chunks
2021-12-08 17:17:41 +00:00
Paul Dix 2c8d17bea8 refactor: change percent to ratio in data generator bucket writers 2021-12-08 12:09:04 -05:00