Commit Graph

3572 Commits (89757d72320772b7c0fa66f5dbf4c9c840a0cefa)

Author SHA1 Message Date
Edd Robinson 7e3df17896 test: update benchmarks 2021-06-21 15:29:23 +01:00
Edd Robinson ac54320821 refactor: update server with new chunk API 2021-06-21 15:12:17 +01:00
Edd Robinson 2ff80162d3 refactor: one table per chunk 2021-06-21 15:08:38 +01:00
Edd Robinson 15416ca223 refactor: enable empty table init 2021-06-21 15:08:38 +01:00
kodiakhq[bot] 15aaf235b6
Merge pull request #1675 from influxdata/cn/kafka-docker-working
test: Starting Kafka Write Buffer integration tests
2021-06-21 13:59:06 +00:00
Carol (Nichols || Goulding) 31ad5c85f9 fix: Consistently refer to docker-compose 2021-06-21 09:41:37 -04:00
Carol (Nichols || Goulding) 4191300a41 fix: Move environment variables to ENV declarations in Dockerfile 2021-06-21 09:41:36 -04:00
Carol (Nichols || Goulding) 485d8cddc4 docs: Remove outdated comment 2021-06-21 09:41:36 -04:00
Carol (Nichols || Goulding) 44c2c0c763 docs: Recommend pulling latest docker image first 2021-06-21 09:41:36 -04:00
Carol (Nichols || Goulding) 135d885ca5 docs: Clarify the differences between the two ways to run the Kafka integration tests 2021-06-21 09:41:36 -04:00
Carol (Nichols || Goulding) 4ba6d8cd4f chore: Add a dockerignore file to not send the target directory 2021-06-21 09:41:36 -04:00
Carol (Nichols || Goulding) b4644e6108 test: Start of Kafka Write Buffer integration tests 2021-06-21 09:41:35 -04:00
Carol (Nichols || Goulding) a0583381a1 chore: Run kafka integration tests using docker-compose
And, importantly, using the CircleCI machine image to avoid running
Docker-in-Docker.
2021-06-21 09:36:35 -04:00
Raphael Taylor-Davies ea04ce40dc
feat: transactional lifecycle API (#1753)
* feat: transactional lifecycle API

* chore: remove redundant upgrade

* feat: lifecycle error propagation

* chore: add usage doctest

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-21 13:09:53 +00:00
kodiakhq[bot] 40dfdbf1fb
Merge pull request #1572 from influxdata/crepererum/catalog_cli
feat: wire catalog wiping all the way to the CLI
2021-06-21 11:49:35 +00:00
Marco Neumann 47f8310a27 docs: clarify wipe behavior 2021-06-21 13:27:18 +02:00
Marco Neumann 0d7c3ff279
docs: fix typos
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-06-21 13:18:20 +02:00
Marco Neumann cb09762067 docs: document server startup and error recovery 2021-06-21 10:26:52 +02:00
Marco Neumann 4d3432a1e0 docs: improve `server::config` docs 2021-06-21 10:06:50 +02:00
Marco Neumann a153f841d8 feat: add `--force` flag to CLI wipe command 2021-06-21 09:31:23 +02:00
Marco Neumann 29bbc9a384 refactor: recoverable DB init
- store read and parsed DB rules even the the catalog is broken
- allow wiping the catalog for DBs w/ init failures
- try to bring the DB back online after successful wipes

Note that this does yet allow to update rules for broken DBs or to fix
DBs w/ broken rule files. However this can be implemented easily on top
of this.
2021-06-21 09:31:23 +02:00
Marco Neumann c0766f1c26 feat: catalog wiping CLI 2021-06-21 09:31:23 +02:00
Marco Neumann 8e69202270 feat: catalog wiping gRPC 2021-06-21 09:31:23 +02:00
Marco Neumann d17b5710a8 feat: add server functionality to wipe preserved catalogs 2021-06-21 09:31:23 +02:00
Marco Neumann aba973a6e1 refactor: make catalog `wipe` a freestanding function
It does not interact with the `CatalogState` so users can call this
function without that type.
2021-06-21 09:31:23 +02:00
Andrew Lamb 634c9b4c66
refactor: delete unused code (#1759)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-19 08:15:33 +00:00
Andrew Lamb 258a6b1956
chore: remove more dead code (#1760)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-18 21:28:22 +00:00
Andrew Lamb 6559a9e997
refactor: use Schema to compute InfluxDB primary keys (#1757)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-18 21:15:31 +00:00
Andrew Lamb ab052c0501
fix: fix flaky test by updating datafusion dep (#1758)
* chore: update DataFusion dependencies

* fix: Re-enable previously flaking tests

This reverts commit c63ad0ea31.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-18 20:17:18 +00:00
kodiakhq[bot] b855c6317e
Merge pull request #1751 from influxdata/smaller-cache
chore: Switch to smaller cache dep
2021-06-18 18:54:40 +00:00
kodiakhq[bot] 1d8469951f
Merge branch 'main' into smaller-cache 2021-06-18 18:50:10 +00:00
Andrew Lamb de67bd3efe
refactor: Remove PartitionChunk::table_schema (#1756)
* refactor: Remove PartitionChunk::table_schema

* docs: update comments
2021-06-18 16:13:16 +00:00
Andrew Lamb 9beeca3e7c
refactor: Unify schema handling in query crate (#1755)
* refactor: Unify schema handling in query crate

* fix: doclink
2021-06-18 14:10:57 +00:00
Andrew Lamb 1c13d676b4
refactor: Rename query::PartitionChunk --> query::QueryChunk (#1754) 2021-06-18 13:24:09 +00:00
Marko Mikulicic b612c3af4e
chore: Switch to smaller cache dep 2021-06-18 09:43:28 +02:00
Andrew Lamb ec43a87909
chore: Update itertools deps (#1750) 2021-06-17 17:56:44 +00:00
Marko Mikulicic c63ad0ea31
chore: Ignore flaky test (#1749) 2021-06-17 14:51:19 +00:00
Andrew Lamb c5eea9af6a
feat: Implement DeduplicateExec (#1733)
* feat: Implement DeduplicateExec

* fix: Doc comments

* fix: fix comment

* fix: Update with arrow ticket references and use datafusion coalsce batches impl

* refactor: rename inner.rs to algo.rs

* docs: Add additional documentation on rationale for last field value

* docs: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* docs: Update query/src/provider/deduplicate/algo.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* docs: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* refactor: do not use pub(crate)

* docs: fix test comments

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-06-17 14:17:52 +00:00
Raphael Taylor-Davies f6dbc8d6f2
refactor: add ChunkAddr to describe location of chunk in catalog (#1745)
* refactor: add ChunkPath to describe location of chunk in catalog

* refactor: rename ChunkPath to ChunkAddr

* chore: further renames

* chore: even more renames

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-17 12:04:37 +00:00
Andrew Lamb b42218a197
chore: Add proper format for SchemaPivotNode (#1744) 2021-06-17 11:32:48 +00:00
kodiakhq[bot] 3ecfee65b8
Merge pull request #1739 from influxdata/crepererum/issue1725-b
feat: make DB catalog work w/ transaction aborts
2021-06-17 07:10:39 +00:00
kodiakhq[bot] dd61266d90
Merge branch 'main' into crepererum/issue1725-b 2021-06-17 07:06:47 +00:00
kodiakhq[bot] cac54de5e1
Merge pull request #1724 from influxdata/pd-persistence-windows
feat: Add persistence windows ingestion tracking
2021-06-16 20:19:51 +00:00
Paul Dix ecfabe3408 chore: address PR feedback, changed closed persistence window to dequeue. 2021-06-16 15:28:37 -04:00
Paul Dix 6bc4b383a4 chore: update based on PR feedback, add more comments 2021-06-16 15:28:37 -04:00
Paul Dix 802ecacf61 feat: Add persistence windows ingestion tracking
This adds a new module, persistence_windows, to the mutable buffer crate. Later PRs will add this into the mutable buffer chunk where it can be used to track when the lifecycle for persistence should be triggered.
2021-06-16 15:28:37 -04:00
Raphael Taylor-Davies 27e499a1dd
feat: instrumented, upgradeable RWLocks (#1741)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-16 16:15:10 +00:00
kodiakhq[bot] 9cb89c1542
Merge pull request #1742 from influxdata/logtracingsplit
chore: Split LoggingConfig out of TracingConfig
2021-06-16 15:15:16 +00:00
Marko Mikulicic 4adf3b5e52
chore: Split LoggingConfig out of TracingConfig
Client CLI utils only need LoggingConfig. Servers need also the TracingConfig
2021-06-16 16:07:57 +02:00
kodiakhq[bot] 8c0831f410
Merge pull request #1738 from influxdata/betterdefaults
chore: Allow custom default log filter
2021-06-16 11:34:57 +00:00