4d3432a1e0
docs: improve `server::config` docs
2021-06-21 10:06:50 +02:00
a153f841d8
feat: add `--force` flag to CLI wipe command
2021-06-21 09:31:23 +02:00
29bbc9a384
refactor: recoverable DB init
...
- store read and parsed DB rules even the the catalog is broken
- allow wiping the catalog for DBs w/ init failures
- try to bring the DB back online after successful wipes
Note that this does yet allow to update rules for broken DBs or to fix
DBs w/ broken rule files. However this can be implemented easily on top
of this.
2021-06-21 09:31:23 +02:00
c0766f1c26
feat: catalog wiping CLI
2021-06-21 09:31:23 +02:00
8e69202270
feat: catalog wiping gRPC
2021-06-21 09:31:23 +02:00
d17b5710a8
feat: add server functionality to wipe preserved catalogs
2021-06-21 09:31:23 +02:00
aba973a6e1
refactor: make catalog `wipe` a freestanding function
...
It does not interact with the `CatalogState` so users can call this
function without that type.
2021-06-21 09:31:23 +02:00
634c9b4c66
refactor: delete unused code ( #1759 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-19 08:15:33 +00:00
258a6b1956
chore: remove more dead code ( #1760 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-18 21:28:22 +00:00
6559a9e997
refactor: use Schema to compute InfluxDB primary keys ( #1757 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-18 21:15:31 +00:00
ab052c0501
fix: fix flaky test by updating datafusion dep ( #1758 )
...
* chore: update DataFusion dependencies
* fix: Re-enable previously flaking tests
This reverts commit c63ad0ea31
2021-06-18 20:17:18 +00:00
b855c6317e
Merge pull request #1751 from influxdata/smaller-cache
...
chore: Switch to smaller cache dep
2021-06-18 18:54:40 +00:00
1d8469951f
Merge branch 'main' into smaller-cache
2021-06-18 18:50:10 +00:00
de67bd3efe
refactor: Remove PartitionChunk::table_schema ( #1756 )
...
* refactor: Remove PartitionChunk::table_schema
* docs: update comments
2021-06-18 16:13:16 +00:00
9beeca3e7c
refactor: Unify schema handling in query crate ( #1755 )
...
* refactor: Unify schema handling in query crate
* fix: doclink
2021-06-18 14:10:57 +00:00
1c13d676b4
refactor: Rename query::PartitionChunk --> query::QueryChunk ( #1754 )
2021-06-18 13:24:09 +00:00
b612c3af4e
chore: Switch to smaller cache dep
2021-06-18 09:43:28 +02:00
ec43a87909
chore: Update itertools deps ( #1750 )
2021-06-17 17:56:44 +00:00
c63ad0ea31
chore: Ignore flaky test ( #1749 )
2021-06-17 14:51:19 +00:00
c5eea9af6a
feat: Implement DeduplicateExec ( #1733 )
...
* feat: Implement DeduplicateExec
* fix: Doc comments
* fix: fix comment
* fix: Update with arrow ticket references and use datafusion coalsce batches impl
* refactor: rename inner.rs to algo.rs
* docs: Add additional documentation on rationale for last field value
* docs: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* docs: Update query/src/provider/deduplicate/algo.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* docs: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* refactor: do not use pub(crate)
* docs: fix test comments
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-06-17 14:17:52 +00:00
f6dbc8d6f2
refactor: add ChunkAddr to describe location of chunk in catalog ( #1745 )
...
* refactor: add ChunkPath to describe location of chunk in catalog
* refactor: rename ChunkPath to ChunkAddr
* chore: further renames
* chore: even more renames
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-17 12:04:37 +00:00
b42218a197
chore: Add proper format for SchemaPivotNode ( #1744 )
2021-06-17 11:32:48 +00:00
3ecfee65b8
Merge pull request #1739 from influxdata/crepererum/issue1725-b
...
feat: make DB catalog work w/ transaction aborts
2021-06-17 07:10:39 +00:00
dd61266d90
Merge branch 'main' into crepererum/issue1725-b
2021-06-17 07:06:47 +00:00
cac54de5e1
Merge pull request #1724 from influxdata/pd-persistence-windows
...
feat: Add persistence windows ingestion tracking
2021-06-16 20:19:51 +00:00
ecfabe3408
chore: address PR feedback, changed closed persistence window to dequeue.
2021-06-16 15:28:37 -04:00
6bc4b383a4
chore: update based on PR feedback, add more comments
2021-06-16 15:28:37 -04:00
802ecacf61
feat: Add persistence windows ingestion tracking
...
This adds a new module, persistence_windows, to the mutable buffer crate. Later PRs will add this into the mutable buffer chunk where it can be used to track when the lifecycle for persistence should be triggered.
2021-06-16 15:28:37 -04:00
27e499a1dd
feat: instrumented, upgradeable RWLocks ( #1741 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-16 16:15:10 +00:00
9cb89c1542
Merge pull request #1742 from influxdata/logtracingsplit
...
chore: Split LoggingConfig out of TracingConfig
2021-06-16 15:15:16 +00:00
4adf3b5e52
chore: Split LoggingConfig out of TracingConfig
...
Client CLI utils only need LoggingConfig. Servers need also the TracingConfig
2021-06-16 16:07:57 +02:00
8c0831f410
Merge pull request #1738 from influxdata/betterdefaults
...
chore: Allow custom default log filter
2021-06-16 11:34:57 +00:00
58e1494ca2
chore: Allow custom default log level
...
This PR moves the definition of the `log_filter` default value from the `structopt` annotation
to the `trogging::Builder`. This allows users of `trogging` to define their own default value
for the log level.
2021-06-16 13:30:32 +02:00
87b2a1eaea
docs: add note about why we write parquets during transactions
2021-06-16 11:01:14 +02:00
e056d97cf6
test: always test transaction aborts
2021-06-16 11:01:14 +02:00
ec053f674c
feat: make DB catalog work w/ transaction aborts
2021-06-16 11:01:14 +02:00
caaf95c6ec
refactor: remove lock from `TestCatalogState`
2021-06-16 10:51:15 +02:00
c8c412f6fe
refactor: rework catalog state interface
...
This now allows not only for copy-based transaction handling but also
for eager exec and rollbacks. This will be useful to properly implement
transaction aborts for the "real" catalog.
2021-06-16 10:51:15 +02:00
2596de072e
feat: make sure DB catalog can correctly add and remove parquet files
...
Note that this does NOT yet allow it to correctly abort transactions.
2021-06-16 10:50:47 +02:00
e064a6bbba
test: add test suite for `CatalogState` impls
...
This makes it easier to check if `CatalogState` correctly implement all
features, including transaction aborting.
2021-06-16 10:50:47 +02:00
a429de4784
Merge pull request #1734 from influxdata/trogging-like-a-pro
...
feat: Factor out tracing/logging CLI options
2021-06-16 07:39:18 +00:00
760bcde3f0
feat: Factor out tracing/logging CLI options
...
This PR factors out the tracing/logging CLI optinos into the `trogging` utility crate,
so that multiple binaries from the IOx suite (such as conductor) can use the same (and quite complex)
logging/tracing configuration options (flags and env vars).
Closes influxdata/conductor#343
2021-06-16 00:54:11 +02:00
38d17a3093
chore: remove unused query dependency ( #1731 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-15 22:06:13 +00:00
2c8060160f
chore: update datafusion deps ( #1721 )
...
* chore: update datafusion deps
* chore: update deps
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-15 19:06:15 +00:00
bf54ab51f2
refactor: split lifecycle into separate crate ( #1730 )
2021-06-15 15:57:47 +00:00
f96e05d26a
refactor: traitify lifecycle policy ( #1729 )
...
* refactor: traitify lifecycle policy
* chore: docs
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-15 14:00:06 +00:00
e333cbbd4e
Merge pull request #1719 from influxdata/trogging
...
chore: Pull tracing+logging setup in its own crate
2021-06-15 13:08:47 +00:00
4dc338e1bc
Merge branch 'main' into trogging
2021-06-15 13:04:06 +00:00
bde35cf5be
chore: Pull tracing+logging setup in its own crate
...
1. so that it can be reused by other binaries (e.g. conductor)
2. so that it's faster to build when working on it.
2021-06-15 14:52:04 +02:00
b756e09904
refactor: Rename parquet_file::Chunk --> ParquetChunk ( #1722 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-15 11:21:49 +00:00
Marco Neumann
Andrew Lamb
kodiakhq[bot]
Marko Mikulicic
Raphael Taylor-Davies
Paul Dix