Commit Graph

3256 Commits (be9b3a48534e2292d9fc26c4fd1c88c1db72254f)

Author SHA1 Message Date
Marco Neumann be9b3a4853 fix: protobuf lint fixes 2021-06-10 15:42:21 +02:00
Marco Neumann 294c304491 feat: impl catalog checkpointing infrastructure
This implements a way to add checkpoints to the preserved catalog and
speed up replay.

Note: This leaves the "hook it up into the actual DB" for a future PR.

Issue: #1381.
2021-06-10 15:42:21 +02:00
Marco Neumann 188cacec54 refactor: use `Arc` to pass `ParquetFileMetaData`
This will be handy when the catalog state must be able to return
metadata objects so that we can create checkpoints, esp. when we use
multi-chunk parquet files in some midterm future.
2021-06-10 15:42:21 +02:00
Marco Neumann c7412740e4 refactor: prepare to read and write multiple file types for catalog
Prepares #1381.
2021-06-10 15:42:21 +02:00
Marco Neumann 33e364ed78 feat: add encoding info to transaction protobuf
This should help with #1381.
2021-06-10 15:42:21 +02:00
Marco Neumann 876f642860 test: add benchmark for catalog loading 2021-06-10 15:42:21 +02:00
Andrew Lamb 29dadba4f3
chore: update dependencies (#1673)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-10 13:01:31 +00:00
kodiakhq[bot] fd8fbeb89d
Merge pull request #1677 from influxdata/crepererum/clippy_future_not_send_part2
chore: enforce `clippy::future_not_send` for `influxdb_iox_client,influxdb2_client,query,tracker`
2021-06-10 12:54:47 +00:00
Marco Neumann 7bacef6835 chore: enforce `clippy::future_not_send` for `tracker` 2021-06-10 09:52:57 +02:00
Marco Neumann 7b1106ff64 chore: enforce `clippy::future_not_send` for `query` 2021-06-10 09:48:35 +02:00
Marco Neumann 3832987ecb chore: enforce `clippy::future_not_send` for `influxdb2_client` 2021-06-10 09:46:35 +02:00
Marco Neumann 1c5a779b83 chore: enforce `clippy::future_not_send` for `influxdb_iox_client` 2021-06-10 09:45:02 +02:00
kodiakhq[bot] 6d3432c702
Merge pull request #1671 from influxdata/crepererum/clippy_future_not_send_part1
chore: enforce `clippy::future_not_send` for `parquet_file,object_store`
2021-06-10 07:24:04 +00:00
kodiakhq[bot] 3ba27bdbd9
Merge branch 'main' into crepererum/clippy_future_not_send_part1 2021-06-10 07:19:31 +00:00
kodiakhq[bot] 21f6f58e97
Merge pull request #1670 from influxdata/crepererum/extract_server_init
refactor: extract server init code
2021-06-10 07:19:04 +00:00
kodiakhq[bot] 5f863a59fd
Merge branch 'main' into crepererum/extract_server_init 2021-06-10 07:14:57 +00:00
kodiakhq[bot] 44d8fb9472
Merge branch 'main' into crepererum/clippy_future_not_send_part1 2021-06-10 07:10:11 +00:00
kodiakhq[bot] feed19ac8b
Merge pull request #1666 from influxdata/ntran/dedup_within_chunk
feat: Add Sort operator into the deduplicate plan
2021-06-09 19:13:36 +00:00
Nga Tran 4478d900ee refactor: capture test output 2021-06-09 15:09:13 -04:00
Nga Tran 8cc99e3420 Merge branch 'ntran/dedup_within_chunk' of https://github.com/influxdata/influxdb_iox into ntran/dedup_within_chunk 2021-06-09 14:40:29 -04:00
Nga Tran b3c94b9d65 refactor: change order of fields to pass circle CI tests 2021-06-09 14:40:10 -04:00
kodiakhq[bot] eed73a30c5
Merge branch 'main' into ntran/dedup_within_chunk 2021-06-09 18:19:17 +00:00
Nga Tran c1c58018fc refactor: address review comments 2021-06-09 14:17:47 -04:00
Andrew Lamb 89fcc457f4
fix: Fix bug in chunk overlap calculation due to nulls (#1669)
* fix: Fix bug in chunk overlap calculation due to nulls

* docs: add note about algorithmic complexity

* fix: avoid recursion in normal case
2021-06-09 17:46:39 +00:00
Marco Neumann 4fe2d7af9c chore: enforce `clippy::future_not_send` for `parquet_file` 2021-06-09 18:18:27 +02:00
Marco Neumann 0e2a291683 chore: enforce `clippy::future_not_send` for `object_store` 2021-06-09 17:56:43 +02:00
Marco Neumann d9c38dfe88 refactor: extract server init code
This prepares for #1624, so the end results looks a bit cleaner.
2021-06-09 16:53:11 +02:00
Andrew Lamb a614fef5bc
chore: remove more unused dependencies (#1658)
* chore: remove more unused deps

* refactor: move benchmarks into server_benchmarks crate
2021-06-09 10:17:20 +00:00
Luke Bond d4b2522650
Merge pull request #1660 from influxdata/chore/check-for-port-conflict
chore: check for grpc & http bind addr conflict
2021-06-09 10:37:23 +01:00
Luke Bond cd3f264666
Merge branch 'main' into chore/check-for-port-conflict 2021-06-09 10:29:53 +01:00
Luke Bond 1293b05fa0
chore: better structured logging of port conflict err 2021-06-09 10:29:27 +01:00
kodiakhq[bot] 7e22a15584
Merge pull request #1618 from influxdata/crepererum/lazy_db_loading
feat: move DB loading into background worker
2021-06-09 07:27:54 +00:00
kodiakhq[bot] b49abf9b02
Merge branch 'main' into crepererum/lazy_db_loading 2021-06-09 07:23:35 +00:00
Raphael Taylor-Davies 07c4277ca7
refactor: schema merge to give more control over field merging (#1653)
* refactor: schema merge to give more control over field merging

* chore: review feedback
2021-06-09 06:30:45 +00:00
Nga Tran 3d50ff7a60 refactor: remove comments 2021-06-08 21:48:57 -04:00
Nga Tran ab7d3384b7 refactor: remove unused comments 2021-06-08 21:43:02 -04:00
Nga Tran 3e10351538 test: add tests for the sort plan 2021-06-08 21:40:46 -04:00
Andrew Lamb cba7f270b4
docs: Improve comments + whitespace (#1663) 2021-06-08 21:13:35 +00:00
Nga Tran 68e3a2121f feat: add SortExec 2021-06-08 15:04:31 -04:00
kodiakhq[bot] afcaa5d7a0
Merge pull request #1662 from influxdata/alamb/hook_up_grouping
feat: Hook up chunk grouping into provider
2021-06-08 19:01:58 +00:00
Andrew Lamb 666204d4a8 fix: remove whitespace changes 2021-06-08 14:46:55 -04:00
Andrew Lamb b23c4e5210 fix: clippy 2021-06-08 14:44:48 -04:00
Andrew Lamb fd8a87484e feat: Hook up chunk grouping into provider 2021-06-08 14:42:37 -04:00
Nga Tran edbf1b7d5e Merge branch 'main' into ntran/dedup_within_chunk 2021-06-08 13:18:40 -04:00
Nga Tran 40cb4f741f feat: initial implementaton 2021-06-08 13:17:36 -04:00
Luke Bond b9b9d25d12
chore: check for grpc & http bind addr conflict 2021-06-08 18:10:34 +01:00
Andrew Lamb 62e8675737
refactor: move primary_key calculaton to TableSummary (#1659) 2021-06-08 17:06:37 +00:00
Andrew Lamb 34ba268cf1
feat: Group chunks by potential overlap (#1654)
* feat: Group chunks by potential overlap

* docs: clarify in what way the calculation is conservative

* fix: Add test for mixed nulls
2021-06-08 16:55:29 +00:00
kodiakhq[bot] 4789183f63
Merge pull request #1657 from influxdata/er/feat/influxrpc/not_eq_op
feat: enable not equal operator for the influxrpc frontend
2021-06-08 16:16:29 +00:00
Edd Robinson b88f277477 feat: enable not eq operator 2021-06-08 15:57:07 +01:00