11b25b3aaf
refactor: swap order of partition and table in in-memory catalog ( #1678 )
...
* refactor: swap order of partition and table in in-memory catalog
* chore: review feedback
* chore: validate panic message
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-10 16:40:30 +00:00
876f642860
test: add benchmark for catalog loading
2021-06-10 15:42:21 +02:00
29dadba4f3
chore: update dependencies ( #1673 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-10 13:01:31 +00:00
a614fef5bc
chore: remove more unused dependencies ( #1658 )
...
* chore: remove more unused deps
* refactor: move benchmarks into server_benchmarks crate
2021-06-09 10:17:20 +00:00
07c4277ca7
refactor: schema merge to give more control over field merging ( #1653 )
...
* refactor: schema merge to give more control over field merging
* chore: review feedback
2021-06-09 06:30:45 +00:00
e9834a907c
feat: Prune on boolean column predicates too ( #1629 )
...
* chore: update deps to get latest DataFusion
* fix: enable boolean pruning tests
* fix: update explain plan tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-07 16:51:30 +00:00
87297f7db4
Merge branch 'main' into cn/delete
2021-06-07 13:32:42 +00:00
5749a2c119
chore: cleanup legacy TSM -> parquet code ( #1639 )
...
* chore: cleanup legacy parquet code
* chore: remove tests of removed functionality
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-07 12:59:33 +00:00
f4a9a5ae56
fix: Remove write buffer
2021-06-04 14:40:17 -04:00
42f26b609b
refactor: Move `query_tests` and `server_benchmarks` into their own crate --> smaller `server` ( #1628 )
...
* refactor: Separate query_tests into its own crate
* fix: references
* refactor: break out server benchmarks
* fix: Update query_tests/src/lib.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-06-04 17:31:19 +00:00
bbd73e59be
feat: jitter background clean-up job + wait on first job
2021-06-03 11:23:29 +02:00
0a625b50e6
feat: store transaction timestamp in preserved catalog
2021-06-02 09:41:19 +02:00
83b2eacea6
chore: update deps for datafusion ( #1601 )
2021-06-01 20:00:39 +00:00
c0e4e6951a
chore: update datafusion ( #1594 )
2021-06-01 15:55:07 +00:00
d3711a5591
refactor: Use ParquetExec from DataFusion to read parquet files ( #1580 )
...
* refactor: use ParquetExec to read parquet files
* fix: test
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-01 14:44:07 +00:00
3338ddcca9
chore: update dependencies ( #1593 )
2021-06-01 14:05:56 +00:00
73cedd2f88
chore: remove unused dependency ( #1587 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-31 14:22:11 +00:00
d50c7c8919
chore: remove unused dependency ( #1581 )
2021-05-31 09:58:10 +00:00
00e735ef0d
chore: remove unused dependencies ( #1583 )
2021-05-29 10:31:57 +00:00
d8f19348bf
feat: per-column dictionaries in MUB ( #1570 )
...
* feat: per-column dictionaries in MUB
* chore: fmt
* refactor: remove chunk-level dictionary
* chore: remove redundant sort
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-28 13:51:56 +00:00
166851d952
Merge branch 'main' into crepererum/in_file_metadata
2021-05-26 07:39:53 +00:00
638d754e0f
chore: Update datafusion + other deps ( #1557 )
2021-05-25 22:32:14 +00:00
c2fd85209c
feat: wait for task shutdown on DedicatedExecutor ( #1537 )
2021-05-25 11:33:55 +00:00
19a2733d30
feat: preserve transaction metadata in parquets
2021-05-25 09:56:12 +02:00
14ba25f86d
chore: Update datafusion and use released version of arrow crates ( #1546 )
...
* chore: Update datafusion and use released version of arrow crate
* fix: Update for change in API
2021-05-24 15:37:22 +00:00
634ceb886b
feat: add Dictionary Values type
2021-05-20 10:49:49 +01:00
ce2f8351be
fix: Cache outbound gRPC connections
2021-05-19 18:28:45 +02:00
40b21dbca1
feat: Add /debug/pprof/profile
2021-05-19 13:08:42 +02:00
133ce12827
chore: update deps ( #1499 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-14 20:22:16 +00:00
f9178dbb5f
feat: push metrics into catalog ( #1488 )
...
* feat: push metrics into catalog
* chore: minor cleanup
* fix: include db labels in chunk metric domains
* chore: fmt
* fix: don't allow dropping moving chunks
* chore: further tweaks
* chore: review feedback
* feat: use new_unregistered() for metric instruments instead of default
* chore: use &[KeyValue] instead of &Vec<KeyValue>
* refactor: make GauageValue non default constructible
2021-05-14 17:37:39 +00:00
ebea554e5a
feat: Report num_cpus seen by IOx on startup ( #1484 )
2021-05-12 13:42:16 +00:00
0b1ef52481
chore: upgrade arrow and datafusion ( #1483 )
2021-05-12 13:08:37 +00:00
b02105e47b
feat: construct StringDictionary from PackedStringArray ( #1475 )
...
* feat: construct StringDictionary from PackedStringArray
* chore: fix formatting
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-11 20:16:25 +00:00
4409d2c8af
feat: instrument catalog locks ( #1464 )
...
* feat: instrument catalog locks (#1355 )
* chore: add metrics test
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-11 18:59:11 +00:00
c85d1574eb
feat: move dictionary and bitset into arrow_utils ( #1459 )
...
* feat: move dictionary and bitset into arrow_utils
* chore: review feedback
* chore: remove redundant dictionary methods
* chore: consistent type parameter name in PackedStringArray
* chore: review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-11 16:43:38 +00:00
3622a92c8b
feat: wire in rb column metrics
2021-05-11 13:00:52 +01:00
b6290f1ff3
chore: update deps ( #1466 )
2021-05-10 22:14:08 +00:00
e2e3c9f77c
feat: workaround bug/limitation in OT handling of observers ( #1457 )
...
* feat: workardound bug/limitation in OT handling of observers
* chore: fix lints
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-07 18:53:28 +00:00
8ccc359cab
refactor: address PR feedback
2021-05-07 13:48:44 +01:00
eae3fec571
feat: wire up regex UDF as predicate filter expr
2021-05-07 13:44:51 +01:00
3fc2c9fc04
feat: add DataFusion regex match operator
...
This commit adds a new custom UDF to IOx that provide a regex operator to Datafusion plans.
Effectively it allows predicates to contain regex operators that are applied as filters, only allowing rows that satisfy the regex to be returned.
I did not use the Arrow regex kernel for this work because that does not return a boolean array indicating which rows matched a regex, but instead returns a new string array of results. This doesn't work well with DF's approach to filtering.
2021-05-07 13:44:51 +01:00
216903a949
refactor: move protobuf conversion logic to generated_types ( #1437 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-06 15:49:27 +00:00
10f89a3e8d
refactor: split entry out into separate crate ( #1428 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-06 11:36:23 +00:00
b153fc9ef3
chore: remove unused dependency from influxdb_line_protocol ( #1435 )
2021-05-06 10:13:00 +00:00
01ba48c1ba
feat: instrumentable RwLocks ( #1355 ) ( #1421 )
2021-05-06 08:36:30 +00:00
7e2e2c4caf
refactor: initial split of observability_deps ( #1429 )
2021-05-06 08:25:09 +00:00
86771ea629
chore: update arrow/datafusion deps ( #1433 )
...
* chore: update datafusion deps
* chore: update arrow deps
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-05 22:37:31 +00:00
ca1c698fd0
chore: update hashbrown ( #1430 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-05 22:32:46 +00:00
7e01650c53
Merge branch 'main' into reflection
2021-05-05 20:29:13 +00:00
0d6d94dc00
feat: Enable gRPC reflection
2021-05-05 22:04:47 +02:00
Raphael Taylor-Davies
Marco Neumann
Andrew Lamb
kodiakhq[bot]
Carol (Nichols || Goulding)
Edd Robinson
Marko Mikulicic