4409d2c8af
feat: instrument catalog locks ( #1464 )
...
* feat: instrument catalog locks (#1355 )
* chore: add metrics test
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-11 18:59:11 +00:00
c85d1574eb
feat: move dictionary and bitset into arrow_utils ( #1459 )
...
* feat: move dictionary and bitset into arrow_utils
* chore: review feedback
* chore: remove redundant dictionary methods
* chore: consistent type parameter name in PackedStringArray
* chore: review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-11 16:43:38 +00:00
3622a92c8b
feat: wire in rb column metrics
2021-05-11 13:00:52 +01:00
b6290f1ff3
chore: update deps ( #1466 )
2021-05-10 22:14:08 +00:00
e2e3c9f77c
feat: workaround bug/limitation in OT handling of observers ( #1457 )
...
* feat: workardound bug/limitation in OT handling of observers
* chore: fix lints
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-07 18:53:28 +00:00
8ccc359cab
refactor: address PR feedback
2021-05-07 13:48:44 +01:00
eae3fec571
feat: wire up regex UDF as predicate filter expr
2021-05-07 13:44:51 +01:00
3fc2c9fc04
feat: add DataFusion regex match operator
...
This commit adds a new custom UDF to IOx that provide a regex operator to Datafusion plans.
Effectively it allows predicates to contain regex operators that are applied as filters, only allowing rows that satisfy the regex to be returned.
I did not use the Arrow regex kernel for this work because that does not return a boolean array indicating which rows matched a regex, but instead returns a new string array of results. This doesn't work well with DF's approach to filtering.
2021-05-07 13:44:51 +01:00
216903a949
refactor: move protobuf conversion logic to generated_types ( #1437 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-06 15:49:27 +00:00
10f89a3e8d
refactor: split entry out into separate crate ( #1428 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-06 11:36:23 +00:00
b153fc9ef3
chore: remove unused dependency from influxdb_line_protocol ( #1435 )
2021-05-06 10:13:00 +00:00
01ba48c1ba
feat: instrumentable RwLocks ( #1355 ) ( #1421 )
2021-05-06 08:36:30 +00:00
7e2e2c4caf
refactor: initial split of observability_deps ( #1429 )
2021-05-06 08:25:09 +00:00
86771ea629
chore: update arrow/datafusion deps ( #1433 )
...
* chore: update datafusion deps
* chore: update arrow deps
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-05 22:37:31 +00:00
ca1c698fd0
chore: update hashbrown ( #1430 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-05 22:32:46 +00:00
7e01650c53
Merge branch 'main' into reflection
2021-05-05 20:29:13 +00:00
0d6d94dc00
feat: Enable gRPC reflection
2021-05-05 22:04:47 +02:00
411cf134e9
refactor: explode arrow_deps ( #1425 )
...
* refactor: explode arrow_deps
* chore: workaround doctest bug
2021-05-05 16:59:12 +00:00
1f42eb89cd
feat: implement parquet metadata handling
...
Closes #1379 and contributes to #1380 .
2021-05-05 13:29:16 +02:00
80d78c2a67
feat: re-export matching `parquet_format` in `arrow_deps`
2021-05-05 13:29:16 +02:00
34754ebcdb
refactor: move MemoryStream to arrow_deps
2021-05-05 13:29:16 +02:00
ce568b4b2f
Merge branch 'main' into crepererum/issue1253
2021-05-03 11:11:42 +00:00
b579ef8646
feat: Add jemalloc stats
2021-05-03 12:10:48 +02:00
136c35cb88
feat: implement transaction handling for catalog
...
Closes #1253 .
2021-05-03 10:04:35 +02:00
234c899631
feat: Add gauge metric
2021-04-30 14:10:42 +02:00
13fbf2e68d
refactor: plumb registry to gRPC server
2021-04-29 14:00:05 +01:00
74d35ce9a4
chore: update deps ( #1365 )
2021-04-29 10:52:43 +00:00
a64e622f6e
feat: Add interactive SQL repl ( #1332 )
...
* feat: Add interactive SQL repl
* fix: try and fix test
* fix: remove test for prompt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-28 20:41:02 +00:00
7ca1da3fcd
feat: pushdown table and partition key predicates to catalog ( #736 ) ( #1327 )
...
* feat: catalog predicate pushdown (#736 )
* chore: fix lints
* chore: review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-27 15:31:47 +00:00
0a835436ac
feat: use bitmasks within MUB ( #1274 ) ( #1289 )
...
* feat: use bitmasks within MUB (#1274 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-26 18:00:16 +00:00
e393d16e36
chore: update arrow versions ( #1300 )
2021-04-26 10:11:25 +00:00
87de656d23
feat: metrics test helpers
2021-04-23 15:58:48 +00:00
97b2369140
refactor: swap existing metrics for THE NEW WAY
2021-04-23 15:58:48 +00:00
ea909f45ad
feat: RED metrics
2021-04-23 15:58:48 +00:00
74c25f541d
feat: fast MUB dictionary arrow conversion ( #1273 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-22 20:31:44 +00:00
fe4fa29930
feat: MUB benchmarks ( #635 ) ( #1271 )
2021-04-22 11:15:32 +00:00
83d6550316
feat: Implement write_entry_downstream
2021-04-21 20:50:46 +00:00
f0b93c5c8c
refactor: Rename the wal crate to write_buffer
2021-04-21 17:43:03 +00:00
fdb22c9d9b
chore: update tonic to 0.4.2 ( #1268 )
2021-04-21 11:54:03 +00:00
dc6637b448
Merge branch 'main' into jgm-tracing-logging
2021-04-20 20:16:40 +00:00
895827aa98
chore: disable SIMD in Arrow
2021-04-20 17:30:50 +00:00
5539887cd6
chore: put env_logger back
2021-04-19 15:48:30 -07:00
87396edc56
chore: incremental dependency update
...
This fixes a metrics CI error, and introduces two new CI errors.
2021-04-19 15:48:30 -07:00
df9dd15ab8
feat(tracing): improve logs and tracing
...
Logs and traces are emitted via one pipeline. For now, it is not
possible to emit both, but it should be possible in a few weeks, as
tokio/tracing/tracing-subscriber is going through some refactoring recently.
All affected flags are well-documented, and I have tested all but the
OTLP output flags.
chore: clippy happy
chore: revert instrumentation changes
feat: add log format logfmt, log destinations stderr, stdout
chore: clippy happy
2021-04-19 15:48:30 -07:00
fd0da7e74a
chore: upgrade arrow and Rust
...
See https://github.com/apache/arrow/pull/10082 for upstream PR.
2021-04-19 14:00:04 +02:00
4b706141de
refactor: log new row groups added to RB
2021-04-19 10:25:57 +00:00
4c23ca8888
feat: full implementation of parquet's read_filter for review
2021-04-16 16:03:24 -04:00
878b1b318e
feat: Initial scaffolding for routing layer
...
Part of #916
Adding first-class concept of ShardId in shard config, fixes #1156
NEXT:
- [ ] implement sharder
- [ ] implement `write_entry_downstream`
- [ ] add tests
2021-04-15 09:02:47 +00:00
a3fc5e2474
refactor: change sync::RwLock to parking_lot
2021-04-14 19:18:03 +00:00
f5f768d750
feat: Add a dedicated threadpool for running queries ( #1191 )
...
* feat: use a dedicated tokio threadpool for running queries
* feat: plumb number of executor threads through to command line
thread through command line
* fix: Logical merge conflict
* fix: another logical conflict
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-14 10:48:09 +00:00
Raphael Taylor-Davies
Edd Robinson
Andrew Lamb
kodiakhq[bot]
Marko Mikulicic
Marco Neumann
Carol (Nichols || Goulding)
Jacob Marble
Nga Tran