32abe2e777
feat: wire up stats to metrics
2021-05-11 13:38:32 +01:00
c4987028fb
feat: expose all column stats
2021-05-11 13:00:52 +01:00
88ed58aa8a
feat: column statistics for int/float
2021-05-11 13:00:52 +01:00
ef2eda04ef
feat: add string encoder statistics
2021-05-11 13:00:52 +01:00
3622a92c8b
feat: wire in rb column metrics
2021-05-11 13:00:52 +01:00
767d6b66f6
Merge pull request #1448 from influxdata/crepererum/optional_stats_minmax
...
refactor: make `StatValues::{min,max}` optional + handle NaNs
2021-05-11 07:39:02 +00:00
6f106c49aa
Merge branch 'main' into crepererum/optional_stats_minmax
2021-05-11 07:35:13 +00:00
b6290f1ff3
chore: update deps ( #1466 )
2021-05-10 22:14:08 +00:00
795f5bfcb7
refactor: make `StatValues::{min,max}` optional + handle NaNs
...
This will allow us to:
- handle all-NULL columns correctly
- be in-line with Parquet (where min/max are optional)
- handle NaNs at least somewhat sane (they do not "poison" stats
anymore)
2021-05-10 17:12:25 +02:00
f037c1281a
feat: Calculate all system tables "on demand" ( #1452 )
...
* feat: compute system.columns table on demand
* feat: compute system.chunk_columns on demand
* feat: compute system.operations on demand
* fix: fixup schemas
* fix: Log errors
* fix: clippy
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-10 14:43:55 +00:00
8dcb4a54a2
Merge pull request #1456 from influxdata/fixrulesexistence
...
fix: Load only databases for which a config exists
2021-05-10 11:18:42 +00:00
9f5350a6c5
fix: Load only databases for which a config exists
...
Closes #1450
2021-05-10 13:14:22 +02:00
ebdab25a3c
Merge pull request #1453 from influxdata/ntran/e2e_with_unload
...
feat: Query Parquet files
2021-05-07 22:44:54 +00:00
c6b933eb63
chore: merge main to branch
2021-05-07 18:40:17 -04:00
6fc05769b1
Merge pull request #1458 from influxdata/cn/path-manipulation
...
fix: Use PathBuf rather than debug formatting and back to String
2021-05-07 21:41:04 +00:00
f2c19ec080
refactor: further address Carol's comment
2021-05-07 17:40:40 -04:00
1c61f9b485
Merge branch 'main' into cn/path-manipulation
2021-05-07 21:36:22 +00:00
971500681f
refactor: address Andrew's and Carol's comment
2021-05-07 17:33:19 -04:00
bc6ade4776
feat: Include system.chunk_columns` in the tables that are scraped ( #1455 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-07 20:31:04 +00:00
e2cc4634bf
fix: Use PathBuf rather than debug formatting and back to String
...
This is the same fix I made in 54c5f98
2021-05-07 15:58:11 -04:00
e2e3c9f77c
feat: workaround bug/limitation in OT handling of observers ( #1457 )
...
* feat: workardound bug/limitation in OT handling of observers
* chore: fix lints
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-07 18:53:28 +00:00
60f4b927e8
Merge pull request #1454 from influxdata/er/fix/null_rb_row_groups
...
fix: don't blow up on all null columns
2021-05-07 17:24:31 +00:00
4a414fc8fb
fix: don't blow up on all null columns
2021-05-07 17:31:18 +01:00
31d49db0ed
chore: a litlle more cleanup
2021-05-07 09:38:41 -04:00
ba015ee4df
refactor: clean up and add comments
2021-05-07 09:31:41 -04:00
07af2e6cda
Merge pull request #1443 from influxdata/er/feat/regex_pred_op
...
feat: Add Regex matching support to IOx's gRPC API
2021-05-07 12:58:21 +00:00
8ccc359cab
refactor: address PR feedback
2021-05-07 13:48:44 +01:00
4c4bd2f164
refactor: update query/src/func/regex.rs
2021-05-07 13:44:51 +01:00
4cc7a99854
refactor: include not match in support check
2021-05-07 13:44:51 +01:00
beee3115f4
feat: expose regex =\~ and to gRPC API
2021-05-07 13:44:51 +01:00
eae3fec571
feat: wire up regex UDF as predicate filter expr
2021-05-07 13:44:51 +01:00
3fc2c9fc04
feat: add DataFusion regex match operator
...
This commit adds a new custom UDF to IOx that provide a regex operator to Datafusion plans.
Effectively it allows predicates to contain regex operators that are applied as filters, only allowing rows that satisfy the regex to be returned.
I did not use the Arrow regex kernel for this work because that does not return a boolean array indicating which rows matched a regex, but instead returns a new string array of results. This doesn't work well with DF's approach to filtering.
2021-05-07 13:44:51 +01:00
b5ea71f45f
feat: Expose the storage usage for each column in system.chunk_columns ( #1441 )
...
* feat: Expose the storage usage for each column in system.chunk_columns
* fix: fixup logical conflicts
* refactor: move coalsce logic into the read buffer
* fix: Update system_tables to not use coalese
* fix: Improve comments
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-05-07 12:36:49 +00:00
9320f59de0
feat: add shard sink indirection ( #1447 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-07 11:04:51 +00:00
2e5a9bf18e
Merge pull request #1449 from influxdata/alamb/downgrade-rust
...
chore: revert to Rust 1.51.0
2021-05-07 10:37:06 +00:00
07f549450f
Merge branch 'main' into alamb/downgrade-rust
2021-05-07 10:32:53 +00:00
d7253c72c0
feat: Only calculate system.chunks table "on demand" ( #1446 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-07 10:26:56 +00:00
c6a8cd1501
fix: clippy
2021-05-07 06:20:57 -04:00
6f46d097f8
chore: revert to Rust 1.51.0
2021-05-07 06:04:07 -04:00
39b8e7eb57
Merge pull request #1438 from influxdata/crepererum/issue1380
...
feat: preserve parquet metadata + proper path in catalog
2021-05-07 07:56:56 +00:00
1a998d4116
feat: preserve parquet metadata in catalog
...
Closes #1380 .
2021-05-07 09:51:44 +02:00
c3d523fc4f
refactor: add col prefixes to make_chunk & Co
2021-05-07 09:51:44 +02:00
9c9196e8f6
docs: add more info to `catalog.proto`
2021-05-07 09:51:44 +02:00
5db504300d
refactor: use parsed paths instead of raw strings for catalog paths
2021-05-07 09:51:44 +02:00
4d105713a5
feat: add convenience `parsed_path!` marco
2021-05-07 09:51:44 +02:00
fadf0176d4
feat: impl Hash for DirsAndFileName and PathParts
2021-05-07 09:51:44 +02:00
d726cf4829
refactor: make `object_store::path::parts` public
2021-05-07 09:51:44 +02:00
febc1538ff
chore: Update Rust version ( #1445 )
...
* chore: Update Rust version
* refactor: Make struct constructor field orderings consistent
Sometimes I changed the struct definition, sometimes changed the struct
construction instance, depending on consistency with code around each
(other similar structs, function argument orders, etc)
More info: https://rust-lang.github.io/rust-clippy/master/index.html#inconsistent_struct_constructor
* refactor: Use flatten where appropriate
One instance is a false positive with a clippy bug.
More info:
- https://rust-lang.github.io/rust-clippy/master/index.html#filter_map_identity
- https://rust-lang.github.io/rust-clippy/master/index.html#manual_flatten
* refactor: Use Option map instead of match
More info: https://rust-lang.github.io/rust-clippy/master/index.html#manual_map
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-06 22:07:10 +00:00
55bf848bd2
feat: Now we can query directly from files in object store
2021-05-06 18:02:17 -04:00
15831a4e75
Merge pull request #1416 from DCjanus/main
...
feat: s3 compatibility storages support
2021-05-06 20:33:17 +00:00
Edd Robinson
kodiakhq[bot]
Andrew Lamb
Marco Neumann
Marko Mikulicic
Nga Tran
Carol (Nichols || Goulding)
Raphael Taylor-Davies
Andrew Lamb
Carol (Nichols || Goulding)