Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
7,419 Commits
205 Branches
453 Tags
382 MiB
Commit Graph

549 Commits (506cdebf384259c166740d910abcafe864d4ffa0)

Author SHA1 Message Date
Andrew Lamb 77b80e7618
fix(InfluxQL): treat null tags as `''` rather than `null` in storagerpc queries (#3557) 
* fix(InfluxQL): treat null tags as `''` rather than `null` in storage rpc queries

* test: add one more case

* fix: Update comment

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-03 12:14:43 +00:00
kodiakhq[bot] a2ed6a1b75
Merge branch 'main' into combine-non-overlapping-chunks 2022-02-02 20:47:51 +00:00
Andrew Lamb 429d59f1b6
feat: Simplify predicates in the `InfluxRpcFrontend` before using them (#3588) 
* feat: normalize + simplify RPC predicates before using them

* docs: Update predicate/src/rpc_predicate.rs

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-02 19:46:57 +00:00
Raphael Taylor-Davies 8a8de19fb5 feat: combine non-overlapping chunks without deletes 2022-02-02 16:40:30 +00:00
Andrew Lamb 030a2cb4c1
chore: Update datafusion (#3613) 
* chore: Update datafusion

* fix: update for latest DF API

* fix: another API change

* fix: clippy

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-02 16:27:11 +00:00
Andrew Lamb b03c900174 fix: handle IS NULL / IS NOT NULL during rpc rewrite 2022-02-01 13:48:58 -05:00
Marco Neumann 22778a3a80
chore: upgrade rskafka and parking_lot (#3592) 2022-02-01 11:50:42 +00:00
Andrew Lamb 7b96a37165
chore: Update datafusion (#3586) 
* chore: update DataFusion to f849968057ddddccc9aa19915ef3ea56bf14d80d

* fix: reduce overhead of creating physical expressions

* chore: use MemTrackingMetrics

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-31 18:15:28 +00:00
Dom 32d7c4cbfe
refactor: remove InfluxColumnType::IOx (#3565) 
* refactor: remove InfluxColumnType::IOx

Remove unused column variant - see #3554 for context.

* refactor: reserve SEMANTIC_TYPE_IOX name in proto

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-27 21:15:36 +00:00
Raphael Taylor-Davies d1d45fe818
feat: columnar predicate pruning across `Chunks` (#3553) 
* feat: columnar predicate pruning

* fix: doc

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-27 17:02:46 +00:00
Andrew Lamb 2062267d0f
chore: Update hashbrown (#3551) 
* chore: Update hashbrown

* fix: hakari

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-27 15:34:10 +00:00
Andrew Lamb 8dd96127d6
fix: Reuse the same DataFusion DiskManager and MemoryManager (Do not recreate temp files) (#3515) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-27 15:04:06 +00:00
Raphael Taylor-Davies 21c1824a7a
refactor: remove table_names from Predicate (#3545) 
* refactor: remove table_names from Predicate

* chore: fix benchmarks

* chore: review feedback

Co-authored-by: Edd Robinson <me@edd.io>

* chore: review feedback

* chore: replace Default::default with InfluxRpcPredicate::default()

Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-27 14:44:49 +00:00
Andrew Lamb 5488c257d1
chore: Update datafusion, upgrade to arrow/parqet/arrow-flight 8.0.0 (#3517) 
* chore: Update datafusion

* chore: update to arrow 8

* fix: update to use new DataFusion APIs

* fix: update case for sortedness

* fix: cargo hakari
 2022-01-27 13:33:27 +00:00
Raphael Taylor-Davies 1b6aed063d
feat: add per-partition tracing (#3532) 
* feat: add per-partition tracing

* chore: docs

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-26 10:39:21 +00:00
Edd Robinson 1c2681c24e
refactor: remove unused const (#3521) 2022-01-26 09:23:50 +00:00
Nga Tran 52866fe6a9
fix: merge record batches into one batch (#3535) 
* fix: merge record batches into one batch

refactor: address review comments

* chore: update test output
 2022-01-25 23:29:16 +00:00
Nga Tran d559561fd7
refactor: have the deduplicate work without chunk statistics (#3519) 
* refactor: have the deduplicate work without chunk statistics

* test: more tests for duplicates data on different combinations of record batches

* refactor: address review comments
 2022-01-25 17:00:25 +00:00
Andrew Lamb f0d50f447a
fix: Special case tag_keys with max timestamp range (#3485) 
* fix: Special case tag_keys with max timestamp range

* docs: comment

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-20 14:14:34 +00:00
Andrew Lamb 9b6e626626
chore: Update datafusion (and get fix for influxql test failure) (#3484) 
* test: add tests for comparing dictionary arrays

* chore: update datafusion deps

* refactor: Update code for DataFusion API changes

* fix: update test

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-20 14:01:47 +00:00
Andrew Lamb 1843476651
chore: Update datafusion deps (#3471) 
* chore: Update datafusion

* refactor: Update to use new Exec plan APIs

* fix: error message

* fix: fixup last bit

* fix: clippy

* fix: doclink

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-17 15:26:19 +00:00
Edd Robinson cdb4f43d62 refactor: address feedback 2022-01-14 10:41:27 +00:00
Edd Robinson 0b343bcf19 feat: add RAII token to time query completion 2022-01-14 10:26:11 +00:00
Andrew Lamb dd23056efd
chore: update datafusion, arrow, prost, tonic, pbjson, etc (#3455) 
* chore: update datafusion, arrow, prost, tonic, etc

* fix: update pprof as well

* chore: update hakari

* fix: update pbjson

* chore: update heappy

* fix: hakari

* fix: workaround https://github.com/influxdata/influxdb_iox/issues/3458

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-13 17:07:15 +00:00
Andrew Lamb cdf5c21cd4
fix: Fix max timestamp value comparison in chunk metadata (#3453) 
* fix: Fix max timestamp value comparison in chunk metadata

* refactor: rename contains to overlaps

Co-authored-by: Edd Robinson <me@edd.io>
 2022-01-13 16:58:30 +00:00
Marco Neumann f3f6f335a9
chore: upgrade to snafu 0.7 (#3440) 2022-01-11 19:22:36 +00:00
Andrew Lamb 80c048528c
refactor: Clean up code by using `lit` and `lit_timestamp_nano` rather than `Expr::Literal(ScalarValue(Some(...)))` (#3427) 2022-01-07 17:23:14 +00:00
Andrew Lamb 336ffd1966
refactor: Remove `Result` in QueryDatabase trait (none of the functions can fail) (#3422) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-06 22:03:08 +00:00
Andrew Lamb a93ae739a9
feat: Add table_name to Partition API (#3421) 2022-01-06 16:38:39 +00:00
Andrew Lamb 527885f7f8
chore: Update datafusion (#3413) 
* chore: Update datafusion and update code to handle timezone aware timestamps

* fix: cargo hakari

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-12-23 14:52:12 +00:00
Andrew Lamb 218042784f
feat: add `system.queries `system table (#3328) 
* feat: Add query log and system table

* docs: Add docstrings for test normalization
 2021-12-08 16:26:24 +00:00
Edd Robinson 7443b3685e refactor: address PR feedback 2021-12-08 12:18:23 +00:00
Edd Robinson 2a2c5889c3 perf: loading chunks doesn't require sort 2021-12-08 12:03:56 +00:00
Carol (Nichols || Goulding) 0b7c7764c6
fix: Use a consistent version range for tokio-stream across all crates 2021-12-06 09:37:16 -05:00
Carol (Nichols || Goulding) 02c297e850
fix: Always specify the parking_lot feature of tokio to get potential perf boost 2021-12-06 09:37:15 -05:00
Carol (Nichols || Goulding) 39862e3886
fix: Disable default chrono features 
Connects to #3117. Won't affect the workspace-hack crate until changes
to some other crates can be upstreamed.
 2021-12-06 09:36:49 -05:00
Carol (Nichols || Goulding) 5d0fd1c603
fix: Allow dead code on fields that are now detected as never read 2021-12-02 11:52:01 -05:00
Andrew Lamb 9e8639f230
chore: Update DataFusion pin (#3279) 
* chore: Update DataFusion pin

* fix: Update for new DF API

* fix: update plan output

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-12-02 12:42:28 +00:00
Raphael Taylor-Davies a286960285
refactor: remove delete predicates from read filter interface (#3212) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-11-26 12:15:29 +00:00
Raphael Taylor-Davies 1722704077
fix: don't project on schema metadata mismatch (#3213) 
* fix: don't project on schema metadata mismatch

* chore: add test
 2021-11-25 17:07:02 +00:00
Edd Robinson 48dd1ae757 fix: ensure time column not returned 2021-11-24 16:23:43 +00:00
Edd Robinson 02d58ca085 refactor: clone series for testing 2021-11-23 17:00:28 +00:00
Edd Robinson cb00dc30c7 refactor: order tag sets by measurement/field 2021-11-23 14:15:06 +00:00
Andrew Lamb 50e9e02ff7
chore: Update datafusion (#3188) 
* chore: Update datafusion

* fix: Update for LogicalPlan changes

* fix: Update explain plan output

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-11-23 12:23:23 +00:00
Carol (Nichols || Goulding) 9fd4a560f5
feat: Results of running cargo hakari manage-deps 2021-11-19 09:21:57 -05:00
Andrew Lamb 0b3df2ab50
fix: reduce verbosity of logs (#3159) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-11-19 10:03:27 +00:00
Raphael Taylor-Davies 3d091208af
refactor: move delete predicate into data_types (#2731) (#3094) 
* refactor: move delete predicate into dml (#2731)

* refactor: move DeletePredicate to data_types

* chore: fix doc

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-11-15 10:28:58 +00:00
Nga Tran 97206b13cb fix: statistics for max/min(time) should have data type timstamp 2021-11-05 18:11:54 -04:00
Andrew Lamb 5e86989990
fix: Only use timestamps in first/last when there is a corresponding … (#2988) 
* fix: Only use timestamps in first/last when there is a corresponding value

* docs: Fix broken English in comment

* docs: clarify expectations in test
 2021-11-01 18:54:43 +00:00
dependabot[bot] 1b377f0377
chore(deps): bump tokio-stream from 0.1.7 to 0.1.8 
Bumps [tokio-stream](https://github.com/tokio-rs/tokio) from 0.1.7 to 0.1.8.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.7...tokio-stream-0.1.8)

---
updated-dependencies:
- dependency-name: tokio-stream
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-11-01 11:38:59 +00:00
dependabot[bot] c540b40f05
chore(deps): bump tokio from 1.12.0 to 1.13.0 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.12.0...tokio-1.13.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-11-01 11:21:59 +00:00
dependabot[bot] a1f978e280
chore(deps): bump tokio-util from 0.6.8 to 0.6.9 
Bumps [tokio-util](https://github.com/tokio-rs/tokio) from 0.6.8 to 0.6.9.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.6.8...tokio-util-0.6.9)

---
updated-dependencies:
- dependency-name: tokio-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-11-01 01:28:41 +00:00
Andrew Lamb d2c2143277
fix: wrong results in read_filter with "complex" OR predicate and IS_NULL (#2977) 
* fix: Allow evaluating general purpose predicates in gRPC by rewriting missing columns to null

* fix: update commands and add some additional tests

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-29 14:30:28 +00:00
kodiakhq[bot] 6760212e86
Merge branch 'main' into ntran/group 2021-10-26 21:25:57 +00:00
Nga Tran 5c2a5e1030 fix: fmt after appying review comments 2021-10-26 17:17:35 -04:00
Nga Tran 6950e7a9b6
chore: Apply suggestions from code review 
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
 2021-10-26 17:08:07 -04:00
Nga Tran 8c43b24745 chore: cleanup 2021-10-26 16:46:50 -04:00
Nga Tran 46cb4df14d fix: now no rows will be return for read_group if the table has no data or all its data is soft deleted or the table does not exist 2021-10-26 16:40:36 -04:00
Andrew Lamb 39b3b8d33f fix: Logical merge conflicts 2021-10-25 13:47:10 -04:00
Andrew Lamb 7cd56cbc56 Merge remote-tracking branch 'origin/main' into er/fix/flux/2691 2021-10-25 13:41:08 -04:00
Marco Neumann bc7244c48e chore: use Rust edition 2021 2021-10-25 10:58:20 +02:00
Andrew Lamb 52cf1a85b9
fix(metadata): Do not report table_names for tables that have no non-null values that match predicate (#2947) 
* fix(metadata): Do not report table_names for tables that have no non-null values that match predicate

* fix: make apply_predicate_to_metadata precise

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-24 11:03:30 +00:00
Andrew Lamb 28962038c1
fix: properly handle null values in table_name plan (#2946) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-23 10:35:02 +00:00
Edd Robinson eb71102d82
Merge branch 'main' into er/fix/flux/2691 2021-10-22 20:21:22 +01:00
Andrew Lamb e8ddc4e0db
feat: Add custom `NonNullChecker` operator that checks for non-null values in columns (#2944) 
* feat: Add custom `NonNullChecker` operator that checks for non-null values in columns

* fix: remove println

* fix: typo

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-22 18:53:20 +00:00
Andrew Lamb fe155e15fb
fix: fix flaky test (#2939) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-22 17:08:27 +00:00
Edd Robinson 4464114330
Merge branch 'main' into er/fix/flux/2691 2021-10-22 09:43:35 +01:00
Carol (Nichols || Goulding) 654d33684a fix: Move code shared between if/else blocks out 2021-10-21 20:47:35 -04:00
Edd Robinson 8898631034
refactor: update query/src/frontend/influxrpc.rs 2021-10-21 22:38:20 +01:00
Edd Robinson fc7ce3535d refactor: fix _value filtering for non-selector aggregates 2021-10-21 22:28:41 +01:00
Edd Robinson f8489fc774 refactor: ensure _value filtering works read_filter 2021-10-21 22:28:41 +01:00
Edd Robinson 8a70bd898b feat: add _value support to selector aggregates 2021-10-21 22:28:38 +01:00
Edd Robinson 0cb9751b0a refactor: extract _value expr from predicate 2021-10-21 22:26:32 +01:00
Marco Neumann f7ca80e29f
test: ensure query cancellation (somewhat) works (#2931) 
* feat: enable reconfiguration of in-use throttled store

This is handy for tests for which a part should run "normal" and another
one should be throttled/blocked.

* feat: keep track of the number of tasks within a `DedicatedExecutor`

* test: ensure query cancellation (somewhat) works

We cannot really test that query cancellation finishes all subtasks
because _tokio_ doesn't provide sufficient stats / inspection, at least
as long we don't want to rely heavily on _tokio_ tracing. So let's at
least check that tasks from the dedicated executors are pruned properly.

For all other regressions we need to add unit tests to the affected
components. See for example:

- https://github.com/apache/arrow-datafusion/issues/1103
- https://github.com/apache/arrow-datafusion/pull/1105
- https://github.com/apache/arrow-datafusion/pull/1112
- https://github.com/apache/arrow-datafusion/pull/1121

Closes #2027.
 2021-10-21 19:10:58 +00:00
Nga Tran aecbdc0468 chore: merge main to branch 2021-10-20 17:09:02 -04:00
Nga Tran 995ffd43f6 chore: run fmt 2021-10-20 16:56:21 -04:00
Nga Tran d9b034cfaa chore: cleanup 2021-10-20 16:08:14 -04:00
Nga Tran 97d5760347 feat: support delelete for table_names 2021-10-20 15:58:25 -04:00
Andrew Lamb ee2ca8fc32
fix(read_group): Support grouping on `_start` and `_stop` (#2918) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-20 15:16:47 +00:00
Edd Robinson 438cd1767a refactor: update query/src/frontend/influxrpc.rs 
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
 2021-10-20 15:00:07 +01:00
Edd Robinson a96284e120 test: use rewriter properly 2021-10-20 14:59:27 +01:00
Edd Robinson a0ba75576e fix: add support for rewriting field column 2021-10-20 14:59:27 +01:00
Edd Robinson 7d86d1144e test: add coverage for field value rewriter 2021-10-20 14:59:27 +01:00
Edd Robinson 3fe8f64aa4 feat: add support for filtering on _value 2021-10-20 14:59:27 +01:00
Edd Robinson f85a9f15b1 refactor: plumb schema to TableNormalizedPredicate 2021-10-20 14:59:27 +01:00
Andrew Lamb 9974a5364c
chore(security): Replace prettytable with comfy-table (#2905) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-20 10:44:36 +00:00
Nga Tran 8c058f791b chore: cleanup 2021-10-19 15:43:06 -04:00
Andrew Lamb b55ca06fe3
fix: read_window_aggregate overflow (#2908) 2021-10-19 19:20:39 +00:00
Nga Tran e664799593 feat: support delete for tag_values 2021-10-19 14:18:01 -04:00
kodiakhq[bot] 254a842eed
Merge branch 'main' into ntran/table_names 2021-10-19 17:53:25 +00:00
Andrew Lamb d5cffb5f54
fix(read_window_aggregates): return aggregates as integer rather than unsigned (#2906) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-19 17:47:15 +00:00
Nga Tran c755678aa1 refactor: address review comments 2021-10-19 13:41:41 -04:00
Nga Tran cabb007956 chore: Merge branch 'main' into ntran/table_names 2021-10-19 13:22:28 -04:00
Andrew Lamb a82dc6f5f0
chore: Update datafusion + arrow (#2903) 
* chore: Update datafusion to latest, arrow to 6.0.0

* fix: Update tests

* fix: bubble internal error

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-19 17:14:08 +00:00
Nga Tran ea85d6478e chore: remove code changes for table_names abecause normal plan needs to be implemented first 2021-10-19 10:36:10 -04:00
Nga Tran afa6e50c9c feat: make tag_keys work with delete 2021-10-18 15:36:19 -04:00
Andrew Lamb f5a84122e3
feat: Support grouping by _field and _measurement (#2874) 
* feat: Support grouping by _field and _measurement

* fix: clippy

* fix: doclink

* refactor: rename SeriesOrGroup --> Either

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-18 15:32:24 +00:00
Andrew Lamb 51276119df
docs: Add better SeriesSet explanation (#2857) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-15 21:53:51 +00:00
Andrew Lamb beaf77cecf
refactor: move Series translation logic into query crate, update gRPC tests (#2852) 
* refactor: move Series translation logic into query crate

* refactor: update grpc_tests to use new display

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-15 11:06:40 +00:00
Andrew Lamb 08e6a01e69
refactor: Move series set converter into its own module (#2847) 
* refactor: Move series set converter into its own module

* fix: add file

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-14 14:16:32 +00:00