Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
6,261 Commits
205 Branches
453 Tags
382 MiB
Commit Graph

237 Commits (16d8ae5e04b18bc41101546b0a618f5b5d8d0e7a)

Author SHA1 Message Date
Andrew Lamb 9974a5364c
chore(security): Replace prettytable with comfy-table (#2905) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-20 10:44:36 +00:00
Raphael Taylor-Davies ce0127a1f7
feat: MutableBatch write API (#2090) (#2724) (#2882) 
* feat: MutableBatch write API (#2090) (#2724)

* chore: fix lint

* fix: handle dictionaries with unused mappings

* chore: review feedback

* chore: further review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-20 08:44:14 +00:00
Andrew Lamb b55ca06fe3
fix: read_window_aggregate overflow (#2908) 2021-10-19 19:20:39 +00:00
Nga Tran e664799593 feat: support delete for tag_values 2021-10-19 14:18:01 -04:00
kodiakhq[bot] 254a842eed
Merge branch 'main' into ntran/table_names 2021-10-19 17:53:25 +00:00
Andrew Lamb d5cffb5f54
fix(read_window_aggregates): return aggregates as integer rather than unsigned (#2906) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-19 17:47:15 +00:00
Nga Tran cabb007956 chore: Merge branch 'main' into ntran/table_names 2021-10-19 13:22:28 -04:00
Andrew Lamb a82dc6f5f0
chore: Update datafusion + arrow (#2903) 
* chore: Update datafusion to latest, arrow to 6.0.0

* fix: Update tests

* fix: bubble internal error

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-19 17:14:08 +00:00
Nga Tran 716515fc5a fix: turn ignore back on for table_name tests as it is not ready yet 2021-10-19 10:38:25 -04:00
Nga Tran ea85d6478e chore: remove code changes for table_names abecause normal plan needs to be implemented first 2021-10-19 10:36:10 -04:00
Marco Neumann a364bdfb5f chore: remove unused `query_tests` => `chrono` dep 2021-10-19 14:45:56 +02:00
Nga Tran afa6e50c9c feat: make tag_keys work with delete 2021-10-18 15:36:19 -04:00
Andrew Lamb 30c3d1e001
test: Add read_group test for missing fields (#2881) 
* test: Add read_group test

* fix: Update to use o2 table
 2021-10-18 18:43:52 +00:00
Andrew Lamb f5a84122e3
feat: Support grouping by _field and _measurement (#2874) 
* feat: Support grouping by _field and _measurement

* fix: clippy

* fix: doclink

* refactor: rename SeriesOrGroup --> Either

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-18 15:32:24 +00:00
Nga Tran 9244e9fc4e test: Delete tests for Influxrpc queries 2021-10-15 17:26:36 -04:00
Andrew Lamb beaf77cecf
refactor: move Series translation logic into query crate, update gRPC tests (#2852) 
* refactor: move Series translation logic into query crate

* refactor: update grpc_tests to use new display

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-15 11:06:40 +00:00
Nga Tran b2d265dc51 chore: run format after accepting reviewer's suggestions 2021-10-14 17:31:42 -04:00
Nga Tran 39a556c5eb
chore: Apply suggestions from code review 
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
 2021-10-14 17:21:11 -04:00
Nga Tran 6a5aa00e2c chore: make the comments and names clearer 2021-10-14 17:11:37 -04:00
kodiakhq[bot] 993c6173d1
Merge branch 'main' into ntran/grpc_storage 2021-10-14 15:28:05 +00:00
Nga Tran 69d1253240 chore: Merge branch 'ntran/grpc_storage' of https://github.com/influxdata/influxdb_iox into ntran/grpc_storage 2021-10-14 11:23:29 -04:00
Nga Tran faf65f38cc refactor: address review comments 2021-10-14 11:23:20 -04:00
Nga Tran 08f1831aef
refactor: Apply suggestions from code review 
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
 2021-10-14 09:25:44 -04:00
Edd Robinson 693c878781 refactor: update test 2021-10-14 09:23:29 +01:00
Edd Robinson 75caa4aad8 test: add test duplicating defect input 2021-10-14 09:23:29 +01:00
Nga Tran 8dd9dcce01 test: verify if all scenarios are created correctly and add a few delete tests for read_filter 2021-10-13 17:21:03 -04:00
Nga Tran f85d6d5da8 chore: cleanup 2021-10-12 17:23:01 -04:00
Nga Tran 401124548c chore: ignore failed newly added test due to bug 2021-10-12 17:20:17 -04:00
Nga Tran 144ce77e39 chore: merge main to branch 2021-10-12 15:59:57 -04:00
Nga Tran 459dd46ae9 refactor: move delete tests to .sql 2021-10-12 15:49:23 -04:00
Andrew Lamb 035654b4f9
refactor: do not rebuild query_test when .sql or .expected files change (#2816) 
* feat: Do not rebuild query_tests if .sql or .expected change

* feat: Add CI check

* refactor: move some sql tests to .sql files

* tests: port tests / expected results to data files

* fix: restore old name check-flatbuffers
 2021-10-12 19:34:54 +00:00
Nga Tran 9b6726b99c refactor: rename to a more general name function 2021-10-12 10:34:55 -04:00
Nga Tran 0b4ae95ca4 refactor: exhaust scenarios for one-chunk test 2021-10-11 17:47:41 -04:00
Raphael Taylor-Davies b39e01f7ba
feat: migrate PersistenceWindows to TimeProvider (#2722) (#2798) 2021-10-11 20:40:00 +00:00
Nga Tran fbf5539336 chore: merge main to branch 2021-10-11 10:47:10 -04:00
Raphael Taylor-Davies afe34751e7
refactor: split out schema crate (#2781) 
* refactor: split out schema crate

* chore: fix doc
 2021-10-11 09:45:08 +00:00
Nga Tran f7475322a6 chore: merge main to branch, resolve conflicts, and discover an inconsitent bug 2021-10-08 15:50:46 -04:00
Nga Tran f2cdb9531f chore: cleanup 2021-10-08 14:52:15 -04:00
Nga Tran adbcd85c26 fix: fully fix 2745 2021-10-08 14:37:34 -04:00
Andrew Lamb 2072b4066e
feat: Implement support for `_measurement` predicate in gRPC plans (#2772) 
* feat: Implement filtering for _measurement in general purpose gRPC plans

* docs: fixup docstrings

* fix: fmt

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-08 17:25:33 +00:00
Nga Tran 22d6f11bea fix: add cols of delete predicates into the schema of scanning columns 2021-10-07 17:37:34 -04:00
Marco Neumann 63d74be490 refactor: make `ChunkId` a UUID 2021-10-07 10:23:27 +02:00
Nga Tran de148337e8 fix: half way fix the bug to inlcude schema of column in delete predicate into the schema of IOx scan to avoid missing reading columns 2021-10-06 17:43:48 -04:00
Andrew Lamb efa2316626
fix: do not sort the output of read_group with no group keys (#2755) 2021-10-06 18:59:58 +00:00
Nga Tran bd93b411c7 chore: cleanup 2021-10-06 10:57:51 -04:00
kodiakhq[bot] 1aee2a49eb
Merge branch 'main' into ntran/no_use_stats 2021-10-06 14:06:01 +00:00
Nga Tran 65a02f7085
refactor: Apply suggestions from code review 
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
 2021-10-06 10:04:28 -04:00
Raphael Taylor-Davies ce5b24e65d
refactor: use DateTime<Utc> in PersistenceWindows (#2722) (#2743) 
* refactor: use DateTime<Utc> in PersistenceWindows (#2722)

* chore: fix benchmark

* chore: fmt

* chore: review feedback
 2021-10-06 09:39:32 +00:00
Nga Tran 055e69439d test: fix auto created tests 2021-10-05 18:11:27 -04:00
Nga Tran aa64daca86 feat: dDisable using statistics to query data if there are soft deleted rows 2021-10-05 17:52:32 -04:00
Raphael Taylor-Davies d0929e3a34
feat: persist no chunks (#2712) (#2718) 
* feat: persist no chunks (#2712)

* fix: persist partition

* fix: chunk ordering test

* chore: fix logical conflict

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-05 15:18:35 +00:00
Marco Neumann 10c1a72402 refactor: remove unused fields from `DeletePredicate` 2021-10-05 09:29:24 +02:00
Nga Tran 1856d7184c fix: make stop time inclusive 2021-10-04 14:14:16 -04:00
Nga Tran 64cd0cb3ce chore: cleanup 2021-10-04 12:43:35 -04:00
Nga Tran ba96b20b1f test: turn on tests of delete all 2021-10-04 12:28:49 -04:00
Marco Neumann 75ac6e8646 refactor: make `DeletePredicate::range` non-optional 2021-10-04 16:36:20 +02:00
Marco Neumann 5a5a929b9e refactor: introduce `DeletePredicate` 
`DeletePredicate` is a simpler version of `Predicate` that is based on
IOx `DeleteExpr` instead of the full-blown DataFusion `Expr`. This will
allow us to do a couple of things (in follow-up changes):

- Order and de-duplicate delete predicates
- Normalize predicates
- Infallible serialization
- Smaller memory footprint

Note that this change only affects delete expressions. Query expressions
that are supported via the API are not changed. The query subsystem also
still uses the full-featured expressions/predicates (delete
expressions/predicates are converted to the more powerful DataFusion
version on-the-fly).
 2021-10-04 16:36:20 +02:00
Edd Robinson 7ab10daa19
Merge branch 'main' into dependabot/cargo/arrow-5.5.0 2021-10-04 12:58:29 +01:00
Raphael Taylor-Davies e8eab2cc97
feat: allow compaction and persistence to retun no chunk (#2664) (#2700) 
* feat: allow compaction and persistence to retun no chunk (#2664)

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-10-04 10:54:47 +00:00
dependabot[bot] d1f5209869
chore(deps): bump arrow from 5.4.0 to 5.5.0 
Bumps [arrow](https://github.com/apache/arrow-rs) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0)

---
updated-dependencies:
- dependency-name: arrow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2021-10-04 08:55:38 +00:00
Raphael Taylor-Davies b402423e9e
feat: remove move lifecycle action (#2674) 
* feat: remove move_chunk lifecycle action

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-09-30 16:58:05 +00:00
Nga Tran 105b63b2af
test: use integer in predicates (#2673) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-09-30 10:46:03 +00:00
Edd Robinson 003f72ba00
Merge branch 'main' into er/fix/read_buffer/pred_validate 2021-09-29 14:50:12 +01:00
Edd Robinson a52b86e070 fix: fallback to no predicate if it can't be validated 
Closes: #1603

If a predicate cannot be executed against a read buffer chunk because of schema conflicts then fall back to applying no predicate and let the query engine apply predicates in the Filter step of the plan.
 2021-09-29 14:42:56 +01:00
Nga Tran 3cfea6e83f refactor: add comment 2021-09-28 17:44:49 -04:00
Nga Tran 2837aae479 test: more tests that inlcude the repro of 2546 2021-09-28 17:42:23 -04:00
Nga Tran d7af1b8290 refactor: exhaust delete test scenarios 2021-09-28 12:31:46 -04:00
Nga Tran 4237d6dcc6 refactor: address review comments and refactor some more obvios ones 2021-09-27 21:38:00 -04:00
Nga Tran cbfa3e85af chore: Merge branch 'main' into ntran/refactor_delete_tests 2021-09-27 14:52:38 -04:00
Nga Tran ff77c53e11 chore: cleanup 2021-09-27 14:43:08 -04:00
Nga Tran ec3e1fda06 refactor: refactor all delete helper functions 2021-09-27 14:39:01 -04:00
Andrew Lamb 8e89dde85c
test: enable stack overflow test (#2618) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-09-27 14:27:08 +00:00
Andrew Lamb a55a21c644
chore: Update datafusion (#2635) 
* chore: Update datafusion and sqlparser

* fix: remove STACK_SIZE workaround

* chore: update datafusion_util

* chore: update predicate

* chore: update query_tests

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-09-27 14:13:19 +00:00
Nga Tran c5e40b50cc refactor: delete tests to make it easier to add more tests 2021-09-24 16:35:55 -04:00
Nga Tran 0799b5ab91 test: query with selection predicate on deleted data 2021-09-23 16:54:40 -04:00
Nga Tran c35ecf257d refactor: address review comments 2021-09-23 11:14:20 -04:00
Nga Tran 713c7f4820 chore: remove empty line 2021-09-22 17:11:09 -04:00
Nga Tran 332649a8ee refactor: another cleanup due to marco -> function and add a test that exposes bug 2021-09-22 17:09:20 -04:00
Nga Tran 10097766d3 refactor: changes to use function instead of marco after merging main 2021-09-22 16:53:51 -04:00
Nga Tran 2399a932fb chore: Merge branch 'main' into ntran/more_delete_tests 2021-09-22 16:47:15 -04:00
Nga Tran 3a00d9c70a refactor: cleanup 2021-09-22 16:42:46 -04:00
Nga Tran 400ec93498 test: more delete tests 2021-09-22 16:38:27 -04:00
Marco Neumann 9a1a3f4de5 refactor: convert `run_tag_keys_test_case` to a function 2021-09-22 13:46:54 +02:00
Marco Neumann 8cc08d6bd9 refactor: convert `run_table_names_test_case` to a function 2021-09-22 13:46:54 +02:00
Marco Neumann 2408dea394 refactor: convert `run_read_window_aggregate_test_case` to a function 2021-09-22 13:46:54 +02:00
Marco Neumann cdfe49f041 refactor: convert `run_read_group_test_case` to a function 2021-09-22 13:46:54 +02:00
Marco Neumann 1db13ec437 refactor: convert `run_tag_values_test_case` to a function 2021-09-22 13:46:54 +02:00
Marco Neumann 20655d10b6 refactor: convert `run_read_window_aggregate_test_case` to a function 2021-09-22 13:46:54 +02:00
Marco Neumann b24be8d17d refactor: convert `run_field_columns_test_case` to a function 2021-09-22 13:46:54 +02:00
Marco Neumann 6ef8c11036 refactor: convert `run_table_schema_test_case` to a function 2021-09-22 13:46:54 +02:00
Marco Neumann 70a754a022 refactor: convert `run_sql_test_case` macro to a function 
This generated less code and speeds up compilation a bit.
 2021-09-22 13:46:53 +02:00
Andrew Lamb d38648952c
chore: Update datafusion (#2602) 
* chore: Update datafusion + other deps

* refactor: update query crate for new async interfaces

* refactor: update server crate for new async interface

* refactor: update query_tests crate for new async interfaces

* refactor: update ioxd and server to use new async interface

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-09-22 10:33:25 +00:00
Nga Tran ae40d93af4 test: more delete tests 2021-09-21 18:10:12 -04:00
Nga Tran b4b33c378e test: turn all delete tests on 2021-09-21 15:23:41 -04:00
Nga Tran 2f371b6f79 refactor: address review comments 2021-09-21 14:46:24 -04:00
Nga Tran 93551bdd1e fix: all chunks now are applied delete predicates during scan 2021-09-21 12:17:59 -04:00
Nga Tran 85989cc8a3 test: add more delete tests and test scenarios 2021-09-20 18:18:08 -04:00
kodiakhq[bot] 77d84ca5ab
Merge branch 'main' into crepererum/chunk_id 2021-09-20 13:39:05 +00:00
Marco Neumann cef5aeee52 refactor: introduce `ChunkId` type 2021-09-20 13:10:41 +02:00
Nga Tran 364d245eae feat: apply negated delete predicates during scan 2021-09-17 16:20:42 -04:00