64cd0cb3ce
chore: cleanup
2021-10-04 12:43:35 -04:00
ba96b20b1f
test: turn on tests of delete all
2021-10-04 12:28:49 -04:00
75ac6e8646
refactor: make `DeletePredicate::range` non-optional
2021-10-04 16:36:20 +02:00
5a5a929b9e
refactor: introduce `DeletePredicate`
...
`DeletePredicate` is a simpler version of `Predicate` that is based on
IOx `DeleteExpr` instead of the full-blown DataFusion `Expr`. This will
allow us to do a couple of things (in follow-up changes):
- Order and de-duplicate delete predicates
- Normalize predicates
- Infallible serialization
- Smaller memory footprint
Note that this change only affects delete expressions. Query expressions
that are supported via the API are not changed. The query subsystem also
still uses the full-featured expressions/predicates (delete
expressions/predicates are converted to the more powerful DataFusion
version on-the-fly).
2021-10-04 16:36:20 +02:00
7ab10daa19
Merge branch 'main' into dependabot/cargo/arrow-5.5.0
2021-10-04 12:58:29 +01:00
e8eab2cc97
feat: allow compaction and persistence to retun no chunk ( #2664 ) ( #2700 )
...
* feat: allow compaction and persistence to retun no chunk (#2664 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-04 10:54:47 +00:00
d1f5209869
chore(deps): bump arrow from 5.4.0 to 5.5.0
...
Bumps [arrow](https://github.com/apache/arrow-rs ) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases )
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md )
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0 )
---
updated-dependencies:
- dependency-name: arrow
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 08:55:38 +00:00
b402423e9e
feat: remove move lifecycle action ( #2674 )
...
* feat: remove move_chunk lifecycle action
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-30 16:58:05 +00:00
105b63b2af
test: use integer in predicates ( #2673 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-30 10:46:03 +00:00
003f72ba00
Merge branch 'main' into er/fix/read_buffer/pred_validate
2021-09-29 14:50:12 +01:00
a52b86e070
fix: fallback to no predicate if it can't be validated
...
Closes : #1603
If a predicate cannot be executed against a read buffer chunk because of schema conflicts then fall back to applying no predicate and let the query engine apply predicates in the Filter step of the plan.
2021-09-29 14:42:56 +01:00
3cfea6e83f
refactor: add comment
2021-09-28 17:44:49 -04:00
2837aae479
test: more tests that inlcude the repro of 2546
2021-09-28 17:42:23 -04:00
d7af1b8290
refactor: exhaust delete test scenarios
2021-09-28 12:31:46 -04:00
4237d6dcc6
refactor: address review comments and refactor some more obvios ones
2021-09-27 21:38:00 -04:00
cbfa3e85af
chore: Merge branch 'main' into ntran/refactor_delete_tests
2021-09-27 14:52:38 -04:00
ff77c53e11
chore: cleanup
2021-09-27 14:43:08 -04:00
ec3e1fda06
refactor: refactor all delete helper functions
2021-09-27 14:39:01 -04:00
8e89dde85c
test: enable stack overflow test ( #2618 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-27 14:27:08 +00:00
a55a21c644
chore: Update datafusion ( #2635 )
...
* chore: Update datafusion and sqlparser
* fix: remove STACK_SIZE workaround
* chore: update datafusion_util
* chore: update predicate
* chore: update query_tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-27 14:13:19 +00:00
c5e40b50cc
refactor: delete tests to make it easier to add more tests
2021-09-24 16:35:55 -04:00
0799b5ab91
test: query with selection predicate on deleted data
2021-09-23 16:54:40 -04:00
c35ecf257d
refactor: address review comments
2021-09-23 11:14:20 -04:00
713c7f4820
chore: remove empty line
2021-09-22 17:11:09 -04:00
332649a8ee
refactor: another cleanup due to marco -> function and add a test that exposes bug
2021-09-22 17:09:20 -04:00
10097766d3
refactor: changes to use function instead of marco after merging main
2021-09-22 16:53:51 -04:00
2399a932fb
chore: Merge branch 'main' into ntran/more_delete_tests
2021-09-22 16:47:15 -04:00
3a00d9c70a
refactor: cleanup
2021-09-22 16:42:46 -04:00
400ec93498
test: more delete tests
2021-09-22 16:38:27 -04:00
9a1a3f4de5
refactor: convert `run_tag_keys_test_case` to a function
2021-09-22 13:46:54 +02:00
8cc08d6bd9
refactor: convert `run_table_names_test_case` to a function
2021-09-22 13:46:54 +02:00
2408dea394
refactor: convert `run_read_window_aggregate_test_case` to a function
2021-09-22 13:46:54 +02:00
cdfe49f041
refactor: convert `run_read_group_test_case` to a function
2021-09-22 13:46:54 +02:00
1db13ec437
refactor: convert `run_tag_values_test_case` to a function
2021-09-22 13:46:54 +02:00
20655d10b6
refactor: convert `run_read_window_aggregate_test_case` to a function
2021-09-22 13:46:54 +02:00
b24be8d17d
refactor: convert `run_field_columns_test_case` to a function
2021-09-22 13:46:54 +02:00
6ef8c11036
refactor: convert `run_table_schema_test_case` to a function
2021-09-22 13:46:54 +02:00
70a754a022
refactor: convert `run_sql_test_case` macro to a function
...
This generated less code and speeds up compilation a bit.
2021-09-22 13:46:53 +02:00
d38648952c
chore: Update datafusion ( #2602 )
...
* chore: Update datafusion + other deps
* refactor: update query crate for new async interfaces
* refactor: update server crate for new async interface
* refactor: update query_tests crate for new async interfaces
* refactor: update ioxd and server to use new async interface
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-22 10:33:25 +00:00
ae40d93af4
test: more delete tests
2021-09-21 18:10:12 -04:00
b4b33c378e
test: turn all delete tests on
2021-09-21 15:23:41 -04:00
2f371b6f79
refactor: address review comments
2021-09-21 14:46:24 -04:00
93551bdd1e
fix: all chunks now are applied delete predicates during scan
2021-09-21 12:17:59 -04:00
85989cc8a3
test: add more delete tests and test scenarios
2021-09-20 18:18:08 -04:00
77d84ca5ab
Merge branch 'main' into crepererum/chunk_id
2021-09-20 13:39:05 +00:00
cef5aeee52
refactor: introduce `ChunkId` type
2021-09-20 13:10:41 +02:00
364d245eae
feat: apply negated delete predicates during scan
2021-09-17 16:20:42 -04:00
243cc1f88c
fix: compile error after merge from main
2021-09-16 17:56:33 -04:00
6cfeeb352b
refactor: address review comments
2021-09-16 17:21:06 -04:00
472e8a9e49
fix: fix compile error
2021-09-16 15:02:18 -04:00
Nga Tran
Marco Neumann
Edd Robinson
Raphael Taylor-Davies
dependabot[bot]
Andrew Lamb
kodiakhq[bot]