58f3e2e559
refactor: move delete predicate proto serialization to generated_types ( #3108 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-15 12:02:14 +00:00
3d091208af
refactor: move delete predicate into data_types ( #2731 ) ( #3094 )
...
* refactor: move delete predicate into dml (#2731 )
* refactor: move DeletePredicate to data_types
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-15 10:28:58 +00:00
3485d70409
chore(deps): bump serde_json from 1.0.69 to 1.0.70
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.69 to 1.0.70.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.69...v1.0.70 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-15 08:48:44 +00:00
6320ce6f55
refactor: move delete predicate proto to own package ( #2731 ) ( #3065 )
...
* refactor: move delete predicate proto to own package (#2731 )
* chore: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-08 13:42:26 +00:00
7214fa9d6a
chore(deps): bump serde_json from 1.0.68 to 1.0.69
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.68 to 1.0.69.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.68...v1.0.69 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-08 10:31:33 +00:00
4c9570b519
refactor: move `catalog` protobuf to `preserved_catalog`
...
This makes it clearer what's going since the contained messages are
only for the preserved part, not the in-mem catalog and its management.
2021-11-01 18:07:25 +01:00
c540b40f05
chore(deps): bump tokio from 1.12.0 to 1.13.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.12.0...tokio-1.13.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 11:21:59 +00:00
7cd56cbc56
Merge remote-tracking branch 'origin/main' into er/fix/flux/2691
2021-10-25 13:41:08 -04:00
bc7244c48e
chore: use Rust edition 2021
2021-10-25 10:58:20 +02:00
52cf1a85b9
fix(metadata): Do not report table_names for tables that have no non-null values that match predicate ( #2947 )
...
* fix(metadata): Do not report table_names for tables that have no non-null values that match predicate
* fix: make apply_predicate_to_metadata precise
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-24 11:03:30 +00:00
6683a1e629
Merge branch 'main' into er/fix/flux/2691
2021-10-22 16:28:21 +01:00
84a00cd5b6
refactor: comma
2021-10-22 15:35:24 +01:00
3d9f9ea53e
refactor: use Expr Display impl
2021-10-21 22:43:13 +01:00
0cb9751b0a
refactor: extract _value expr from predicate
2021-10-21 22:26:32 +01:00
a82dc6f5f0
chore: Update datafusion + arrow ( #2903 )
...
* chore: Update datafusion to latest, arrow to 6.0.0
* fix: Update tests
* fix: bubble internal error
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-19 17:14:08 +00:00
c5405fbd5f
chore(deps): bump sqlparser from 0.11.0 to 0.12.0 ( #2867 )
...
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs ) from 0.11.0 to 0.12.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases )
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md )
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.11.0...v0.12.0 )
---
updated-dependencies:
- dependency-name: sqlparser
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-18 14:11:02 +00:00
fbf5539336
chore: merge main to branch
2021-10-11 10:47:10 -04:00
afe34751e7
refactor: split out schema crate ( #2781 )
...
* refactor: split out schema crate
* chore: fix doc
2021-10-11 09:45:08 +00:00
d0a17ca79d
refactor: address Edd's review comments
2021-10-08 16:47:37 -04:00
f7475322a6
chore: merge main to branch, resolve conflicts, and discover an inconsitent bug
2021-10-08 15:50:46 -04:00
f2cdb9531f
chore: cleanup
2021-10-08 14:52:15 -04:00
22d6f11bea
fix: add cols of delete predicates into the schema of scanning columns
2021-10-07 17:37:34 -04:00
c7727f1b5b
chore: Update datafusion + other deps ( #2760 )
...
* chore: Update datafusion and other deps
* fix: fmt
* fix: cleanup workaround
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-07 16:48:11 +00:00
d72a494198
Merge branch 'main' into crepererum/in_mem_expr_part5
2021-10-05 16:20:24 +00:00
825ddaa34c
chore: merge main to branch
2021-10-05 10:12:17 -04:00
bb7a27e5ed
refactor: use proper sets during delete predicate collection
...
We no longer need hacky pointer tricks to de-duplicate delete predicates
when collecting them for catalog checkpoints. This was once required
when the delete predicates didn't implement `Eq` and `Hash` but now it's
all way easier.
2021-10-05 10:37:34 +02:00
1d18758075
refactor: fold `ParseDeletePredicate` into `DeletePredicate`
...
There is no reason to have yet another intermediate type. This also
makes the intermediate parsing functions private.
2021-10-05 09:29:25 +02:00
10c1a72402
refactor: remove unused fields from `DeletePredicate`
2021-10-05 09:29:24 +02:00
ff8e037971
feat: handle delete response
2021-10-04 20:37:53 -04:00
1856d7184c
fix: make stop time inclusive
2021-10-04 14:14:16 -04:00
150e7c222f
chore: enable linting for `predicate` crate
2021-10-04 17:58:40 +02:00
d8dddc358f
fix: typo in error message
...
Co-authored-by: Nga Tran <ntran@influxdata.com>
2021-10-04 16:56:43 +02:00
75ac6e8646
refactor: make `DeletePredicate::range` non-optional
2021-10-04 16:36:20 +02:00
d1835a3eee
fix: doc links
2021-10-04 16:36:20 +02:00
5a5a929b9e
refactor: introduce `DeletePredicate`
...
`DeletePredicate` is a simpler version of `Predicate` that is based on
IOx `DeleteExpr` instead of the full-blown DataFusion `Expr`. This will
allow us to do a couple of things (in follow-up changes):
- Order and de-duplicate delete predicates
- Normalize predicates
- Infallible serialization
- Smaller memory footprint
Note that this change only affects delete expressions. Query expressions
that are supported via the API are not changed. The query subsystem also
still uses the full-featured expressions/predicates (delete
expressions/predicates are converted to the more powerful DataFusion
version on-the-fly).
2021-10-04 16:36:20 +02:00
d1f5209869
chore(deps): bump arrow from 5.4.0 to 5.5.0
...
Bumps [arrow](https://github.com/apache/arrow-rs ) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases )
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md )
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0 )
---
updated-dependencies:
- dependency-name: arrow
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 08:55:38 +00:00
51246e3c8d
docs: remove outdated comment
2021-10-04 10:44:30 +02:00
ec0f256ad7
refactor: `Expr` => `DeleteExpr`
2021-10-04 10:28:09 +02:00
a0bbbdb197
refactor: use dedicated in-memory `Expr` type during IO
...
Place a dedicated `Expr` type between datafusion and protobuf. While
this type is currently only used during serialization, it will be used
within `Predicate` in a follow-up change to enable de-duplication of
predicates and avoid the double parsing of datafusion expressions (we
are already somewhat parsing them when we check for valid delete
predicates).
This change looks larger than it is. In practice it just separates the
conversion "datafusion => protobuf" into "datafusion => IOx => protobuf"
and "protobuf => datafusion" into "protobuf => IOx => datafusion". So
(apart from the error types) this is functionally the same.
2021-10-04 09:59:14 +02:00
154dd4460e
refactor: address review comments
2021-10-01 16:37:22 -04:00
ee94e9038a
test: finalize codin up delete http endpoints and end-to-end tests
2021-10-01 12:15:00 -04:00
53a45d89b7
feat: support HTTP Delete Endpoints
2021-09-30 18:09:17 -04:00
a55a21c644
chore: Update datafusion ( #2635 )
...
* chore: Update datafusion and sqlparser
* fix: remove STACK_SIZE workaround
* chore: update datafusion_util
* chore: update predicate
* chore: update query_tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-27 14:13:19 +00:00
e80d8aa391
chore: update latest datafusion and use its lit_timestamp_nano
2021-09-22 11:30:42 -04:00
2f371b6f79
refactor: address review comments
2021-09-21 14:46:24 -04:00
85989cc8a3
test: add more delete tests and test scenarios
2021-09-20 18:18:08 -04:00
c7e6fffaaa
Merge branch 'main' into ntran/delete_scan
2021-09-20 13:29:47 +00:00
acf698c366
fix: delete predicate sorting
2021-09-20 10:48:32 +02:00
364d245eae
feat: apply negated delete predicates during scan
2021-09-17 16:20:42 -04:00
60a866ddcb
refactor: merge delete predicates into select predicate
2021-09-17 07:52:33 -04:00
Raphael Taylor-Davies
dependabot[bot]
Marco Neumann
Andrew Lamb
Andrew Lamb
Edd Robinson
Nga Tran
kodiakhq[bot]