Commit Graph

641 Commits (e911cf9ac172f09e3716d23bb8d0ec95c9a78c4c)

Author SHA1 Message Date
Andrew Lamb 0568452a0c
chore: Update datafusion (#2838)
* chore: update datafusion version

* refactor: Update to use new datafusion apis

* fix: do not upgrade other packages
2021-10-13 20:51:19 +00:00
Raphael Taylor-Davies 8a82f92c5d
refactor: add TimeProvider abstraction (#2722) (#2815)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-12 21:19:03 +00:00
Raphael Taylor-Davies 3dfe400e6b
feat: migrate write path to TimeProvider (#2722) (#2807) 2021-10-12 12:09:08 +00:00
Raphael Taylor-Davies 0554173684
feat: migrate write buffer to TimeProvider (#2722) (#2804)
* feat: migrate write buffer to TimeProvider (#2722)

* chore: review feedback

Co-authored-by: Marco Neumann <marco@crepererum.net>

Co-authored-by: Marco Neumann <marco@crepererum.net>
2021-10-12 10:32:34 +00:00
Raphael Taylor-Davies b39e01f7ba
feat: migrate PersistenceWindows to TimeProvider (#2722) (#2798) 2021-10-11 20:40:00 +00:00
dependabot[bot] 49c63d35b1
chore(deps): bump cache_loader_async from 0.1.1 to 0.1.2
Bumps [cache_loader_async](https://github.com/ZeroTwo-Bot/cache-loader-async-rs) from 0.1.1 to 0.1.2.
- [Release notes](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/releases)
- [Changelog](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ZeroTwo-Bot/cache-loader-async-rs/commits)

---
updated-dependencies:
- dependency-name: cache_loader_async
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 15:02:12 +00:00
Raphael Taylor-Davies 7bd32a4916
feat: add time provider abstraction (#2722) (#2792)
* feat: add time provider abstraction (#2722)

* chore: tests

* chore: more tests

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 15:00:29 +00:00
dependabot[bot] 61190a9d08
chore(deps): bump predicates from 2.0.2 to 2.0.3
Bumps [predicates](https://github.com/assert-rs/predicates-rs) from 2.0.2 to 2.0.3.
- [Release notes](https://github.com/assert-rs/predicates-rs/releases)
- [Changelog](https://github.com/assert-rs/predicates-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/predicates-rs/compare/v2.0.2...v2.0.3)

---
updated-dependencies:
- dependency-name: predicates
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 14:32:08 +00:00
Marco Neumann 24ae269b3a refactor: cancel executor jobs on drop
Our executor is not meant as a fire-and-forget system. Instead the
submitter should always poll the result. Dropping the receiver side (aka
the job handle) should cancel the job.
2021-10-11 16:13:04 +02:00
Raphael Taylor-Davies afe34751e7
refactor: split out schema crate (#2781)
* refactor: split out schema crate

* chore: fix doc
2021-10-11 09:45:08 +00:00
Raphael Taylor-Davies f35a49edd0
refactor: move Sequence to data_types (#2780)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 09:23:00 +00:00
dependabot[bot] 435a8aec91
chore(deps): bump reqwest from 0.11.4 to 0.11.5 (#2789)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.4...v0.11.5)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 09:14:15 +00:00
dependabot[bot] 7cf3cf252a
chore(deps): bump tracing from 0.1.28 to 0.1.29 (#2788)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.28 to 0.1.29.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.28...tracing-0.1.29)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-11 09:05:33 +00:00
dependabot[bot] 42d4ad61e1
chore(deps): bump ahash from 0.7.4 to 0.7.5
Bumps [ahash](https://github.com/tkaitchuck/ahash) from 0.7.4 to 0.7.5.
- [Release notes](https://github.com/tkaitchuck/ahash/releases)
- [Commits](https://github.com/tkaitchuck/ahash/commits)

---
updated-dependencies:
- dependency-name: ahash
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:36:50 +00:00
dependabot[bot] 71a9aa2120
chore(deps): bump ouroboros from 0.12.0 to 0.13.0
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:24:27 +00:00
dependabot[bot] 3f921d43e6
chore(deps): bump tracing-subscriber from 0.2.24 to 0.2.25
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.24 to 0.2.25.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.24...tracing-subscriber-0.2.25)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:14:11 +00:00
dependabot[bot] 1327735537
chore(deps): bump thiserror from 1.0.29 to 1.0.30
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.29...1.0.30)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:01:21 +00:00
dependabot[bot] 01a44899fd
chore(deps): bump assert_cmd from 2.0.1 to 2.0.2
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd) from 2.0.1 to 2.0.2.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases)
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.1...v2.0.2)

---
updated-dependencies:
- dependency-name: assert_cmd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 01:45:35 +00:00
Andrew Lamb c7727f1b5b
chore: Update datafusion + other deps (#2760)
* chore: Update datafusion and other deps

* fix: fmt

* fix: cleanup workaround

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-07 16:48:11 +00:00
Raphael Taylor-Davies d4bc33b921
refactor: use RWLock instead of TaskTracker for query exec shutdown (#2761) 2021-10-07 14:18:40 +00:00
Raphael Taylor-Davies fb9119fa4c
feat: remove serde from data types (#2652) 2021-10-07 08:58:06 +00:00
Marco Neumann 63d74be490 refactor: make `ChunkId` a UUID 2021-10-07 10:23:27 +02:00
Raphael Taylor-Davies 39157828b1
feat: remove remaining usages of Instant (#2722) (#2749)
* feat: remove remaining usages of Instant (#2722)

* chore: review feedback

Co-authored-by: Andrew Lamb <alamb@influxdata.com>

Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-06 16:44:02 +00:00
Raphael Taylor-Davies ce5b24e65d
refactor: use DateTime<Utc> in PersistenceWindows (#2722) (#2743)
* refactor: use DateTime<Utc> in PersistenceWindows (#2722)

* chore: fix benchmark

* chore: fmt

* chore: review feedback
2021-10-06 09:39:32 +00:00
dependabot[bot] 7a6f381e69
chore(deps): bump arrow-flight from 5.4.0 to 5.5.0
Bumps [arrow-flight](https://github.com/apache/arrow-rs) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0)

---
updated-dependencies:
- dependency-name: arrow-flight
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 13:27:26 +00:00
dependabot[bot] 6d9205d87c chore(deps): bump parquet from 5.4.0 to 5.5.0
Bumps [parquet](https://github.com/apache/arrow-rs) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0)

---
updated-dependencies:
- dependency-name: parquet
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 12:15:11 +01:00
kodiakhq[bot] 5060bf71fe
Merge branch 'main' into crepererum/in_mem_expr_part1 2021-10-04 08:45:50 +00:00
dependabot[bot] 425ecebf9d
chore(deps): bump ouroboros from 0.11.1 to 0.12.0
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.11.1 to 0.12.0.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 08:25:32 +00:00
dependabot[bot] b33e1386e0
chore(deps): bump prometheus from 0.12.0 to 0.13.0
Bumps [prometheus](https://github.com/tikv/rust-prometheus) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/tikv/rust-prometheus/releases)
- [Changelog](https://github.com/tikv/rust-prometheus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tikv/rust-prometheus/compare/v0.12.0...v0.13.0)

---
updated-dependencies:
- dependency-name: prometheus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 08:10:07 +00:00
Marco Neumann a0bbbdb197 refactor: use dedicated in-memory `Expr` type during IO
Place a dedicated `Expr` type between datafusion and protobuf. While
this type is currently only used during serialization, it will be used
within `Predicate` in a follow-up change to enable de-duplication of
predicates and avoid the double parsing of datafusion expressions (we
are already somewhat parsing them when we check for valid delete
predicates).

This change looks larger than it is. In practice it just separates the
conversion "datafusion => protobuf" into "datafusion => IOx => protobuf"
and "protobuf => datafusion" into "protobuf => IOx => datafusion". So
(apart from the error types) this is functionally the same.
2021-10-04 09:59:14 +02:00
dependabot[bot] 8893b483a3
chore(deps): bump smallvec from 1.6.1 to 1.7.0
Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/servo/rust-smallvec/releases)
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.6.1...v1.7.0)

---
updated-dependencies:
- dependency-name: smallvec
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 07:57:00 +00:00
dependabot[bot] 44d7479db1
chore(deps): bump cloud-storage from 0.10.2 to 0.10.3
Bumps [cloud-storage](https://github.com/ThouCheese/cloud-storage-rs) from 0.10.2 to 0.10.3.
- [Release notes](https://github.com/ThouCheese/cloud-storage-rs/releases)
- [Changelog](https://github.com/ThouCheese/cloud-storage-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ThouCheese/cloud-storage-rs/commits)

---
updated-dependencies:
- dependency-name: cloud-storage
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 01:43:09 +00:00
Nga Tran 154dd4460e refactor: address review comments 2021-10-01 16:37:22 -04:00
Nga Tran bd22c73b8a chore: Merge branch 'main' into ntran/delete_endpoint 2021-10-01 13:33:39 -04:00
Marko Mikulicic a1f91207b3
fix: Fix RUSTSEC-2021-0119
Update `nix` dependency which includes a fix for RUSTSEC-2021-0119
2021-10-01 09:08:47 +02:00
Nga Tran 53a45d89b7 feat: support HTTP Delete Endpoints 2021-09-30 18:09:17 -04:00
Carol (Nichols || Goulding) edd6c12e93 fix: Add logging when retrying AWS requests 2021-09-29 08:19:32 -04:00
Carol (Nichols || Goulding) 9cf343db08 refactor: Use loop instead of futures_retry 2021-09-29 08:19:32 -04:00
Carol (Nichols || Goulding) 5c78b7d5ae refactor: Extract an s3_request method that handles retries 2021-09-29 08:19:32 -04:00
Raphael Taylor-Davies 86cee568d5
feat: use upstream pbjson (#2650)
* feat: use upstream pbjson

* chore: fmt
2021-09-28 16:29:26 +00:00
Raphael Taylor-Davies daa2ec2f4c
revert: "fix: Revert "feat: remove routerify (#2586)" (#2623)" (#2649)
This reverts commit 18941fcbff.
2021-09-28 14:42:17 +00:00
dependabot[bot] 82de0af79b
chore(deps): bump libc from 0.2.102 to 0.2.103
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.102 to 0.2.103.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.102...0.2.103)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-27 19:46:32 +00:00
Andrew Lamb a55a21c644
chore: Update datafusion (#2635)
* chore: Update datafusion and sqlparser

* fix: remove STACK_SIZE workaround

* chore: update datafusion_util

* chore: update predicate

* chore: update query_tests

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-27 14:13:19 +00:00
dependabot[bot] 8813801e67
chore(deps): bump ouroboros from 0.10.1 to 0.11.1
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.10.1 to 0.11.1.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-27 01:27:24 +00:00
Raphael Taylor-Davies c427f84848
feat: correct pbjson serialization of Timestamp and Duration (#2620)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-24 13:20:50 +00:00
Andrew Lamb 18941fcbff
fix: Revert "feat: remove routerify (#2586)" (#2623)
This reverts commit 9fba7b0ba0.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-24 09:50:24 +00:00
Raphael Taylor-Davies b46541406a
chore: update itertools (#2615)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-22 18:25:35 +00:00
Raphael Taylor-Davies 92e6173f49
feat: implement jaeger-agent protocol directly (#2607)
* feat: implement jaeger-agent protocol directly

* chore: review feedback

* fix: remove jaeger feature flag
2021-09-22 17:30:37 +00:00
Nga Tran e80d8aa391 chore: update latest datafusion and use its lit_timestamp_nano 2021-09-22 11:30:42 -04:00
Andrew Lamb d38648952c
chore: Update datafusion (#2602)
* chore: Update datafusion + other deps

* refactor: update query crate for new async interfaces

* refactor: update server crate for new async interface

* refactor: update query_tests crate for new async interfaces

* refactor: update ioxd and server to use new async interface

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-22 10:33:25 +00:00