Commit Graph

5266 Commits (1718b55283a5abcbe00b460f975ca69087e9ad50)

Author SHA1 Message Date
Nga Tran 1718b55283 chore: Merge branch 'ntran/count_star' of https://github.com/influxdata/influxdb_iox into ntran/count_star 2021-10-11 10:47:38 -04:00
Nga Tran fbf5539336 chore: merge main to branch 2021-10-11 10:47:10 -04:00
Marco Neumann 470de36f3b
Merge pull request #2794 from influxdata/crepererum/issue1846
refactor: remove `time_closed`
2021-10-11 16:12:33 +02:00
Marco Neumann c4a2641764 refactor: remove `time_closed`
The "time closed" is a leftover from an old lifecycle system, where
chunks moved through the system (open=>closed=>persisted) without being
merged. Now we have the compaction as well as the split query for
persistence that can merge chunks, so a single "time closed" doesn't
make sense any longer. So in fact it is `None` for many chunks and is
also not persisted. Also the current lifecycle policy doesn't use this
value. So let's just remove it.

Closes #1846.
2021-10-11 15:49:34 +02:00
Nga Tran d13e61c201
fix: Apply suggestions from code review
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-10-11 09:33:23 -04:00
Raphael Taylor-Davies afe34751e7
refactor: split out schema crate (#2781)
* refactor: split out schema crate

* chore: fix doc
2021-10-11 09:45:08 +00:00
Raphael Taylor-Davies f35a49edd0
refactor: move Sequence to data_types (#2780)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 09:23:00 +00:00
dependabot[bot] 435a8aec91
chore(deps): bump reqwest from 0.11.4 to 0.11.5 (#2789)
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.4...v0.11.5)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-11 09:14:15 +00:00
dependabot[bot] 7cf3cf252a
chore(deps): bump tracing from 0.1.28 to 0.1.29 (#2788)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.28 to 0.1.29.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.28...tracing-0.1.29)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-11 09:05:33 +00:00
kodiakhq[bot] a6aa7fa0c8
Merge pull request #2787 from influxdata/dependabot/cargo/ahash-0.7.5
chore(deps): bump ahash from 0.7.4 to 0.7.5
2021-10-11 08:45:07 +00:00
dependabot[bot] 42d4ad61e1
chore(deps): bump ahash from 0.7.4 to 0.7.5
Bumps [ahash](https://github.com/tkaitchuck/ahash) from 0.7.4 to 0.7.5.
- [Release notes](https://github.com/tkaitchuck/ahash/releases)
- [Commits](https://github.com/tkaitchuck/ahash/commits)

---
updated-dependencies:
- dependency-name: ahash
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:36:50 +00:00
kodiakhq[bot] d6bde08f7e
Merge pull request #2785 from influxdata/dependabot/cargo/ouroboros-0.13.0
chore(deps): bump ouroboros from 0.12.0 to 0.13.0
2021-10-11 08:34:54 +00:00
dependabot[bot] 71a9aa2120
chore(deps): bump ouroboros from 0.12.0 to 0.13.0
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:24:27 +00:00
kodiakhq[bot] 185a78862b
Merge pull request #2783 from influxdata/dependabot/cargo/tracing-subscriber-0.2.25
chore(deps): bump tracing-subscriber from 0.2.24 to 0.2.25
2021-10-11 08:22:24 +00:00
dependabot[bot] 3f921d43e6
chore(deps): bump tracing-subscriber from 0.2.24 to 0.2.25
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.24 to 0.2.25.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.24...tracing-subscriber-0.2.25)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:14:11 +00:00
kodiakhq[bot] d6f2689d15
Merge pull request #2782 from influxdata/dependabot/cargo/thiserror-1.0.30
chore(deps): bump thiserror from 1.0.29 to 1.0.30
2021-10-11 08:10:05 +00:00
dependabot[bot] 1327735537
chore(deps): bump thiserror from 1.0.29 to 1.0.30
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.29...1.0.30)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:01:21 +00:00
kodiakhq[bot] 94852f9c44
Merge pull request #2786 from influxdata/dependabot/cargo/assert_cmd-2.0.2
chore(deps): bump assert_cmd from 2.0.1 to 2.0.2
2021-10-11 07:59:35 +00:00
dependabot[bot] 01a44899fd
chore(deps): bump assert_cmd from 2.0.1 to 2.0.2
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd) from 2.0.1 to 2.0.2.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases)
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.1...v2.0.2)

---
updated-dependencies:
- dependency-name: assert_cmd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 01:45:35 +00:00
Nga Tran d0a17ca79d refactor: address Edd's review comments 2021-10-08 16:47:37 -04:00
Nga Tran f7475322a6 chore: merge main to branch, resolve conflicts, and discover an inconsitent bug 2021-10-08 15:50:46 -04:00
Nga Tran bea310db76 chore: remove comments 2021-10-08 14:58:06 -04:00
Nga Tran 2556639bb5 chore: more cleanup 2021-10-08 14:57:22 -04:00
Nga Tran f2cdb9531f chore: cleanup 2021-10-08 14:52:15 -04:00
Nga Tran adbcd85c26 fix: fully fix 2745 2021-10-08 14:37:34 -04:00
kodiakhq[bot] 0666e0908c
Merge pull request #2773 from influxdata/cn/chunk-addr-smaller
fix: Remove some calls to iox_object_store.database_name
2021-10-08 17:34:58 +00:00
kodiakhq[bot] 559a7e0221
Merge branch 'main' into cn/chunk-addr-smaller 2021-10-08 17:26:20 +00:00
Andrew Lamb 2072b4066e
feat: Implement support for `_measurement` predicate in gRPC plans (#2772)
* feat: Implement filtering for _measurement in general purpose gRPC plans

* docs: fixup docstrings

* fix: fmt

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-08 17:25:33 +00:00
kodiakhq[bot] 7f3fcd99c4
Merge pull request #2770 from influxdata/crepererum/better_chunk_id_dbg
feat: improve `Debug`/`Display` for test `ChunkId`s
2021-10-08 15:41:28 +00:00
kodiakhq[bot] 8b7fd77abe
Merge branch 'main' into crepererum/better_chunk_id_dbg 2021-10-08 15:33:12 +00:00
Andrew Lamb ca38eafe20
refactor: Reduce string comparisons in gRPC expr tests (#2765)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-08 15:03:03 +00:00
kodiakhq[bot] b887a0896c
Merge pull request #2769 from influxdata/er/fix/read_buffer/negation_pred
fix: ensure correct PredicateMatch for columns
2021-10-08 14:07:57 +00:00
Carol (Nichols || Goulding) fbe76935f4 fix: Remove some calls to iox_object_store.database_name 2021-10-08 09:50:14 -04:00
Marco Neumann 64bda1fc08 feat: improve `Debug`/`Display` for test `ChunkId`s 2021-10-08 13:55:56 +02:00
Edd Robinson 5b71a3668a
Merge branch 'main' into er/fix/read_buffer/negation_pred 2021-10-08 12:47:04 +01:00
Edd Robinson 04a7ddd34a fix: ensure correct PredicateMatch for columns 2021-10-08 12:46:24 +01:00
kodiakhq[bot] 222f452a8a
Merge pull request #2768 from influxdata/crepererum/issue2225_b
fix: review comments of #2767
2021-10-08 10:45:50 +00:00
Marco Neumann f8b5c0ee50 fix: remove obsolete TODO 2021-10-08 12:36:23 +02:00
Marco Neumann d3de6bb6e4 refactor: `max_persisted_timestamp` => `flush_timestamp`
There might be data left before this timestamp that wasn't persisted
(e.g. incoming data while the persistence was running).
2021-10-08 12:36:23 +02:00
kodiakhq[bot] f59ee631c1
Merge pull request #2767 from influxdata/crepererum/issue2225
refactor: "min unpersisted ts" => "max persisted ts"
2021-10-08 10:35:43 +00:00
Marco Neumann 63a932fa37 refactor: "min unpersisted ts" => "max persisted ts"
Store the "maximum persisted timestamp" instead of the "minimum
unpersisted timestamp". This avoids the need to calculate the next
timestamp from the current one (which was done via "max TS + 1ns").

The old calculation was prone to overflow panics. Since the
timestamps in this calculation originate from user-provided data (and
not the wall clock), this was an easy DoS vector that could be triggered
via the following line protocol:

```text
table_1 foo=1 <i64::MAX>
```

which is

```text
table_1 foo=1 9223372036854775807
```

Bonus points: the timestamp persisted in the partition
checkpoints is now the very same that was used by the split query during
persistence. Consistence FTW!

Fixes #2225.
2021-10-08 11:52:49 +02:00
kodiakhq[bot] 8817ee8d61
Merge pull request #2759 from influxdata/crepererum/issue2627
fix: interaction of lifecycle actions and delete predicates
2021-10-08 07:43:09 +00:00
kodiakhq[bot] 001ed36da4
Merge branch 'main' into crepererum/issue2627 2021-10-08 07:35:08 +00:00
Nga Tran 22d6f11bea fix: add cols of delete predicates into the schema of scanning columns 2021-10-07 17:37:34 -04:00
Andrew Lamb e590ac4da2
fix: remove outdated "supported predicate" check in gRPC planner (#2763) 2021-10-07 20:34:05 +00:00
Andrew Lamb c7727f1b5b
chore: Update datafusion + other deps (#2760)
* chore: Update datafusion and other deps

* fix: fmt

* fix: cleanup workaround

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-07 16:48:11 +00:00
kodiakhq[bot] 4ca9120192
Merge pull request #2682 from influxdata/cn/uuid
feat: Add UUID to databases
2021-10-07 14:41:16 +00:00
kodiakhq[bot] 0b24564309
Merge branch 'main' into cn/uuid 2021-10-07 14:31:48 +00:00
Carol (Nichols || Goulding) 15b396c720 fix: Set a UUID if none was read from object storage on server startup
This enables a smoother transition to this state when this gets
deployed.
2021-10-07 10:19:17 -04:00
Carol (Nichols || Goulding) 169f2499d3 fix: Use bytes for UUID in the protobuf instead of string 2021-10-07 10:19:17 -04:00