Commit Graph

5032 Commits (c107434d2008694461499b38257f79be34b7d054)

Author SHA1 Message Date
Nga Tran 19848eb132 chore: turn a test on 2021-09-08 17:30:55 -04:00
Nga Tran 00df7b064c feat: finally have the delete predicate parsed 2021-09-08 17:30:10 -04:00
kodiakhq[bot] 29b1041305
Merge pull request #2474 from influxdata/crepererum/issue2455c
feat: auto-creation of sequencers by write buffer
2021-09-08 09:30:42 +00:00
Marco Neumann bbb8898d36 refactor: make writer buffer auto-creation types nicer to read 2021-09-08 11:13:48 +02:00
Marco Neumann 0a823cd75c refactor: simplify error handling 2021-09-07 18:24:59 +02:00
Marco Neumann 2cc1297c96 fix: typos
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-09-07 18:24:59 +02:00
Marco Neumann b4fe829da0 fix: docs 2021-09-07 18:24:59 +02:00
Marco Neumann c0cc239781 fix: improve Kafka error handling 2021-09-07 18:24:59 +02:00
Marco Neumann 801cf08be7 feat: auto-creation of sequencers by write buffer
For Kafka, that basically means that we create a topic if it doesn't
exist yet.

Closed #2455.
Fixes #2189.
2021-09-07 18:24:57 +02:00
Marco Neumann 924e460bf7 feat: sequencer auto-creation for mocked write buffer 2021-09-07 18:18:20 +02:00
Marco Neumann e7cf91bfc4 refactor: more structures write buffer auto-creation config 2021-09-07 18:18:20 +02:00
Marco Neumann ecbd0165fc feat: prepare auto-creation of sequencers for mocked write buffer 2021-09-07 18:18:20 +02:00
Marco Neumann 82f9750ba7 test: prepare write buffer test suite for failable reader and writer creation
This is required to work w/ sequencer auto-creation.
2021-09-07 18:18:20 +02:00
Marco Neumann f1864813f4 docs: document write buffer test suite 2021-09-07 18:18:20 +02:00
Marco Neumann 2ea3b600d0 test: harden `assert_reader_content` a bit
- entries should be sorted by the stream, there is no need to sort the
  results
- ensure that there are no leftover entries in the stream by asserting
  that it is "pending"
2021-09-07 18:18:20 +02:00
Marco Neumann d5662328b0 refactor: `n_sequencers` should be non-zero 2021-09-07 18:18:20 +02:00
kodiakhq[bot] 92b7322e84
Merge pull request #2448 from influxdata/ntran/delete_predicates
feat: API and framework for Delete parsing and testing
2021-09-07 15:34:23 +00:00
Nga Tran dbe4bcff22 chore: merge main to branch 2021-09-07 10:54:59 -04:00
Nga Tran 9ee1bdeeb9 refactor: address review comments 2021-09-07 10:24:38 -04:00
kodiakhq[bot] d5939fa91d
Merge pull request #2258 from influxdata/crepererum/replay_prune_rows
feat: row-level pruning during replay
2021-09-07 10:09:18 +00:00
Marco Neumann 6d5fb8b53d refactor: simplify nested code 2021-09-07 12:01:06 +02:00
Marco Neumann 31cbb646b9 feat: skip individual rows during replay based on timestamp 2021-09-07 11:44:52 +02:00
Marco Neumann c31d837915 refactor: make `TableBatch::timestamps` public 2021-09-07 11:44:52 +02:00
kodiakhq[bot] 7f0cd00e3a
Merge pull request #2476 from influxdata/crepererum/lint_data_gen
chore: `iox_data_generator` QA
2021-09-07 09:13:31 +00:00
Marco Neumann bb4fba0c4c chore: `iox_data_generator` QA
- Move main binary to `src/bin`. It's easier to reason about when all
  binaries are in a single directory and the other files in `src` just
  belong to the lib. Note that `cargo run [-p iox_data_generator]` still
  works.
- Enable lints that we use elsewhere. Fix the few issues that were found
  by this (e.g. broken intradoc links).
2021-09-07 11:05:09 +02:00
kodiakhq[bot] 3731658b74
Merge pull request #2475 from influxdata/dom/arrow-dep-management-client
build: feature-gate arrow dep for mgmt client
2021-09-06 13:30:20 +00:00
Edd Robinson 66525a4489
Merge branch 'main' into dom/arrow-dep-management-client 2021-09-06 14:21:48 +01:00
Dom dd88b5ed1d build: feature-gate arrow dep for mgmt client
Avoid pulling in & compiling arrow for consumers of the managemnt
client.

PR #2098 added arrow_util as a dependency of this crate which in turn
pulls in all of arrow, however it is only used in the flight client.

This commit feature-gates arrow_util behind "flight" with the rest of
the arrow deps.
2021-09-06 14:11:20 +01:00
kodiakhq[bot] ac5ccd1f35
Merge pull request #2451 from influxdata/crepererum/issue2416
feat: expose job metrics
2021-09-06 13:06:37 +00:00
Marco Neumann fe0df2ab0c fix: job metric race condition 2021-09-06 14:33:59 +02:00
Marco Neumann 998bafcd85 fix: typo
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-09-06 13:39:22 +02:00
Marco Neumann 77287ad228 feat: rework job metrics to be push-based, add wall/cpu time histograms 2021-09-06 13:39:22 +02:00
Marco Neumann e6f12f965c feat: expose job metrics
Closes #2416.
2021-09-06 13:39:22 +02:00
Marco Neumann 04a301cc64 refactor: make job descriptions a static string 2021-09-06 13:39:22 +02:00
kodiakhq[bot] 28e655f38b
Merge pull request #2470 from influxdata/dependabot/cargo/tokio-1.11.0
chore(deps): bump tokio from 1.10.1 to 1.11.0
2021-09-06 09:27:14 +00:00
kodiakhq[bot] f6e040df3d
Merge branch 'main' into dependabot/cargo/tokio-1.11.0 2021-09-06 09:18:55 +00:00
Raphael Taylor-Davies a4b0cbc0e7
feat: migrate jemalloc metrics to `metric` crate (#2435)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-06 09:18:27 +00:00
dependabot[bot] b67610d9b9
chore(deps): bump tokio from 1.10.1 to 1.11.0
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.10.1 to 1.11.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.10.1...tokio-1.11.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 09:11:38 +00:00
kodiakhq[bot] 0a0afa9485
Merge pull request #2469 from influxdata/dependabot/cargo/parquet-5.3.0
chore(deps): bump parquet from 5.2.0 to 5.3.0
2021-09-06 09:09:00 +00:00
kodiakhq[bot] 06311970fa
Merge branch 'main' into dependabot/cargo/parquet-5.3.0 2021-09-06 09:00:27 +00:00
kodiakhq[bot] a72c2a6192
Merge pull request #2472 from influxdata/dependabot/cargo/structopt-0.3.23
chore(deps): bump structopt from 0.3.22 to 0.3.23
2021-09-06 08:59:47 +00:00
dependabot[bot] ccfd6ad3da
chore(deps): bump parquet from 5.2.0 to 5.3.0
Bumps [parquet](https://github.com/apache/arrow-rs) from 5.2.0 to 5.3.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.3.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.2.0...5.3.0)

---
updated-dependencies:
- dependency-name: parquet
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 08:52:30 +00:00
kodiakhq[bot] 0fd92c8b45
Merge branch 'main' into dependabot/cargo/structopt-0.3.23 2021-09-06 08:51:32 +00:00
kodiakhq[bot] 78bc638aeb
Merge pull request #2473 from influxdata/dependabot/cargo/parking_lot-0.11.2
chore(deps): bump parking_lot from 0.11.1 to 0.11.2
2021-09-06 08:50:33 +00:00
Nga Tran 9de3b79a90 refactor: more cleanup 2021-09-06 01:45:47 -04:00
Nga Tran de0bd80c3d refactor: cleanup 2021-09-06 01:07:07 -04:00
Nga Tran 4801b2c238 feat: Have the ParseDelete message and its corresponding ProvidedParseDelete struct ready for building delete parser 2021-09-06 00:13:59 -04:00
dependabot[bot] b1bb390893
chore(deps): bump parking_lot from 0.11.1 to 0.11.2
Bumps [parking_lot](https://github.com/Amanieu/parking_lot) from 0.11.1 to 0.11.2.
- [Release notes](https://github.com/Amanieu/parking_lot/releases)
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Amanieu/parking_lot/compare/0.11.1...0.11.2)

---
updated-dependencies:
- dependency-name: parking_lot
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 01:18:24 +00:00
dependabot[bot] 2cf9505029
chore(deps): bump structopt from 0.3.22 to 0.3.23
Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.22 to 0.3.23.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.22...v0.3.23)

---
updated-dependencies:
- dependency-name: structopt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 01:17:41 +00:00
Nga Tran d8b3208896 refactor: sstart building parser in the client side 2021-09-03 17:56:45 -04:00