Commit Graph

4769 Commits (b8f7319704ea7f802843a5e383769d917c9c29ac)

Author SHA1 Message Date
Raphael Taylor-Davies b8f7319704
feat: migrate read buffer metrics to metric crate (#2510)
* feat: migrate read buffer metrics to metric crate

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 19:51:43 +00:00
Andrew Lamb eb72799f0d
chore: Update datafusion (and arrow et al) dependencies (#2509)
* chore: update datafusion and other deps

* fix: Update InfluxRPC frontend with new op types

* fix: Update test output for new column names

* fix: typos and unintended changes

* fix: Update query_tests

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 17:57:25 +00:00
Andrew Lamb cfb198b653
chore: log tracing config (#2506)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 17:03:47 +00:00
Jacob Marble 083fd06d0c
fix: checksum rustup in CI (#2501)
* fix: checksum rustup in CI

* chore: integrate feedback

Co-authored-by: Andrew Lamb <alamb@influxdata.com>

Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 16:28:09 +00:00
Andrew Lamb fa47fb5582
docs: Add usecase to freezable doc string (#2504)
* docs: Add usecase to freezable doc string

* docs: remove mention of during actual update

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 15:28:54 +00:00
Andrew Lamb 88a021da6e
docs: Add instructions for locally seeing distributed tracing (#2507)
* docs: Add instructions for locally seeing distriubuted tracing

* docs: Update CONTRIBUTING.md

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-09-10 15:12:38 +00:00
Raphael Taylor-Davies e07bc85670
feat: enable tracing for http API (#2273) (#2498)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 11:21:02 +00:00
kodiakhq[bot] a715aeb31f
Merge pull request #2503 from influxdata/crepererum/rust_155
chore: Rust 1.55
2021-09-10 11:09:26 +00:00
kodiakhq[bot] f6c0d94991
Merge branch 'main' into crepererum/rust_155 2021-09-10 10:59:59 +00:00
Andrew Lamb ec63321bb0
feat: Less errors on update_database_rules (#2433)
* fix: serialize concurrent database rules updates

* fix: second attempt

* docs: Apply suggestions from code review

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-09-10 10:46:26 +00:00
Marco Neumann 368f0369ee chore: Rust 1.55 2021-09-10 12:36:49 +02:00
Raphael Taylor-Davies eed81e752d
feat: remove deprecated catalog metrics (#2489)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 10:12:04 +00:00
Raphael Taylor-Davies 8a1d759fb7
feat: response classification (#2273) (#2502)
* feat: response classification (#2273)

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 09:05:47 +00:00
kodiakhq[bot] fd49fac7b2
Merge pull request #2500 from influxdata/ntran/refactor_delete
refactor: move ParseDeletePredicate to predicate.rs
2021-09-09 20:58:52 +00:00
Nga Tran 5d2b4a87c3 refactor: use snafu 2021-09-09 16:34:20 -04:00
Nga Tran 1effb11ad9 refactor: move delete parseing work out of influxdb_line_protocol crate 2021-09-09 15:27:17 -04:00
kodiakhq[bot] 33a2c061cd
Merge pull request #2488 from influxdata/ntran/parse_delete_2
feat: Finally have the delete predicate parsed and invoked from client
2021-09-09 18:40:08 +00:00
kodiakhq[bot] faa05f394b
Merge branch 'main' into ntran/parse_delete_2 2021-09-09 18:28:39 +00:00
Nga Tran b4f8fad400 refactor: address review comments and remove uno longer needed dependencies 2021-09-09 14:24:58 -04:00
Raphael Taylor-Davies 44918e4afc
feat: migrate chunk metrics (#2491)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-09 16:02:16 +00:00
Raphael Taylor-Davies 3e3437ba39
fix: test_logging port collision (#2492)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-09 15:49:14 +00:00
Raphael Taylor-Davies e5e68b1978
feat: add RecorderCollection (#2494)
* feat: add MetricRecorder

* chore: rename MetricRecorder to RecorderCollection

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-09 15:38:21 +00:00
kodiakhq[bot] ede83befc9
Merge pull request #2496 from influxdata/jgm-fix-perf
chore: fix perf again, after another protobuf change
2021-09-09 15:28:18 +00:00
kodiakhq[bot] 02b5361b7b
Merge branch 'main' into jgm-fix-perf 2021-09-09 15:19:18 +00:00
kodiakhq[bot] 0909d761f3
Merge pull request #2495 from influxdata/crepererum/remove_process_clock
refactor: remove process clock
2021-09-09 15:18:39 +00:00
kodiakhq[bot] 76271a141a
Merge branch 'main' into crepererum/remove_process_clock 2021-09-09 15:08:40 +00:00
Jacob Marble 0407d5228e chore: fix perf again, after another protobuf change 2021-09-09 08:02:35 -07:00
kodiakhq[bot] a60c0f4d53
Merge pull request #2482 from influxdata/crepererum/fix_job_metrics
fix: job metrics
2021-09-09 15:02:04 +00:00
Marco Neumann 4d6ec4bfe6 refactor: remove process clock
The process clock is a leftover from the pre-Kafka writer buffer design
and is no longer required.
2021-09-09 16:55:48 +02:00
kodiakhq[bot] a9e2ed4c14
Merge branch 'main' into crepererum/fix_job_metrics 2021-09-09 14:53:00 +00:00
Raphael Taylor-Davies 3cee899f77
feat: migrate catalog timestamp summary to `metric` crate (#2486)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-09 14:52:25 +00:00
Marco Neumann a5d4d954fb fix: increase job duration histogram range
The default upper limit of 10s is too tight for many jobs. This now
increases the histogram range to 5000s (no joke, we've seen jobs w/ over
40min run time, even though that shouldn't happen).
2021-09-09 16:48:21 +02:00
Marco Neumann 40d3f53aee feat: add DB and table name to job metrics 2021-09-09 16:37:44 +02:00
Marco Neumann a6416d145f fix: do not report error when prometheus metric was registered but not used 2021-09-09 16:37:44 +02:00
Marco Neumann 0a31f5f2e5 fix: fix job metrics naming
For duration historgrams, the exporter takes care of the correct suffix
depending on the resolution used by it. For example the prometheus
exporter will use a `..._seconds` metric to encode the historgram. The
IOx internal metric should therefore NOT append any resolution. This
then removes the `_nanoseconds` suffix, renaming the externally visible
metric from

```text
influxdb_iox_job_completed_{cpu,wall}_nanoseconds_seconds
```

to

```text
influxdb_iox_job_completed_{cpu,wall}_seconds
```
2021-09-09 16:37:44 +02:00
Jacob Marble c415d357cc
fix: perf broken after Protobuf changes (#2479)
* fix: perf broken after proto changes

* feat: add perf to CI

* chore: try different python version selector script

* chore: several fixes to CI

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-09 14:24:24 +00:00
Raphael Taylor-Davies 9de12745e7
feat: migrate lock metrics to metric crate (#2481)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-09 08:56:19 +00:00
Nga Tran 19848eb132 chore: turn a test on 2021-09-08 17:30:55 -04:00
Nga Tran 00df7b064c feat: finally have the delete predicate parsed 2021-09-08 17:30:10 -04:00
kodiakhq[bot] 29b1041305
Merge pull request #2474 from influxdata/crepererum/issue2455c
feat: auto-creation of sequencers by write buffer
2021-09-08 09:30:42 +00:00
Marco Neumann bbb8898d36 refactor: make writer buffer auto-creation types nicer to read 2021-09-08 11:13:48 +02:00
Marco Neumann 0a823cd75c refactor: simplify error handling 2021-09-07 18:24:59 +02:00
Marco Neumann 2cc1297c96 fix: typos
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-09-07 18:24:59 +02:00
Marco Neumann b4fe829da0 fix: docs 2021-09-07 18:24:59 +02:00
Marco Neumann c0cc239781 fix: improve Kafka error handling 2021-09-07 18:24:59 +02:00
Marco Neumann 801cf08be7 feat: auto-creation of sequencers by write buffer
For Kafka, that basically means that we create a topic if it doesn't
exist yet.

Closed #2455.
Fixes #2189.
2021-09-07 18:24:57 +02:00
Marco Neumann 924e460bf7 feat: sequencer auto-creation for mocked write buffer 2021-09-07 18:18:20 +02:00
Marco Neumann e7cf91bfc4 refactor: more structures write buffer auto-creation config 2021-09-07 18:18:20 +02:00
Marco Neumann ecbd0165fc feat: prepare auto-creation of sequencers for mocked write buffer 2021-09-07 18:18:20 +02:00
Marco Neumann 82f9750ba7 test: prepare write buffer test suite for failable reader and writer creation
This is required to work w/ sequencer auto-creation.
2021-09-07 18:18:20 +02:00