e32fa43a53
docs: Add note about implication of write buffer errors
2021-05-05 10:55:12 -04:00
7d5c988fba
feat: Actually route SequencedEntry to the Write Buffer, if present
...
Connects to #1157 .
Rearrange some code and comments to be consistent with the design. Make
some more places not care whether they're getting an owned or borrowed
SequencedEntry.
2021-05-05 10:55:11 -04:00
54c5f984d5
fix: Use stdlib's path manipulation rather than format
...
The syntax highlighting in my editor broke because of the unmatched
double quote, which got me to look a bit closer at this test. These
tests would have failed on Windows.
2021-05-05 10:55:11 -04:00
231abd221f
refactor: Extract a TestDbBuilder
2021-05-05 10:55:11 -04:00
62dfb47825
refactor: Reorganize test imports
2021-05-05 10:55:11 -04:00
14127ca13b
docs: fix greater than / less than comment
...
Segments will always be less-than-or-equal-to the configured maximum
size, not greater-than-or-equal-to.
2021-05-05 15:24:09 +01:00
dab086821c
Merge pull request #1409 from influxdata/crepererum/issue1379
...
feat: implement parquet metadata handling
2021-05-05 13:30:23 +00:00
7d2a48eb3b
ci: disable incremental compilation
...
This is to avoid overhead. We are not preserving these files anyway.
2021-05-05 13:30:25 +02:00
1f42eb89cd
feat: implement parquet metadata handling
...
Closes #1379 and contributes to #1380 .
2021-05-05 13:29:16 +02:00
80d78c2a67
feat: re-export matching `parquet_format` in `arrow_deps`
2021-05-05 13:29:16 +02:00
056c29aaa2
feat: add a way to retrieve timestamp range from parquet chunk
2021-05-05 13:29:16 +02:00
c54109113e
feat: add a way to retrieve storage path from parquet chunks
2021-05-05 13:29:16 +02:00
9e61b470e7
feat: change MemoryStream to accept multiple record batches
2021-05-05 13:29:16 +02:00
34754ebcdb
refactor: move MemoryStream to arrow_deps
2021-05-05 13:29:16 +02:00
44e134792b
refactor: Use slice patterns for CLI command matching ( #1412 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-05 10:59:56 +00:00
39234468f0
Merge pull request #1419 from influxdata/er/refactor/spelling
...
refactor: spelling and log info
2021-05-05 10:13:21 +00:00
b4b048127d
refactor: add column count to log line
2021-05-05 11:08:15 +01:00
2f789485e6
refactor: fix spelling
2021-05-05 11:06:04 +01:00
9882087321
Merge pull request #1410 from influxdata/er/feat/chunk_metrics
...
feat: metrics for total size of chunks across stages
2021-05-04 17:47:53 +00:00
6f280d631e
Merge branch 'main' into er/feat/chunk_metrics
2021-05-04 17:41:15 +00:00
733d502350
refactor: fix tests
2021-05-04 18:38:42 +01:00
57616c3d16
feat: implement --format and `SET FORMAT <pretty,csv,json>` in SQL repl ( #1404 )
...
* feat: add `set format` and --format flag to sql REPL
* fix: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-04 17:02:10 +00:00
9aa144e0f4
feat: add per-stage current chunk storage
2021-05-04 17:43:53 +01:00
f36f1efeeb
feat: add row counts to SQL repl output ( #1405 )
...
* feat: add row counts to repl
* fix: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-04 10:57:16 +00:00
3b7c5ac350
fix(storage rpc): do not send back tags with empty values ( #1403 )
2021-05-04 10:35:24 +00:00
8374dc3176
Merge pull request #1386 from influxdata/crepererum/issue1253
...
feat: implement transaction handling for catalog
2021-05-03 11:16:16 +00:00
ce568b4b2f
Merge branch 'main' into crepererum/issue1253
2021-05-03 11:11:42 +00:00
3a5c1dc03b
Merge pull request #1394 from influxdata/jemalloc-stats
...
feat: Add jemalloc stats
2021-05-03 10:26:05 +00:00
b579ef8646
feat: Add jemalloc stats
2021-05-03 12:10:48 +02:00
1269b5e3f0
docs: adjust catalog preservation design to implementation
...
For #1253 .
2021-05-03 10:04:35 +02:00
136c35cb88
feat: implement transaction handling for catalog
...
Closes #1253 .
2021-05-03 10:04:35 +02:00
ab142efdd3
feat: add protobuf types for preserved catalog
...
This only includes a minimal type structure to get the transaction layer
going and tested.
See issue #1253 .
2021-05-03 09:51:00 +02:00
1f5a9b5ebf
feat: make DirsAndFileName members public
...
Helps with #1253 .
2021-05-03 09:51:00 +02:00
ca5351088a
feat: add Path->DirsAndFileName conversion
...
This allows for easier parsing of received paths.
Helps with #1253 .
2021-05-03 09:51:00 +02:00
a2fd01e388
Merge pull request #1399 from influxdata/ntran/unload_chunks
...
feat: unload chunks from read buffer but keep them in object store
2021-04-30 22:08:00 +00:00
3c5595d046
Merge branch 'main' into ntran/unload_chunks
2021-04-30 22:02:38 +00:00
2c07edd9bb
Merge pull request #1398 from influxdata/pd-refactor-wal
...
refactor: write buffer to use sequenced entry and new segment
2021-04-30 21:05:02 +00:00
979f5f9347
refactor: write buffer to use sequenced entry and new segment
...
This refactors the write buffer to use the sequenced entry structure and the new segment definition. It removes the old replicated write and write_buffer.fbs.
Finally, it updates the SequencedEntry wrapper type around the Flatbuffer structure to be a trait so that SequencedEntry can be initialized from a borrowed Flatbuffer or an owned Vec<u8>.
How writes go into segments in the buffer and any kind of validation will likely have to be updated based on what kinds of guarantees we want to make in the buffer. However, that should probably come after we've rethought the design a bit around the new layout of chunks in the Parquet persistence.
2021-04-30 17:00:23 -04:00
a967ebfabd
refactor: rename closing to closed ( #1396 )
...
* refactor: rename closing to closed
* refactor: further renames
2021-04-30 20:59:45 +00:00
34a3388a49
feat: unload chunks from read buffer but keep them in object store
2021-04-30 16:12:02 -04:00
c2f7e7efea
feat: warn on dropping from open partition ( #1395 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-30 18:56:49 +00:00
da440d8d27
feat: add a id() method to `management::Operations` ( #1385 )
...
* feat: add Operation::id()
* feat: add a id() method to Operations management client
* fix: broken doc link
2021-04-30 17:09:28 +00:00
d74aebbc5e
Merge pull request #1393 from influxdata/memgauge
...
feat: Add user provided callback to Gauge
2021-04-30 15:27:31 +00:00
73700d767a
feat: Add user provided callback to Gauge
2021-04-30 17:22:52 +02:00
d8c3140961
Merge pull request #1392 from influxdata/er/feat/gauge_up_down
...
feat: allow gauges to be increased or decreased
2021-04-30 13:52:02 +00:00
72490219e7
Merge branch 'main' into er/feat/gauge_up_down
2021-04-30 13:47:38 +00:00
028e7f2aa6
refactor: change gauge to f64
2021-04-30 14:32:00 +01:00
d1fb8b1f87
Merge pull request #1391 from influxdata/er/refactor/tidy_metrics
...
refactor: include default labels
2021-04-30 13:29:17 +00:00
778bf39705
feat: add support for add/sub on gauges
2021-04-30 14:28:16 +01:00
ebe671e59e
refactor: include default labels
2021-04-30 14:03:09 +01:00
Carol (Nichols || Goulding)
Dom
kodiakhq[bot]
Marco Neumann
Andrew Lamb
Edd Robinson
Marko Mikulicic
Paul Dix
Raphael Taylor-Davies
Nga Tran