38f4eec20e
feat: implement `seek` for write buffer
...
This is required to control replay ranges.
2021-07-20 10:25:56 +02:00
07c8dea6b9
Merge pull request #2055 from influxdata/uplimit
...
feat(iox): Make max http request size configurable
2021-07-20 07:19:56 +00:00
6fd40d9e18
Merge branch 'main' into uplimit
2021-07-20 07:11:35 +00:00
1668420ded
feat: new algorth to compute key ranges for deduplicating data
2021-07-19 18:04:25 -04:00
dde7d19ae9
Merge pull request #2041 from influxdata/jg/snafu-driveby
...
refactor: Make more use of SNAFU's context methods and ensure! macro
2021-07-19 20:27:29 +00:00
5bf68c4a57
Merge branch 'main' into jg/snafu-driveby
2021-07-19 20:20:30 +00:00
45ff5e214c
feat(iox): Make max http request size configurable
2021-07-19 22:14:51 +02:00
1c16988a51
chore: Update datafusion references ( #2056 )
2021-07-19 18:09:06 +00:00
1c8c227668
refactor: push database rules update into Db ( #2052 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-19 16:05:21 +00:00
8c022e2cbb
Merge pull request #2044 from influxdata/crepererum/write_buffer_multiple_streams
...
refactor: use one stream per sequencer/partition
2021-07-19 15:57:56 +00:00
1d1ac12522
Merge branch 'main' into crepererum/write_buffer_multiple_streams
2021-07-19 15:50:42 +00:00
ff6b0ac030
fix: Cargo git deps use rev rather than ref ( #2050 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-19 13:31:24 +00:00
eea530acde
Merge pull request #2048 from influxdata/er/feat/compaction_limiter
...
feat: add per-db in-memory compaction limiter
2021-07-19 13:22:46 +00:00
2d6a23f7c3
fix: avoid potential race
2021-07-19 14:02:29 +01:00
a852dce450
refactor: default to num_cpu
2021-07-19 14:00:10 +01:00
dfda23f24a
test: update e2e tests
2021-07-19 14:00:10 +01:00
dd93b2cdec
feat: add db-level compaction limiter
2021-07-19 14:00:10 +01:00
4ca19ad13f
refactor: add max_active_compaction config option
2021-07-19 14:00:10 +01:00
4da8a16c18
chore: update to arrow 5.0 and master datafusion ( #2049 )
...
* chore: update to arrow 5.0 and master datafusion
* fix: Update test for change in object size
2021-07-19 12:49:51 +00:00
e2a23c7ac3
fix: persist deadlock ( #2045 ) ( #2046 )
2021-07-19 11:52:48 +00:00
592424c896
refactor: use one stream per sequencer/partition
...
Advantages are:
- for large DBs w/ many partitions we can ingest data in-parallel
- on top of this change we can implement per-sequencer seeking, which is
required for replay
2021-07-19 12:26:58 +02:00
ee6a14b7dc
Merge pull request #2042 from DCjanus/main
...
refactor: explicitly specify the type, more friendly to IDE
2021-07-19 10:06:57 +00:00
eb00a72782
Merge branch 'main' into main
2021-07-19 10:00:23 +00:00
f5f8e507bb
Merge pull request #2043 from influxdata/crepererum/simplify_testdb_lifecycle_rules
...
test: make TestDb lifecycle better for testing
2021-07-19 09:59:55 +00:00
a1d47a8a7a
Merge branch 'main' into crepererum/simplify_testdb_lifecycle_rules
2021-07-19 09:53:35 +00:00
5fc98c7c56
feat: add failure reporting to TaskTracker ( #2031 )
...
* feat: add failure reporting to TaskTracker
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-19 09:17:20 +00:00
2263189e09
test: make TestDb lifecycle better for testing
...
This is a leftover from #1972 .
2021-07-19 09:50:44 +02:00
cdb81cbc60
refactor: explicitly specify the type, more friendly to IDE
2021-07-17 22:45:54 +08:00
449ba46b22
refactor: Make more use of SNAFU's context methods and ensure! macro
2021-07-16 16:31:50 -04:00
0c17080e40
Merge pull request #2034 from influxdata/jg/snafu-driveby
...
refactor: clean up some SNAFU usage
2021-07-16 18:05:58 +00:00
c896f0e100
Merge branch 'main' into jg/snafu-driveby
2021-07-16 17:58:58 +00:00
d00d56027b
docs: add comment to trigger build ( #2039 )
2021-07-16 17:53:55 +00:00
77633315f3
Merge pull request #2037 from influxdata/acks
...
fix: Set acks=all for kafka writes
2021-07-16 16:03:22 +00:00
1676c33113
refactor: update write_buffer/src/kafka.rs
2021-07-16 16:56:03 +01:00
de0dce79f6
docs: Update write_buffer/src/kafka.rs
...
Co-authored-by: Edd Robinson <me@edd.io>
2021-07-16 17:53:02 +02:00
92623b5f32
fix: Set acks=all for kafka writes
2021-07-16 17:49:04 +02:00
98a0bdf31d
Merge pull request #2009 from influxdata/cn/meta-data-size-bug
...
fix: use correct row group size when updating table meta data
2021-07-16 15:03:03 +00:00
54ad69ed86
fix: ensure correct table meta size used
2021-07-16 10:48:45 -04:00
42b56ad657
refactor: Use SNAFU's context instead of `ok_or_else`
2021-07-16 09:59:54 -04:00
939d15a21f
perf: Avoid clone when an error doesn't occur
2021-07-16 09:59:54 -04:00
f3175ed291
test: use of different size values
2021-07-16 09:47:56 -04:00
abe2fe7262
test: MetaData new with a row group vs default then update_with should have the same size
2021-07-16 09:47:56 -04:00
f57ba6afdb
fix: use fixed-size timestamps for parquet metadata ( #2032 )
...
This fixes flaky tests that rely on predictable files sizes.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-16 13:14:02 +00:00
3d5c07a5d8
Merge pull request #1890 from influxdata/fbroot
...
feat: Flatbuffers root_type and file_identifier
2021-07-16 12:38:23 +00:00
256aaa21e5
Merge branch 'main' into fbroot
2021-07-16 11:48:58 +00:00
0319d7d4d2
Merge pull request #1972 from influxdata/crepererum/persist_real_checkpoints
...
feat: persist real (non-fake) part.+DB checkpoints
2021-07-16 10:53:28 +00:00
2498642c00
fix: `persist_partition` docstring
2021-07-16 12:46:07 +02:00
1ef2bc1887
refactor: `Db::{write_chunk_to_object_store => Db::persist_partition}`
...
The previous method allowed to persist any chunk -- even ones that
should not be persisted yet and w/o any order of peristence. That will
break our persistence windows. So instead offer a sane higher-level
interface that can trigger persistence of a partition within the
boundaries of the lifecycle rules. This needs some adjustments for our
test suite.
2021-07-16 12:07:58 +02:00
9683d91f32
refactor: adjust to upstream changes
2021-07-16 11:45:34 +02:00
2b0a4bbe0a
feat: persist real (non-fake) part.+DB checkpoints
2021-07-16 11:45:34 +02:00
Marco Neumann
kodiakhq[bot]
Nga Tran
Marko Mikulicic
Andrew Lamb
Raphael Taylor-Davies
Carol (Nichols || Goulding)
Edd Robinson
DCjanus
Jake Goulding
Carol (Nichols || Goulding)