387667330a
chore: Update datafusion deps ( #2073 )
...
* chore: Update datafusion deps
* fix: update tests
2021-07-21 08:27:03 +00:00
a4704dd165
chore: update parquet_cache_limit to u64 and 0 for default
2021-07-20 15:41:06 -04:00
297e059085
feat: add parquet cache size setting to database rules
2021-07-20 15:41:06 -04:00
d547c22e97
refactor: comments
2021-07-20 15:27:41 -04:00
150e166813
refactor: fix comments
2021-07-20 15:16:24 -04:00
fa6d216a85
refactor: cleanup
2021-07-20 15:11:02 -04:00
b98888e8d6
feat: implement key_ranges function that uses new range identify algo
2021-07-20 14:58:54 -04:00
61da0fe4df
fix: update last_instant when rotating into persistable window ( #2067 )
2021-07-20 16:38:28 +00:00
091837420f
feat: add PersistenceWindows sytem table ( #2030 ) ( #2062 )
...
* feat: add PersistenceWindows sytem table (#2030 )
* chore: update log
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-20 13:10:57 +00:00
e4d2c51e8b
fix: update PersistenceWindows on rules update ( #2018 ) ( #2060 )
...
* fix: update PersistenceWindows on rules update (#2018 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-20 12:44:47 +00:00
7d9e1f9704
Merge pull request #2059 from influxdata/crepererum/writer_buffer_seek
...
feat: implement `seek` for write buffer
2021-07-20 12:36:20 +00:00
58dd7e9532
Merge branch 'main' into crepererum/writer_buffer_seek
2021-07-20 12:29:18 +00:00
2a7848cbf2
Merge pull request #2064 from influxdata/biggermsg
...
fix: Increase kafka message size to 30MiB
2021-07-20 12:28:55 +00:00
a4951b5835
Merge branch 'main' into biggermsg
2021-07-20 12:22:19 +00:00
cf8a60252d
refactor: split system_tables module into smaller modules ( #2061 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-20 12:19:20 +00:00
c01cfbc34c
fix: Increase kafka message size
2021-07-20 14:17:37 +02:00
cf30d19fd7
Merge pull request #2063 from influxdata/er/fix/flaky_compact_test
...
test: ensure high enough limit
2021-07-20 12:02:27 +00:00
ec7ebdff29
refactor: use lifetimes to ensure single stream / no seek while streaming
2021-07-20 13:52:33 +02:00
cc0aaa58a7
test: ensure high enough limit
2021-07-20 12:43:10 +01:00
b0663a0337
feat: disallow multiple write buffer streams and seeking while streams
...
Multiple streams will mess up ordering. Seeking while streaming is
likely a bug and should not work.
2021-07-20 12:35:20 +02:00
767c2a6fe1
refactor: explicit server startup state machine ( #2040 )
...
* refactor: explicit server startup state machine
* chore: update `ServerStage` docs
* chore: further docs
* chore: more logging
* chore: format
2021-07-20 10:11:18 +00:00
2c20528c69
chore: use upstream versions of some workarounds ( #2057 )
...
* chore: use upstream versions of some workarounds
* docs: update docstring
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-20 08:53:46 +00:00
8e5d5928cf
feat: compute WriteSummary from PersistenceWindows ( #2030 ) ( #2054 )
...
* feat: compute WriteSummary from PersistenceWindows (#2030 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-20 08:46:52 +00:00
38f4eec20e
feat: implement `seek` for write buffer
...
This is required to control replay ranges.
2021-07-20 10:25:56 +02:00
07c8dea6b9
Merge pull request #2055 from influxdata/uplimit
...
feat(iox): Make max http request size configurable
2021-07-20 07:19:56 +00:00
6fd40d9e18
Merge branch 'main' into uplimit
2021-07-20 07:11:35 +00:00
1668420ded
feat: new algorth to compute key ranges for deduplicating data
2021-07-19 18:04:25 -04:00
dde7d19ae9
Merge pull request #2041 from influxdata/jg/snafu-driveby
...
refactor: Make more use of SNAFU's context methods and ensure! macro
2021-07-19 20:27:29 +00:00
5bf68c4a57
Merge branch 'main' into jg/snafu-driveby
2021-07-19 20:20:30 +00:00
45ff5e214c
feat(iox): Make max http request size configurable
2021-07-19 22:14:51 +02:00
1c16988a51
chore: Update datafusion references ( #2056 )
2021-07-19 18:09:06 +00:00
1c8c227668
refactor: push database rules update into Db ( #2052 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-19 16:05:21 +00:00
8c022e2cbb
Merge pull request #2044 from influxdata/crepererum/write_buffer_multiple_streams
...
refactor: use one stream per sequencer/partition
2021-07-19 15:57:56 +00:00
1d1ac12522
Merge branch 'main' into crepererum/write_buffer_multiple_streams
2021-07-19 15:50:42 +00:00
ff6b0ac030
fix: Cargo git deps use rev rather than ref ( #2050 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-19 13:31:24 +00:00
eea530acde
Merge pull request #2048 from influxdata/er/feat/compaction_limiter
...
feat: add per-db in-memory compaction limiter
2021-07-19 13:22:46 +00:00
2d6a23f7c3
fix: avoid potential race
2021-07-19 14:02:29 +01:00
a852dce450
refactor: default to num_cpu
2021-07-19 14:00:10 +01:00
dfda23f24a
test: update e2e tests
2021-07-19 14:00:10 +01:00
dd93b2cdec
feat: add db-level compaction limiter
2021-07-19 14:00:10 +01:00
4ca19ad13f
refactor: add max_active_compaction config option
2021-07-19 14:00:10 +01:00
4da8a16c18
chore: update to arrow 5.0 and master datafusion ( #2049 )
...
* chore: update to arrow 5.0 and master datafusion
* fix: Update test for change in object size
2021-07-19 12:49:51 +00:00
e2a23c7ac3
fix: persist deadlock ( #2045 ) ( #2046 )
2021-07-19 11:52:48 +00:00
592424c896
refactor: use one stream per sequencer/partition
...
Advantages are:
- for large DBs w/ many partitions we can ingest data in-parallel
- on top of this change we can implement per-sequencer seeking, which is
required for replay
2021-07-19 12:26:58 +02:00
ee6a14b7dc
Merge pull request #2042 from DCjanus/main
...
refactor: explicitly specify the type, more friendly to IDE
2021-07-19 10:06:57 +00:00
eb00a72782
Merge branch 'main' into main
2021-07-19 10:00:23 +00:00
f5f8e507bb
Merge pull request #2043 from influxdata/crepererum/simplify_testdb_lifecycle_rules
...
test: make TestDb lifecycle better for testing
2021-07-19 09:59:55 +00:00
a1d47a8a7a
Merge branch 'main' into crepererum/simplify_testdb_lifecycle_rules
2021-07-19 09:53:35 +00:00
5fc98c7c56
feat: add failure reporting to TaskTracker ( #2031 )
...
* feat: add failure reporting to TaskTracker
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-19 09:17:20 +00:00
2263189e09
test: make TestDb lifecycle better for testing
...
This is a leftover from #1972 .
2021-07-19 09:50:44 +02:00
Andrew Lamb
Paul Dix
Nga Tran
Raphael Taylor-Davies
kodiakhq[bot]
Marko Mikulicic
Marco Neumann
Edd Robinson
Carol (Nichols || Goulding)