Nga Tran
11ba4b5f6a
fix: fix unit_test setting to have the desired results
2021-07-22 14:22:08 -04:00
Nga Tran
b2063fb29f
test: fix the stats and discover a bug in compaction/split/dedupplication
2021-07-21 17:40:48 -04:00
kodiakhq[bot]
13ae2f0d78
Merge pull request #2070 from influxdata/ntran/dedup_compare_cols_order
...
feat: new algorithm to compute key ranges for deduplication
2021-07-21 15:50:11 +00:00
kodiakhq[bot]
18dd108ba6
Merge branch 'main' into ntran/dedup_compare_cols_order
2021-07-21 15:42:30 +00:00
Nga Tran
86add39175
refactor: address review comments
2021-07-21 11:41:21 -04:00
kodiakhq[bot]
56dd430d8f
Merge pull request #2077 from influxdata/crepererum/sequencer_metrics
...
feat: write buffer ingestion metrics
2021-07-21 13:30:22 +00:00
kodiakhq[bot]
91acf3911c
Merge branch 'main' into crepererum/sequencer_metrics
2021-07-21 13:23:23 +00:00
Marco Neumann
55490c279a
fix: Kafka watermark error for new partitions
2021-07-21 15:21:52 +02:00
Marco Neumann
cddf94653c
refactor: use `write_buffer` subsystem for ingest metrics
2021-07-21 15:07:59 +02:00
Marco Neumann
fd00206fbb
refactor: increase watermark update frequence to once per 10s
2021-07-21 15:02:48 +02:00
Marco Neumann
2f1efcf517
docs: clarify difference
2021-07-21 15:00:53 +02:00
Marco Neumann
4d5f209030
docs: do not repeat unix that often
2021-07-21 14:59:07 +02:00
Marco Neumann
7d597d1d5c
refactor: make ingest metrics easier to understand
2021-07-21 13:57:53 +02:00
Raphael Taylor-Davies
ffe6e62aee
feat: add instant to datetime conversion ( #2078 )
...
* feat: add instant to datetime conversion
* chore: review feedback
2021-07-21 11:43:27 +00:00
Marco Neumann
fb931bb1ca
feat: write buffer ingestion metrics
2021-07-21 11:59:52 +02:00
Marco Neumann
5df88c70aa
feat: add ability to fetch watermarks from write buffer
2021-07-21 11:59:52 +02:00
kodiakhq[bot]
58108b79ec
Merge pull request #2058 from influxdata/pd/add-cache-config
...
feat: add parquet cache size setting to database rules
2021-07-21 09:42:07 +00:00
kodiakhq[bot]
94a45339fd
Merge branch 'main' into pd/add-cache-config
2021-07-21 09:35:26 +00:00
Andrew Lamb
387667330a
chore: Update datafusion deps ( #2073 )
...
* chore: Update datafusion deps
* fix: update tests
2021-07-21 08:27:03 +00:00
Paul Dix
a4704dd165
chore: update parquet_cache_limit to u64 and 0 for default
2021-07-20 15:41:06 -04:00
Paul Dix
297e059085
feat: add parquet cache size setting to database rules
2021-07-20 15:41:06 -04:00
Nga Tran
d547c22e97
refactor: comments
2021-07-20 15:27:41 -04:00
Nga Tran
150e166813
refactor: fix comments
2021-07-20 15:16:24 -04:00
Nga Tran
fa6d216a85
refactor: cleanup
2021-07-20 15:11:02 -04:00
Nga Tran
b98888e8d6
feat: implement key_ranges function that uses new range identify algo
2021-07-20 14:58:54 -04:00
Raphael Taylor-Davies
61da0fe4df
fix: update last_instant when rotating into persistable window ( #2067 )
2021-07-20 16:38:28 +00:00
Raphael Taylor-Davies
091837420f
feat: add PersistenceWindows sytem table ( #2030 ) ( #2062 )
...
* feat: add PersistenceWindows sytem table (#2030 )
* chore: update log
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-20 13:10:57 +00:00
Raphael Taylor-Davies
e4d2c51e8b
fix: update PersistenceWindows on rules update ( #2018 ) ( #2060 )
...
* fix: update PersistenceWindows on rules update (#2018 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-20 12:44:47 +00:00
kodiakhq[bot]
7d9e1f9704
Merge pull request #2059 from influxdata/crepererum/writer_buffer_seek
...
feat: implement `seek` for write buffer
2021-07-20 12:36:20 +00:00
kodiakhq[bot]
58dd7e9532
Merge branch 'main' into crepererum/writer_buffer_seek
2021-07-20 12:29:18 +00:00
kodiakhq[bot]
2a7848cbf2
Merge pull request #2064 from influxdata/biggermsg
...
fix: Increase kafka message size to 30MiB
2021-07-20 12:28:55 +00:00
kodiakhq[bot]
a4951b5835
Merge branch 'main' into biggermsg
2021-07-20 12:22:19 +00:00
Raphael Taylor-Davies
cf8a60252d
refactor: split system_tables module into smaller modules ( #2061 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-20 12:19:20 +00:00
Marko Mikulicic
c01cfbc34c
fix: Increase kafka message size
2021-07-20 14:17:37 +02:00
kodiakhq[bot]
cf30d19fd7
Merge pull request #2063 from influxdata/er/fix/flaky_compact_test
...
test: ensure high enough limit
2021-07-20 12:02:27 +00:00
Marco Neumann
ec7ebdff29
refactor: use lifetimes to ensure single stream / no seek while streaming
2021-07-20 13:52:33 +02:00
Edd Robinson
cc0aaa58a7
test: ensure high enough limit
2021-07-20 12:43:10 +01:00
Marco Neumann
b0663a0337
feat: disallow multiple write buffer streams and seeking while streams
...
Multiple streams will mess up ordering. Seeking while streaming is
likely a bug and should not work.
2021-07-20 12:35:20 +02:00
Raphael Taylor-Davies
767c2a6fe1
refactor: explicit server startup state machine ( #2040 )
...
* refactor: explicit server startup state machine
* chore: update `ServerStage` docs
* chore: further docs
* chore: more logging
* chore: format
2021-07-20 10:11:18 +00:00
Andrew Lamb
2c20528c69
chore: use upstream versions of some workarounds ( #2057 )
...
* chore: use upstream versions of some workarounds
* docs: update docstring
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-20 08:53:46 +00:00
Raphael Taylor-Davies
8e5d5928cf
feat: compute WriteSummary from PersistenceWindows ( #2030 ) ( #2054 )
...
* feat: compute WriteSummary from PersistenceWindows (#2030 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-20 08:46:52 +00:00
Marco Neumann
38f4eec20e
feat: implement `seek` for write buffer
...
This is required to control replay ranges.
2021-07-20 10:25:56 +02:00
kodiakhq[bot]
07c8dea6b9
Merge pull request #2055 from influxdata/uplimit
...
feat(iox): Make max http request size configurable
2021-07-20 07:19:56 +00:00
kodiakhq[bot]
6fd40d9e18
Merge branch 'main' into uplimit
2021-07-20 07:11:35 +00:00
Nga Tran
1668420ded
feat: new algorth to compute key ranges for deduplicating data
2021-07-19 18:04:25 -04:00
kodiakhq[bot]
dde7d19ae9
Merge pull request #2041 from influxdata/jg/snafu-driveby
...
refactor: Make more use of SNAFU's context methods and ensure! macro
2021-07-19 20:27:29 +00:00
kodiakhq[bot]
5bf68c4a57
Merge branch 'main' into jg/snafu-driveby
2021-07-19 20:20:30 +00:00
Marko Mikulicic
45ff5e214c
feat(iox): Make max http request size configurable
2021-07-19 22:14:51 +02:00
Andrew Lamb
1c16988a51
chore: Update datafusion references ( #2056 )
2021-07-19 18:09:06 +00:00
Raphael Taylor-Davies
1c8c227668
refactor: push database rules update into Db ( #2052 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-19 16:05:21 +00:00