feb2a356ee
Merge branch 'main' into crepererum/write_buffer_crate
2021-07-14 12:19:24 +00:00
84826699c2
Merge pull request #1988 from influxdata/crepererum/peristence_windows_lint
...
chore: enable more aggressive linting for `persistence_windows` crate
2021-07-14 12:18:32 +00:00
886a87f011
chore: enable linting for `persistence_windows` crate
2021-07-14 14:11:51 +02:00
9b413c6bd7
Merge branch 'main' into er/chore/e2elogs
2021-07-14 13:10:39 +01:00
9cb9ae0874
chore: move write buffer into its own crate
2021-07-14 14:09:18 +02:00
d6885760d8
Merge pull request #1991 from influxdata/noreqsize
...
fix: Remove bad max.request.size config param
2021-07-14 12:07:54 +00:00
44f9689093
chore: emit server logs
2021-07-14 12:56:17 +01:00
d427fed9dc
fix: Remove bad max.request.size config param
2021-07-14 13:54:18 +02:00
8fd0df04f2
feat: continue buidling and using sort_key if available
2021-07-13 16:25:58 -04:00
781c4fa666
fix: update server tests
2021-07-13 15:44:57 -04:00
97c727a2c2
fix: update read_buffer tests
2021-07-13 15:44:57 -04:00
4800b36949
chore: Update IOx to a pre-release version of arrow and datafusion to test out performance improvement
2021-07-13 15:44:57 -04:00
ef3269ee1d
chore: Update datafusion deps ( #1984 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 17:29:29 +00:00
0164cabbf3
refactor: do not use DataFrame DataFusion API / stop optimizing twice ( #1982 )
...
* refactor: do not use DataFrame DataFusion API
* fix: update output to reflect not running optimizer twice
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 16:29:43 +00:00
09a744fce3
Merge pull request #1981 from influxdata/writebuffergotcha
...
fix: Raise kafka max request size to 10M
2021-07-13 15:55:51 +00:00
239c931f26
fix: Raise max message to 10M
...
And log message size on kafka write error.
Turns out the kafka partition message size limit default is 1MB, but also the
client side "max request size" default is also 1MB.
The error message we get from our kafka client is misleading: it says
```
KafkaError (Message production error: MessageSizeTooLarge (Broker: Message size too large)) }
```
which to my mind it seemed like if ("Broker:") the broker said "Message size too large".
That was a lie; I killed the broker and the client kept saying the same error message which means
it didn't even try to send the message out.
TODO: make this a proper parameter. (but let's unblock)
2021-07-13 17:47:36 +02:00
c272933627
Merge pull request #1980 from influxdata/crepererum/update_deps
...
chore: update some dependencies
2021-07-13 14:26:23 +00:00
6a09678f34
Merge branch 'main' into crepererum/update_deps
2021-07-13 14:18:57 +00:00
cc4c7cfad7
Merge pull request #1978 from influxdata/genscript
...
feat: Add small genlp.py script
2021-07-13 14:18:19 +00:00
c5f97211ba
Merge branch 'main' into genscript
2021-07-13 14:11:37 +00:00
6c8b2b4fa7
feat: add integration test of compaction freezing ( #1938 ) ( #1975 )
...
* feat: add integration test of compaction freezing (#1938 )
* chore: update server/src/db/lifecycle/compact.rs
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-07-13 14:11:10 +00:00
04e77f6b3c
feat: assert PersistenceWindows pre-conditions ( #1976 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 14:04:04 +00:00
157a0cc98c
chore: update flatbuffers to 2.0
2021-07-13 15:44:45 +02:00
f5c63b2ae6
chore: update nom to version 6
...
Triggered some changes from `Fn` to `FnMut`.
2021-07-13 15:28:42 +02:00
f11be17523
chore: update mockito to 0.30
2021-07-13 15:17:28 +02:00
2e391deb34
chore: update croaring to 0.5.0
...
Upstreame changelog:
- CRoaring updated to 0.3.1
- `-march=native` is not a default for croaring-sys anymore
- Impl Default for `Bitmap` and `Treemap`
2021-07-13 15:15:41 +02:00
5c16bd2085
chore: cargo update
2021-07-13 15:13:37 +02:00
018b6cbf85
Merge pull request #1979 from influxdata/moarlogging
...
chore: Log whether the write buffer is enabled
2021-07-13 12:33:44 +00:00
bf20641d78
chore: Log whether the write buffer is enabled
2021-07-13 14:15:52 +02:00
076b8f74fb
feat: Add small genlp.py script
2021-07-13 14:04:35 +02:00
60c055778e
chore: Log body size when inserting lines ( #1974 )
...
Both for the http and grpc write entrypoints.
(the gRPC write entry point and the http write entry point would benefit from being
factored out together, but that cleanup is for another time).
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 11:05:57 +00:00
5a0caeab44
feat: skip over fully persisted partitions ( #1962 ) ( #1973 )
...
* feat: skip over fully persisted partitions (#1962 )
* chore: review feedback
2021-07-13 10:40:45 +00:00
d35b74c226
fix: Fix doc build warnings ( #1945 )
...
* fix: Fix doc build warnings
* refactor: add deny bare_urls to crates
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 08:03:42 +00:00
cf49880ff5
refactor: reduce some repetition in http tests ( #1969 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 07:56:10 +00:00
708aebaeb3
fix: freeze open chunk when compacting ( #1971 )
...
Closes #1938 . Unfortunately, this contains only a unit test to ensure that open chunks are frozen when set_compacting is called. It would be better to have a more end-to-end integration test that ensurest his behavior, but I've confirmed by hand (with some sleeps and a hacked up end-to-end test) that this fixes it.
2021-07-13 07:44:02 +00:00
5418a1fe6b
refactor: remove unused comments
2021-07-12 18:14:38 -04:00
23895e6673
feat: Using sort_key to avoid resorts
2021-07-12 18:08:45 -04:00
68a2e02e1f
fix: do not panic on client hangup ( #1968 )
2021-07-12 22:04:27 +00:00
d2147e76cc
Merge pull request #1964 from influxdata/cn/known-wb-error
...
fix: Write Buffer errors are known, not UnknownDatabaseErrors
2021-07-12 18:55:16 +00:00
b7337e7f9a
Merge branch 'main' into cn/known-wb-error
2021-07-12 18:48:13 +00:00
a54cd04b30
feat: log reason for triggering persistence ( #1957 ) ( #1967 )
...
* feat: log reason for triggering persistence (#1957 )
Don't panic on no eligible chunks to persist (#1966 )
* fix: fix conditions and logging
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-12 18:28:00 +00:00
7328efbc74
Merge pull request #1948 from influxdata/ntran/use_sortkey
...
feat: add try_merge_key function to compute a super sort key set
2021-07-12 18:20:00 +00:00
f26f844ed2
Merge branch 'main' into ntran/use_sortkey
2021-07-12 18:12:47 +00:00
4b0e4c6f41
fix: Allow large enum variants for an error enum
2021-07-12 13:54:50 -04:00
0d74ce576b
refactor: address review comments and put more restrictions on time (must have and in last position) column to avoid treating it differently
2021-07-12 13:53:44 -04:00
466416016b
Merge pull request #1950 from influxdata/cn/refactor-mock-chunks
...
refactor: Go from 3 mock chunks to 1
2021-07-12 17:52:52 +00:00
fd7705d9b4
Merge branch 'main' into cn/refactor-mock-chunks
2021-07-12 17:46:07 +00:00
07b25ad73c
Merge pull request #1952 from influxdata/crepererum/issue1919
...
feat: add API+CLI to unload chunks
2021-07-12 16:21:37 +00:00
c62c7d78d2
Merge branch 'main' into crepererum/issue1919
2021-07-12 16:14:13 +00:00
6764a2d68e
fix: Write Buffer errors are known, not UnknownDatabaseErrors
...
Fixes #1956 .
2021-07-12 11:21:31 -04:00
kodiakhq[bot]
Marco Neumann
Edd Robinson
Marko Mikulicic
Nga Tran
Andrew Lamb
Andrew Lamb
Raphael Taylor-Davies
Paul Dix
Carol (Nichols || Goulding)