018b6cbf85
Merge pull request #1979 from influxdata/moarlogging
...
chore: Log whether the write buffer is enabled
2021-07-13 12:33:44 +00:00
bf20641d78
chore: Log whether the write buffer is enabled
2021-07-13 14:15:52 +02:00
076b8f74fb
feat: Add small genlp.py script
2021-07-13 14:04:35 +02:00
60c055778e
chore: Log body size when inserting lines ( #1974 )
...
Both for the http and grpc write entrypoints.
(the gRPC write entry point and the http write entry point would benefit from being
factored out together, but that cleanup is for another time).
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 11:05:57 +00:00
5a0caeab44
feat: skip over fully persisted partitions ( #1962 ) ( #1973 )
...
* feat: skip over fully persisted partitions (#1962 )
* chore: review feedback
2021-07-13 10:40:45 +00:00
d35b74c226
fix: Fix doc build warnings ( #1945 )
...
* fix: Fix doc build warnings
* refactor: add deny bare_urls to crates
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 08:03:42 +00:00
cf49880ff5
refactor: reduce some repetition in http tests ( #1969 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 07:56:10 +00:00
708aebaeb3
fix: freeze open chunk when compacting ( #1971 )
...
Closes #1938 . Unfortunately, this contains only a unit test to ensure that open chunks are frozen when set_compacting is called. It would be better to have a more end-to-end integration test that ensurest his behavior, but I've confirmed by hand (with some sleeps and a hacked up end-to-end test) that this fixes it.
2021-07-13 07:44:02 +00:00
5418a1fe6b
refactor: remove unused comments
2021-07-12 18:14:38 -04:00
23895e6673
feat: Using sort_key to avoid resorts
2021-07-12 18:08:45 -04:00
68a2e02e1f
fix: do not panic on client hangup ( #1968 )
2021-07-12 22:04:27 +00:00
d2147e76cc
Merge pull request #1964 from influxdata/cn/known-wb-error
...
fix: Write Buffer errors are known, not UnknownDatabaseErrors
2021-07-12 18:55:16 +00:00
b7337e7f9a
Merge branch 'main' into cn/known-wb-error
2021-07-12 18:48:13 +00:00
a54cd04b30
feat: log reason for triggering persistence ( #1957 ) ( #1967 )
...
* feat: log reason for triggering persistence (#1957 )
Don't panic on no eligible chunks to persist (#1966 )
* fix: fix conditions and logging
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-12 18:28:00 +00:00
7328efbc74
Merge pull request #1948 from influxdata/ntran/use_sortkey
...
feat: add try_merge_key function to compute a super sort key set
2021-07-12 18:20:00 +00:00
f26f844ed2
Merge branch 'main' into ntran/use_sortkey
2021-07-12 18:12:47 +00:00
4b0e4c6f41
fix: Allow large enum variants for an error enum
2021-07-12 13:54:50 -04:00
0d74ce576b
refactor: address review comments and put more restrictions on time (must have and in last position) column to avoid treating it differently
2021-07-12 13:53:44 -04:00
466416016b
Merge pull request #1950 from influxdata/cn/refactor-mock-chunks
...
refactor: Go from 3 mock chunks to 1
2021-07-12 17:52:52 +00:00
fd7705d9b4
Merge branch 'main' into cn/refactor-mock-chunks
2021-07-12 17:46:07 +00:00
07b25ad73c
Merge pull request #1952 from influxdata/crepererum/issue1919
...
feat: add API+CLI to unload chunks
2021-07-12 16:21:37 +00:00
c62c7d78d2
Merge branch 'main' into crepererum/issue1919
2021-07-12 16:14:13 +00:00
6764a2d68e
fix: Write Buffer errors are known, not UnknownDatabaseErrors
...
Fixes #1956 .
2021-07-12 11:21:31 -04:00
3bd7486016
test: Rename a test type alias to not shadow super::Error
2021-07-12 10:46:29 -04:00
c681da1031
refactor: Define the TestChunk methods with macros
2021-07-12 09:59:12 -04:00
4e53a32928
refactor: Completely replace query::provider::overlap::TestChunk with query::test::TestChunk
2021-07-12 09:59:12 -04:00
1698edcc39
refactor: Implement query::provider::overlap::TestChunk in terms of query::test::TestChunk
2021-07-12 09:59:12 -04:00
dc0b97e121
refactor: Completely replace TestChunkMeta with TestChunk
2021-07-12 09:59:12 -04:00
96f9485792
refactor: Move a with_no_stats method to be entirely defined on TestChunk
2021-07-12 09:59:12 -04:00
b4c5a87088
refactor: Rename int field to i64 field to be more consistent
2021-07-12 09:59:12 -04:00
54f7ee8b8d
refactor: Implement TestChunkMeta in terms of TestChunk
...
This is a temporary step to make sure TestChunk does everything
TestChunkMeta needs
2021-07-12 09:59:12 -04:00
ee545ce90e
test: Make _with_stats methods able to optionally take max/min
...
Not used yet, but will be when this is unified with query/src/pruning.rs
2021-07-12 09:59:12 -04:00
b26aae1cb4
test: Add an arg to control whether to add a column summary at all
...
Always true for now, but there are some cases in query/src/pruning.rs
that don't add any column summaries that will use this with `false`.
2021-07-12 09:59:12 -04:00
6cd75bc688
test: Optionally take stats in add_schema_to_table
...
This gets rid of a lookup and construction of default stats that aren't
necessary
2021-07-12 09:59:12 -04:00
e05ca7f98b
fix: Change a method name that says null to not say null
...
The comment and implementation seem to indicate this is creating
non-null data.
2021-07-12 09:59:12 -04:00
4406d8a219
test: Always initialize a TableSummary on TestChunk
2021-07-12 09:59:12 -04:00
22d4040c81
test: Always initialize a Schema for TestChunk
2021-07-12 09:59:12 -04:00
92cb5986f1
test: Initialize a schema on TestChunk to always exist
2021-07-12 09:59:12 -04:00
78f1c4fc80
test: Chunks can only have one table; no need to specify repeatedly
...
This lets us make the name required and always present on TestChunks,
and make the ID optional.
2021-07-12 09:59:12 -04:00
d2aadddeef
refactor: Remove unneeded reassignments
2021-07-12 09:59:11 -04:00
15aac65c2c
fix: Arrange use statements so rustfmt can manage their order
2021-07-12 09:59:11 -04:00
670826daf9
refactor: make object_store construction interface consistent ( #1944 )
...
* refactor: make object_store construction interface consistent
* fix: benchmarks
* fix: doc build
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-12 12:56:36 +00:00
3d008f4d27
feat: add API+CLI to unload chunks
...
Closes #1919 .
2021-07-12 14:06:01 +02:00
3dae69ede9
Merge pull request #1949 from influxdata/pd-remove-mutable-linger
...
refactor: remove mutable_linger_seconds from lifecycle
2021-07-11 12:31:48 +00:00
6f2d20cb19
chore: reword comment on late_arrive_window_seconds for clarity
2021-07-11 08:25:31 -04:00
2854b54420
refactor: leave deprecated mutable_linger_seconds in proto
2021-07-10 12:48:50 -04:00
0c8c81a321
refactor: remove mutable_linger_seconds from lifecycle
...
The interplay between mutable_linger_seconds, late_arrive_window and persist_age_threshold_seconds can be tricky to reason about. I realized that the lifecycle rules can be simplified by removing mutable_linger_seconds and instead using late_arrive_window_seconds for the same purpose. Semantically, they basically mean the same thing. We want to give data around this amount of time to arrive before the system persists it, which gives it more of an opportunity to persist non-overlapping data.
When a partition goes cold for writes, after we've waiting past this window, we should compact and persist that partition. This removes one unnecessary knob from the lifecycle configuration and also removes the potential for conflicting configuration options.
2021-07-10 08:04:33 -04:00
7b7a60993d
feat: consider time as a special key
2021-07-09 18:54:22 -04:00
8f4463664c
feat: add super_key function
2021-07-09 15:37:04 -04:00
9534220035
feat: Add any lifecycle_action to system.chunks and API ( #1947 )
2021-07-09 17:38:29 +00:00
kodiakhq[bot]
Marko Mikulicic
Raphael Taylor-Davies
Andrew Lamb
Paul Dix
Nga Tran
Carol (Nichols || Goulding)
Marco Neumann
Paul Dix