Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
3,431 Commits
270 Branches
456 Tags
361 MiB
Commit Graph

3431 Commits (f79349bce02ee04f910786d57ea5cf1e97fd9a16)

Author SHA1 Message Date
Marco Neumann bf0ba6ba6c test: rename some server init tests to better reflect their nature 2021-06-14 17:34:57 +02:00
kodiakhq[bot] c905530e90
Merge pull request #1713 from influxdata/er/fix/dictionary_dupe_keys 
fix: ensure values are different
 2021-06-14 15:13:50 +00:00
Edd Robinson 6657e6f596 refactor: update query/src/exec/seriesset.rs 2021-06-14 16:09:02 +01:00
Edd Robinson 58f4073a7d
Merge branch 'main' into er/fix/dictionary_dupe_keys 2021-06-14 15:59:58 +01:00
Edd Robinson ec52bca309 fix: ensure values are different 2021-06-14 15:28:35 +01:00
Marco Neumann 776b6c011c feat: remove path parsing functionality 
Paths to parquet files are an implementation detail and should not be
parsed.

Closes #1506.
 2021-06-14 16:24:50 +02:00
Marco Neumann 250ccdcdcd refactor: use `IOxMetadata` instead of path parsing for parquet chunks 2021-06-14 16:24:50 +02:00
Marco Neumann d51e7a127c feat: include table name, partition key, and chunk ID in `IoxMetadata` 2021-06-14 16:24:50 +02:00
kodiakhq[bot] 3bfd97bef5
Merge pull request #1697 from influxdata/crepererum/checkpoint_during_restore 
feat: create checkpoint during catalog rebuild
 2021-06-14 13:58:18 +00:00
kodiakhq[bot] b57f397057
Merge branch 'main' into crepererum/checkpoint_during_restore 2021-06-14 13:54:03 +00:00
Andrew Lamb a14e9ab27c
refactor: rename mutable_buffer::Chunk --> mutable_buffer::MBChunk (#1711) 
* refactor: rename mutable_buffer::Chunk --> mutable_buffer::MBChunk

* fix: fmt
 2021-06-14 13:35:20 +00:00
Andrew Lamb 856751deec
feat: Lifecycle manager unloads, rather than drop, chunks when soft limit is hit (#1701) 
* feat: unload chunks from memory rather than dropping them

* docs: Update server/src/db/lifecycle.rs

Co-authored-by: Marco Neumann <marco@crepererum.net>

* docs: Update comment wording

Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-06-14 13:14:39 +00:00
kodiakhq[bot] 1cc926112b
Merge pull request #1710 from influxdata/crepererum/more_parquet_stats_work2 
fix: fix NaN+NULL handling in parquet stats
 2021-06-14 13:09:15 +00:00
kodiakhq[bot] c781b36bd7
Merge branch 'main' into crepererum/more_parquet_stats_work2 2021-06-14 13:04:35 +00:00
Andrew Lamb 8c21eb79af
refactor: inline Table into mutable_buffer::chunk::Chunk (#1704) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-06-14 13:04:06 +00:00
Marco Neumann 0a7dcc3779 test: adjust read-write parquet test to newest test data 2021-06-14 14:24:24 +02:00
Marco Neumann d6f6ddfdaa fix: fix NULL handling in parquet stats 2021-06-14 14:24:09 +02:00
Marco Neumann eae56630fb test: add test for all-NULL float column metadata 2021-06-14 13:48:34 +02:00
Marco Neumann 3f9bcf7cd9 fix: fix NaN handling in parquet stats 2021-06-14 13:44:52 +02:00
Marco Neumann ea96210e98 test: enable unblocked test 2021-06-14 13:44:52 +02:00
kodiakhq[bot] a58f43bf2a
Merge pull request #1707 from influxdata/crepererum/parquet_metadata_wrapper 
refactor: wrap upstream parquet MD into struct + clean up interface
 2021-06-14 11:25:54 +00:00
kodiakhq[bot] fc1b5ea165
Merge branch 'main' into crepererum/parquet_metadata_wrapper 2021-06-14 11:20:39 +00:00
Andrew Lamb 9d1ca95a52
refactor: Rename catalog::Chunk --> catalog::CatalogChunk (#1702) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-06-14 11:20:14 +00:00
Marco Neumann 518f7c6f15 refactor: wrap upstream parquet MD into struct + clean up interface 
This prevents users from `parquet_file::metadata` to also depend on
`parquet` directly. Furthermore they don't need to important dozend of
functions and can instead just use `IoxParquetMetaData` directly.
 2021-06-14 13:17:01 +02:00
kodiakhq[bot] 93cc3cc25f
Merge pull request #1709 from influxdata/er/duplicate_keys 
refactor: more debug logging
 2021-06-14 11:15:20 +00:00
kodiakhq[bot] cf6b658ee3
Merge branch 'main' into er/duplicate_keys 2021-06-14 11:10:45 +00:00
Andrew Lamb 0d8d32fd8f
chore: Update deps to get latest arrow (#1708) 
* chore: Update deps to get latest arrow

* fix: Update to rust 1.52

* fix: clippy
 2021-06-14 11:08:09 +00:00
Edd Robinson 1612ebcbdb refactor: more debug logging 2021-06-14 12:07:51 +01:00
kodiakhq[bot] 963669e052
Merge pull request #1706 from influxdata/er/duplicate_keys 
refactor: read_filter debugging
 2021-06-14 09:34:22 +00:00
Edd Robinson 927d6f890f
Merge branch 'main' into er/duplicate_keys 2021-06-14 10:29:46 +01:00
Edd Robinson 96fb595cc0 refactor: read_filter debugging 2021-06-14 10:22:05 +01:00
kodiakhq[bot] 909d9d3e18
Merge pull request #1696 from influxdata/crepererum/issue1381-b 
feat: wire up catalog checkpointing
 2021-06-14 08:56:42 +00:00
Marco Neumann 030d0d2b9a feat: create checkpoint during catalog rebuild 2021-06-14 10:55:56 +02:00
Marco Neumann 665919786e test: fix test 2021-06-14 10:52:23 +02:00
Marco Neumann f4693e36c0 refactor: `catalog_checkpoint_interval` => `catalog_transactions_until_checkpoint` 2021-06-14 10:34:32 +02:00
Marco Neumann 2eb2aca091 fix: fix discrepancy of ckpting config over CLI and protobuf 2021-06-14 10:27:47 +02:00
Marco Neumann 88ec1ef0cf test: enable checkpointing in catalog benchmark 
This now creates a checkpoint every 10 transactions. To make it a bit
more fair increase the chunk count to 109, so we have some transactions
after the last checkpoint. With that we improve performance from 10.5s
to 1.2s (or even 0.3s if we would keep the chunk count at 100).
 2021-06-14 10:08:32 +02:00
Marco Neumann 2e6f51cbfb fix: fix `server_benchmarks::benches::catalog_persistence` 2021-06-14 10:08:32 +02:00
Marco Neumann 898c638630 feat: wire up catalog checkpointing 
Closes #1381.
 2021-06-14 10:08:32 +02:00
Marco Neumann df866f72e0 refactor: store parquet metadata in chunk 
This will be useful for #1381.

At the moment we parse schema and stats eagerly and store them alongside
the parquet metadata in memory. Technically this is not required since
this is basically duplicate data. In the future we might trade-off some
of this memory against CPU consumption by parsing schema and stats on
demand.
 2021-06-14 10:08:31 +02:00
Marco Neumann e6699ff15a test: ensure that `find_last_transaction_timestamp` considers checkpoints 2021-06-14 10:04:50 +02:00
Marco Neumann eae73591f3 feat: add `catalog_checkpoint_interval` lifecycle config 2021-06-14 10:04:50 +02:00
Nga Tran 11729b9aa7
test: select non-key from 2 chunks with different key/tag sets (#1703) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-06-11 18:52:36 +00:00
kodiakhq[bot] cf523e23b9
Merge pull request #1700 from influxdata/er/refactor/rb_chunk 
refactor: export Read Buffer Chunk as RBChunk
 2021-06-11 17:57:50 +00:00
Edd Robinson ff19beb0ad refactor: export rb chunk as RBChunk 2021-06-11 18:33:10 +01:00
kodiakhq[bot] d7428f568f
Merge pull request #1681 from influxdata/layeredtracing 
feat: Implement LayeredTracing
 2021-06-11 14:19:53 +00:00
kodiakhq[bot] a8759c8b7e
Merge branch 'main' into layeredtracing 2021-06-11 14:15:03 +00:00
kodiakhq[bot] 80db086426
Merge pull request #1693 from influxdata/ntran/dedupe_final_union 
feat: add UnionExec on top of the scan activities
 2021-06-11 13:50:43 +00:00
Nga Tran 736cf1ff6f
Merge branch 'main' into ntran/dedupe_final_union 2021-06-11 09:45:54 -04:00
Nga Tran 7dd0416960 refactor: address review comments 2021-06-11 09:43:39 -04:00