aa90329c1f
feat: Add remote_template for simpler remote configuration
2021-05-20 12:45:08 +02:00
8c3e597cf3
Merge pull request #1524 from influxdata/er/perf/read_buffer_dict_read_filter
...
perf: teach Read Buffer to materialise string column results in dictionary format
2021-05-20 10:15:56 +00:00
4cb76e367b
refactor: fix change to Chunk API
2021-05-20 11:11:18 +01:00
663a38862d
refactor: address PR feedback
2021-05-20 10:49:49 +01:00
76caef89b1
refactor: apply suggestions from code review
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-05-20 10:49:49 +01:00
c901fe1023
perf: improve values_as_dictionary with predicates
2021-05-20 10:49:49 +01:00
723ff2553b
feat: teach read_filter to return dictionaries
2021-05-20 10:49:49 +01:00
3de6f3f8bd
feat: teach string encoding to production Dictionary values
2021-05-20 10:49:49 +01:00
634ceb886b
feat: add Dictionary Values type
2021-05-20 10:49:49 +01:00
0dc699252f
feat: add with_capacity constructor
2021-05-20 10:49:49 +01:00
b7b87c1c96
test: add read_filter benchmark
2021-05-20 10:49:49 +01:00
4e766d7085
refactor: reorganise benchmarks
2021-05-20 10:49:49 +01:00
c8e2c9224e
chore: rename benchmark
2021-05-20 10:49:49 +01:00
ac5e51b955
Merge pull request #1528 from influxdata/logcachedconnections
...
feat: Log cached connections
2021-05-20 09:38:52 +00:00
91d7189e6d
feat: Log cached connections
2021-05-20 10:27:20 +02:00
37880ee89a
refactor: store chunk IDs only in catalog ( #1521 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-20 04:07:14 +00:00
41093b54d6
docs: tweak profiling.md ( #1526 )
2021-05-19 17:04:59 +00:00
3f3e3673ab
Merge pull request #1525 from influxdata/cacheremotes
...
fix: Cache outbound gRPC connections
2021-05-19 16:33:16 +00:00
ce2f8351be
fix: Cache outbound gRPC connections
2021-05-19 18:28:45 +02:00
9b42c1a065
docs: Document how to use pprof tool ( #1520 )
2021-05-19 13:57:50 +00:00
7e223780f3
feat: Implement Display for query::predicate to improve debug printing of plans ( #1519 )
...
* feat: Implement Display for query::predicate to improve debug printing of plans
* fix: clippy
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-19 12:38:34 +00:00
08eb4ad775
Merge pull request #1518 from influxdata/pprofrs
...
feat: Add /debug/pprof/profile
2021-05-19 11:15:54 +00:00
40b21dbca1
feat: Add /debug/pprof/profile
2021-05-19 13:08:42 +02:00
8417301c59
Merge pull request #1514 from influxdata/crepererum/issue1382-b
...
refactor: empty transaction during catalog creation
2021-05-18 16:29:51 +00:00
8db26485a4
refactor: empty transaction during catalog creation
...
That involves some refactoring which we are going to need anyway for
hooking up the "read" path of the catalog into the DB startup, namely:
- make `Db::new` require a preserved catalog
- introduce a helper function that can provide that
- as a consequence, all test-creations of a Db are now async
This prepares for #1382 .
2021-05-18 17:42:07 +02:00
0680a5167f
chore: Improve DataFusion plan logging ( #1508 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-18 11:08:06 +00:00
206ffae5fa
Merge pull request #1505 from influxdata/er/feat/read_buffer/byte_trim_arrow
...
feat: extend Byte trimming to all nullable integer encodings
2021-05-17 21:22:52 +00:00
094de69b57
Merge branch 'main' into er/feat/read_buffer/byte_trim_arrow
2021-05-17 22:18:41 +01:00
c1ea6be101
fix: Add context to panic error on tools ( #1501 )
...
* fix: Add context to panic error on tools
* fix: log new unwrap that has snuck in
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 21:05:01 +00:00
76ee9793dd
Merge pull request #1482 from influxdata/crepererum/issue1382
...
feat: wire up catalog preservation write path
2021-05-17 18:01:46 +00:00
c3cc58b2ff
Merge branch 'main' into crepererum/issue1382
2021-05-17 17:57:26 +00:00
4f0e46bcd5
refactor: track ingest metrics in one place ( #1503 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 16:25:01 +00:00
2963d63b5e
feat: implement byte trimming on nullable encodings
2021-05-17 14:32:55 +01:00
6a72274517
feat: extend implementations to more Arrow arrays
2021-05-17 14:32:55 +01:00
2b98bca9ca
test: allow from slice to be testable
2021-05-17 14:32:55 +01:00
b7ea53f5db
refactor: remove unnecessary from imps
2021-05-17 14:32:55 +01:00
18f0a7f614
docs: reference open issue
2021-05-17 14:01:51 +02:00
cdf0ada6a6
test: test preserved catalog <-> Db write wiring
2021-05-17 13:57:31 +02:00
91a45fd380
feat: simplify shutdown ( #1502 )
...
* feat: simplify shutdown
* chore: fix lint
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 11:50:14 +00:00
4299371cf2
refactor: remove some code
2021-05-17 12:32:48 +02:00
68729dd5ee
refactor: avoid string allocation
2021-05-17 12:32:34 +02:00
adcd8132e7
docs: more comments regarding catalog transaction handling
2021-05-17 12:05:08 +02:00
a99d53e771
docs: document `OpenTransaction::handle_action*`
2021-05-17 11:48:51 +02:00
840c11dab2
feat: wire up catalog preservation write path
...
Required a bit of refactoring:
- Add an extra layer between DB an catalog which is the "preserved
catalog" wrapper. This is required to make the ownership model
somewhat sane, because during the read operations the "preserved
catalog" is going to act on the in-mem catalog.
- Move "parquet file written" logic into binding `preserved catalog <->
catalog state`, so we have a single place where new parquet files are
announced. For now this only works for chunks that are already known
(i.e. the writing->written transation when coming from read buffer),
however in the next PR this will be extended to also handle totally
new parquet files during transaction playback.
**NOTE: This does NOT include the read path yet!**
Issue: #1382 .
2021-05-17 11:33:22 +02:00
4fb800c7a6
refactor: make PreservedCatalog easier to integrate
2021-05-17 11:33:22 +02:00
f4d7154746
fix: table summaries must include timestamp as well
2021-05-17 11:33:22 +02:00
ff004c1a51
feat: add `ObjectStore.path_from_dirs_and_filename`
2021-05-17 11:33:22 +02:00
7cced3242f
feat: add a way to parse infos from parquet paths
2021-05-17 11:33:22 +02:00
5969caccb0
feat: return parquet metadata from `write_to_object_store`
2021-05-17 11:33:22 +02:00
07db4932ee
refactor: rename data_types/src/chunk.rs -> data_types/src/chunk_metadata.rs ( #1500 )
2021-05-15 10:18:01 +00:00
Marko Mikulicic
kodiakhq[bot]
Edd Robinson
Raphael Taylor-Davies
Andrew Lamb
Marco Neumann