3de6f3f8bd
feat: teach string encoding to production Dictionary values
2021-05-20 10:49:49 +01:00
634ceb886b
feat: add Dictionary Values type
2021-05-20 10:49:49 +01:00
0dc699252f
feat: add with_capacity constructor
2021-05-20 10:49:49 +01:00
b7b87c1c96
test: add read_filter benchmark
2021-05-20 10:49:49 +01:00
4e766d7085
refactor: reorganise benchmarks
2021-05-20 10:49:49 +01:00
c8e2c9224e
chore: rename benchmark
2021-05-20 10:49:49 +01:00
ac5e51b955
Merge pull request #1528 from influxdata/logcachedconnections
...
feat: Log cached connections
2021-05-20 09:38:52 +00:00
7e55544eef
fix: correctly track chunk ID counter during catalog replay
2021-05-20 10:32:40 +02:00
93251f22c7
feat: read perserved catalog during DB startup
...
Closes #1382 .
2021-05-20 10:28:31 +02:00
91d7189e6d
feat: Log cached connections
2021-05-20 10:27:20 +02:00
ccd094dfcf
refactor: streamline directory handling in path parsing
2021-05-20 10:22:20 +02:00
25ec0ab4ca
test: make end2end server fixture restartable
2021-05-20 10:22:20 +02:00
8bdddfd475
docs: mention that catalog wiping does not delete parquet files
2021-05-20 10:22:20 +02:00
b1a06246d6
feat: implement function to wipe a preserved catalog
2021-05-20 10:22:20 +02:00
6c405aa6f9
feat: check if preserved catalog exists when creating an empty one
2021-05-20 10:22:20 +02:00
c6a6005f65
feat: add `PreservedCatalog.exists`
2021-05-20 10:22:20 +02:00
37880ee89a
refactor: store chunk IDs only in catalog ( #1521 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-20 04:07:14 +00:00
00dacb5394
feat: add tests to verify the correctness as well as the explain of the plan
2021-05-19 17:31:16 -04:00
51de37e752
chore: run fmt
2021-05-19 15:28:44 -04:00
11561111d5
chore: merge main to branch
2021-05-19 15:11:15 -04:00
1f13842550
chore: modify comments
2021-05-19 14:49:48 -04:00
087d61f229
feat: Part 1 of predicate push down - Send predicates to MUB, RUB, and Parquet File. Note that MUB has not handled predicates yet
2021-05-19 13:59:51 -04:00
41093b54d6
docs: tweak profiling.md ( #1526 )
2021-05-19 17:04:59 +00:00
3f3e3673ab
Merge pull request #1525 from influxdata/cacheremotes
...
fix: Cache outbound gRPC connections
2021-05-19 16:33:16 +00:00
ce2f8351be
fix: Cache outbound gRPC connections
2021-05-19 18:28:45 +02:00
9b42c1a065
docs: Document how to use pprof tool ( #1520 )
2021-05-19 13:57:50 +00:00
7e223780f3
feat: Implement Display for query::predicate to improve debug printing of plans ( #1519 )
...
* feat: Implement Display for query::predicate to improve debug printing of plans
* fix: clippy
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-19 12:38:34 +00:00
08eb4ad775
Merge pull request #1518 from influxdata/pprofrs
...
feat: Add /debug/pprof/profile
2021-05-19 11:15:54 +00:00
40b21dbca1
feat: Add /debug/pprof/profile
2021-05-19 13:08:42 +02:00
8417301c59
Merge pull request #1514 from influxdata/crepererum/issue1382-b
...
refactor: empty transaction during catalog creation
2021-05-18 16:29:51 +00:00
8db26485a4
refactor: empty transaction during catalog creation
...
That involves some refactoring which we are going to need anyway for
hooking up the "read" path of the catalog into the DB startup, namely:
- make `Db::new` require a preserved catalog
- introduce a helper function that can provide that
- as a consequence, all test-creations of a Db are now async
This prepares for #1382 .
2021-05-18 17:42:07 +02:00
0680a5167f
chore: Improve DataFusion plan logging ( #1508 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-18 11:08:06 +00:00
206ffae5fa
Merge pull request #1505 from influxdata/er/feat/read_buffer/byte_trim_arrow
...
feat: extend Byte trimming to all nullable integer encodings
2021-05-17 21:22:52 +00:00
094de69b57
Merge branch 'main' into er/feat/read_buffer/byte_trim_arrow
2021-05-17 22:18:41 +01:00
c1ea6be101
fix: Add context to panic error on tools ( #1501 )
...
* fix: Add context to panic error on tools
* fix: log new unwrap that has snuck in
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 21:05:01 +00:00
76ee9793dd
Merge pull request #1482 from influxdata/crepererum/issue1382
...
feat: wire up catalog preservation write path
2021-05-17 18:01:46 +00:00
c3cc58b2ff
Merge branch 'main' into crepererum/issue1382
2021-05-17 17:57:26 +00:00
4f0e46bcd5
refactor: track ingest metrics in one place ( #1503 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 16:25:01 +00:00
2963d63b5e
feat: implement byte trimming on nullable encodings
2021-05-17 14:32:55 +01:00
6a72274517
feat: extend implementations to more Arrow arrays
2021-05-17 14:32:55 +01:00
2b98bca9ca
test: allow from slice to be testable
2021-05-17 14:32:55 +01:00
b7ea53f5db
refactor: remove unnecessary from imps
2021-05-17 14:32:55 +01:00
18f0a7f614
docs: reference open issue
2021-05-17 14:01:51 +02:00
cdf0ada6a6
test: test preserved catalog <-> Db write wiring
2021-05-17 13:57:31 +02:00
91a45fd380
feat: simplify shutdown ( #1502 )
...
* feat: simplify shutdown
* chore: fix lint
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 11:50:14 +00:00
4299371cf2
refactor: remove some code
2021-05-17 12:32:48 +02:00
68729dd5ee
refactor: avoid string allocation
2021-05-17 12:32:34 +02:00
adcd8132e7
docs: more comments regarding catalog transaction handling
2021-05-17 12:05:08 +02:00
a99d53e771
docs: document `OpenTransaction::handle_action*`
2021-05-17 11:48:51 +02:00
840c11dab2
feat: wire up catalog preservation write path
...
Required a bit of refactoring:
- Add an extra layer between DB an catalog which is the "preserved
catalog" wrapper. This is required to make the ownership model
somewhat sane, because during the read operations the "preserved
catalog" is going to act on the in-mem catalog.
- Move "parquet file written" logic into binding `preserved catalog <->
catalog state`, so we have a single place where new parquet files are
announced. For now this only works for chunks that are already known
(i.e. the writing->written transation when coming from read buffer),
however in the next PR this will be extended to also handle totally
new parquet files during transaction playback.
**NOTE: This does NOT include the read path yet!**
Issue: #1382 .
2021-05-17 11:33:22 +02:00
Edd Robinson
kodiakhq[bot]
Marco Neumann
Marko Mikulicic
Raphael Taylor-Davies
Nga Tran
Andrew Lamb