Commit Graph

2993 Commits (7f4b181c67d10f132097cfadf373c0fd5b9e8a4d)

Author SHA1 Message Date
Edd Robinson 76caef89b1 refactor: apply suggestions from code review
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-05-20 10:49:49 +01:00
Edd Robinson c901fe1023 perf: improve values_as_dictionary with predicates 2021-05-20 10:49:49 +01:00
Edd Robinson 723ff2553b feat: teach read_filter to return dictionaries 2021-05-20 10:49:49 +01:00
Edd Robinson 3de6f3f8bd feat: teach string encoding to production Dictionary values 2021-05-20 10:49:49 +01:00
Edd Robinson 634ceb886b feat: add Dictionary Values type 2021-05-20 10:49:49 +01:00
Edd Robinson 0dc699252f feat: add with_capacity constructor 2021-05-20 10:49:49 +01:00
Edd Robinson b7b87c1c96 test: add read_filter benchmark 2021-05-20 10:49:49 +01:00
Edd Robinson 4e766d7085 refactor: reorganise benchmarks 2021-05-20 10:49:49 +01:00
Edd Robinson c8e2c9224e chore: rename benchmark 2021-05-20 10:49:49 +01:00
kodiakhq[bot] ac5e51b955
Merge pull request #1528 from influxdata/logcachedconnections
feat: Log cached connections
2021-05-20 09:38:52 +00:00
Marco Neumann 7e55544eef fix: correctly track chunk ID counter during catalog replay 2021-05-20 10:32:40 +02:00
Marco Neumann 93251f22c7 feat: read perserved catalog during DB startup
Closes #1382.
2021-05-20 10:28:31 +02:00
Marko Mikulicic 91d7189e6d
feat: Log cached connections 2021-05-20 10:27:20 +02:00
Marco Neumann ccd094dfcf refactor: streamline directory handling in path parsing 2021-05-20 10:22:20 +02:00
Marco Neumann 25ec0ab4ca test: make end2end server fixture restartable 2021-05-20 10:22:20 +02:00
Marco Neumann 8bdddfd475 docs: mention that catalog wiping does not delete parquet files 2021-05-20 10:22:20 +02:00
Marco Neumann b1a06246d6 feat: implement function to wipe a preserved catalog 2021-05-20 10:22:20 +02:00
Marco Neumann 6c405aa6f9 feat: check if preserved catalog exists when creating an empty one 2021-05-20 10:22:20 +02:00
Marco Neumann c6a6005f65 feat: add `PreservedCatalog.exists` 2021-05-20 10:22:20 +02:00
Raphael Taylor-Davies 37880ee89a
refactor: store chunk IDs only in catalog (#1521)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-20 04:07:14 +00:00
Nga Tran 00dacb5394 feat: add tests to verify the correctness as well as the explain of the plan 2021-05-19 17:31:16 -04:00
Nga Tran 51de37e752 chore: run fmt 2021-05-19 15:28:44 -04:00
Nga Tran 11561111d5 chore: merge main to branch 2021-05-19 15:11:15 -04:00
Nga Tran 1f13842550 chore: modify comments 2021-05-19 14:49:48 -04:00
Nga Tran 087d61f229 feat: Part 1 of predicate push down - Send predicates to MUB, RUB, and Parquet File. Note that MUB has not handled predicates yet 2021-05-19 13:59:51 -04:00
Andrew Lamb 41093b54d6
docs: tweak profiling.md (#1526) 2021-05-19 17:04:59 +00:00
kodiakhq[bot] 3f3e3673ab
Merge pull request #1525 from influxdata/cacheremotes
fix: Cache outbound gRPC connections
2021-05-19 16:33:16 +00:00
Marko Mikulicic ce2f8351be
fix: Cache outbound gRPC connections 2021-05-19 18:28:45 +02:00
Andrew Lamb 9b42c1a065
docs: Document how to use pprof tool (#1520) 2021-05-19 13:57:50 +00:00
Andrew Lamb 7e223780f3
feat: Implement Display for query::predicate to improve debug printing of plans (#1519)
* feat: Implement Display for query::predicate to improve debug printing of plans

* fix: clippy

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-19 12:38:34 +00:00
kodiakhq[bot] 08eb4ad775
Merge pull request #1518 from influxdata/pprofrs
feat: Add /debug/pprof/profile
2021-05-19 11:15:54 +00:00
Marko Mikulicic 40b21dbca1
feat: Add /debug/pprof/profile 2021-05-19 13:08:42 +02:00
kodiakhq[bot] 8417301c59
Merge pull request #1514 from influxdata/crepererum/issue1382-b
refactor: empty transaction during catalog creation
2021-05-18 16:29:51 +00:00
Marco Neumann 8db26485a4 refactor: empty transaction during catalog creation
That involves some refactoring which we are going to need anyway for
hooking up the "read" path of the catalog into the DB startup, namely:

- make `Db::new` require a preserved catalog
- introduce a helper function that can provide that
- as a consequence, all test-creations of a Db are now async

This prepares for #1382.
2021-05-18 17:42:07 +02:00
Andrew Lamb 0680a5167f
chore: Improve DataFusion plan logging (#1508)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-18 11:08:06 +00:00
kodiakhq[bot] 206ffae5fa
Merge pull request #1505 from influxdata/er/feat/read_buffer/byte_trim_arrow
feat: extend Byte trimming to all nullable integer encodings
2021-05-17 21:22:52 +00:00
Edd Robinson 094de69b57
Merge branch 'main' into er/feat/read_buffer/byte_trim_arrow 2021-05-17 22:18:41 +01:00
Andrew Lamb c1ea6be101
fix: Add context to panic error on tools (#1501)
* fix: Add context to panic error on tools

* fix: log new unwrap that has snuck in

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 21:05:01 +00:00
kodiakhq[bot] 76ee9793dd
Merge pull request #1482 from influxdata/crepererum/issue1382
feat: wire up catalog preservation write path
2021-05-17 18:01:46 +00:00
kodiakhq[bot] c3cc58b2ff
Merge branch 'main' into crepererum/issue1382 2021-05-17 17:57:26 +00:00
Raphael Taylor-Davies 4f0e46bcd5
refactor: track ingest metrics in one place (#1503)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 16:25:01 +00:00
Edd Robinson 2963d63b5e feat: implement byte trimming on nullable encodings 2021-05-17 14:32:55 +01:00
Edd Robinson 6a72274517 feat: extend implementations to more Arrow arrays 2021-05-17 14:32:55 +01:00
Edd Robinson 2b98bca9ca test: allow from slice to be testable 2021-05-17 14:32:55 +01:00
Edd Robinson b7ea53f5db refactor: remove unnecessary from imps 2021-05-17 14:32:55 +01:00
Marco Neumann 18f0a7f614 docs: reference open issue 2021-05-17 14:01:51 +02:00
Marco Neumann cdf0ada6a6 test: test preserved catalog <-> Db write wiring 2021-05-17 13:57:31 +02:00
Raphael Taylor-Davies 91a45fd380
feat: simplify shutdown (#1502)
* feat: simplify shutdown

* chore: fix lint

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 11:50:14 +00:00
Marco Neumann 4299371cf2 refactor: remove some code 2021-05-17 12:32:48 +02:00
Marco Neumann 68729dd5ee refactor: avoid string allocation 2021-05-17 12:32:34 +02:00