Commit Graph

2952 Commits (f028a356f4e2c7190f76c9169f9d4b61f2c3d675)

Author SHA1 Message Date
kodiakhq[bot] f028a356f4
Merge branch 'main' into crepererum/issue1382-c 2021-05-20 15:51:47 +00:00
kodiakhq[bot] 0f53937779
Merge pull request #1529 from influxdata/crepererum/remove_snapshotting
feat: remove snapshot feature
2021-05-20 14:20:05 +00:00
kodiakhq[bot] aac00d4fa6
Merge branch 'main' into crepererum/remove_snapshotting 2021-05-20 14:14:58 +00:00
kodiakhq[bot] d89455459e
Merge pull request #1530 from influxdata/detremotemap
feat: Add remote_template for simpler remote configuration
2021-05-20 13:03:24 +00:00
Marco Neumann 0e37d500eb feat: remove snapshot feature
The parquet files produced by this code path are only semi-specified and
will miss many important metadata aspects that we will require for data
lineage.
2021-05-20 14:59:04 +02:00
Marko Mikulicic 462a5590c6
fix: fmt 2021-05-20 14:58:50 +02:00
Marko Mikulicic c908cf0f98
fix: review suggestion
Co-authored-by: Edd Robinson <me@edd.io>
2021-05-20 14:40:02 +02:00
Marko Mikulicic 172530f7db
fix: review suggestion
Co-authored-by: Edd Robinson <me@edd.io>
2021-05-20 14:39:50 +02:00
Marco Neumann e28c031f76 refactor: make `ServerFixture::restart_server` consume the fixture
This also avoids an additional channel lock.
2021-05-20 14:30:30 +02:00
Marco Neumann 464667d8b8 docs: clarify the FilePath directory handling 2021-05-20 14:20:56 +02:00
Marko Mikulicic aa90329c1f
feat: Add remote_template for simpler remote configuration 2021-05-20 12:45:08 +02:00
kodiakhq[bot] 8c3e597cf3
Merge pull request #1524 from influxdata/er/perf/read_buffer_dict_read_filter
perf: teach Read Buffer to materialise string column results in dictionary format
2021-05-20 10:15:56 +00:00
Edd Robinson 4cb76e367b refactor: fix change to Chunk API 2021-05-20 11:11:18 +01:00
Edd Robinson 663a38862d refactor: address PR feedback 2021-05-20 10:49:49 +01:00
Edd Robinson 76caef89b1 refactor: apply suggestions from code review
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-05-20 10:49:49 +01:00
Edd Robinson c901fe1023 perf: improve values_as_dictionary with predicates 2021-05-20 10:49:49 +01:00
Edd Robinson 723ff2553b feat: teach read_filter to return dictionaries 2021-05-20 10:49:49 +01:00
Edd Robinson 3de6f3f8bd feat: teach string encoding to production Dictionary values 2021-05-20 10:49:49 +01:00
Edd Robinson 634ceb886b feat: add Dictionary Values type 2021-05-20 10:49:49 +01:00
Edd Robinson 0dc699252f feat: add with_capacity constructor 2021-05-20 10:49:49 +01:00
Edd Robinson b7b87c1c96 test: add read_filter benchmark 2021-05-20 10:49:49 +01:00
Edd Robinson 4e766d7085 refactor: reorganise benchmarks 2021-05-20 10:49:49 +01:00
Edd Robinson c8e2c9224e chore: rename benchmark 2021-05-20 10:49:49 +01:00
kodiakhq[bot] ac5e51b955
Merge pull request #1528 from influxdata/logcachedconnections
feat: Log cached connections
2021-05-20 09:38:52 +00:00
Marco Neumann 7e55544eef fix: correctly track chunk ID counter during catalog replay 2021-05-20 10:32:40 +02:00
Marco Neumann 93251f22c7 feat: read perserved catalog during DB startup
Closes #1382.
2021-05-20 10:28:31 +02:00
Marko Mikulicic 91d7189e6d
feat: Log cached connections 2021-05-20 10:27:20 +02:00
Marco Neumann ccd094dfcf refactor: streamline directory handling in path parsing 2021-05-20 10:22:20 +02:00
Marco Neumann 25ec0ab4ca test: make end2end server fixture restartable 2021-05-20 10:22:20 +02:00
Marco Neumann 8bdddfd475 docs: mention that catalog wiping does not delete parquet files 2021-05-20 10:22:20 +02:00
Marco Neumann b1a06246d6 feat: implement function to wipe a preserved catalog 2021-05-20 10:22:20 +02:00
Marco Neumann 6c405aa6f9 feat: check if preserved catalog exists when creating an empty one 2021-05-20 10:22:20 +02:00
Marco Neumann c6a6005f65 feat: add `PreservedCatalog.exists` 2021-05-20 10:22:20 +02:00
Raphael Taylor-Davies 37880ee89a
refactor: store chunk IDs only in catalog (#1521)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-20 04:07:14 +00:00
Andrew Lamb 41093b54d6
docs: tweak profiling.md (#1526) 2021-05-19 17:04:59 +00:00
kodiakhq[bot] 3f3e3673ab
Merge pull request #1525 from influxdata/cacheremotes
fix: Cache outbound gRPC connections
2021-05-19 16:33:16 +00:00
Marko Mikulicic ce2f8351be
fix: Cache outbound gRPC connections 2021-05-19 18:28:45 +02:00
Andrew Lamb 9b42c1a065
docs: Document how to use pprof tool (#1520) 2021-05-19 13:57:50 +00:00
Andrew Lamb 7e223780f3
feat: Implement Display for query::predicate to improve debug printing of plans (#1519)
* feat: Implement Display for query::predicate to improve debug printing of plans

* fix: clippy

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-19 12:38:34 +00:00
kodiakhq[bot] 08eb4ad775
Merge pull request #1518 from influxdata/pprofrs
feat: Add /debug/pprof/profile
2021-05-19 11:15:54 +00:00
Marko Mikulicic 40b21dbca1
feat: Add /debug/pprof/profile 2021-05-19 13:08:42 +02:00
kodiakhq[bot] 8417301c59
Merge pull request #1514 from influxdata/crepererum/issue1382-b
refactor: empty transaction during catalog creation
2021-05-18 16:29:51 +00:00
Marco Neumann 8db26485a4 refactor: empty transaction during catalog creation
That involves some refactoring which we are going to need anyway for
hooking up the "read" path of the catalog into the DB startup, namely:

- make `Db::new` require a preserved catalog
- introduce a helper function that can provide that
- as a consequence, all test-creations of a Db are now async

This prepares for #1382.
2021-05-18 17:42:07 +02:00
Andrew Lamb 0680a5167f
chore: Improve DataFusion plan logging (#1508)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-18 11:08:06 +00:00
kodiakhq[bot] 206ffae5fa
Merge pull request #1505 from influxdata/er/feat/read_buffer/byte_trim_arrow
feat: extend Byte trimming to all nullable integer encodings
2021-05-17 21:22:52 +00:00
Edd Robinson 094de69b57
Merge branch 'main' into er/feat/read_buffer/byte_trim_arrow 2021-05-17 22:18:41 +01:00
Andrew Lamb c1ea6be101
fix: Add context to panic error on tools (#1501)
* fix: Add context to panic error on tools

* fix: log new unwrap that has snuck in

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 21:05:01 +00:00
kodiakhq[bot] 76ee9793dd
Merge pull request #1482 from influxdata/crepererum/issue1382
feat: wire up catalog preservation write path
2021-05-17 18:01:46 +00:00
kodiakhq[bot] c3cc58b2ff
Merge branch 'main' into crepererum/issue1382 2021-05-17 17:57:26 +00:00
Raphael Taylor-Davies 4f0e46bcd5
refactor: track ingest metrics in one place (#1503)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 16:25:01 +00:00