07fdbe7c6b
fix: apply `_field` restrictions that do not match any fields ( #3903 )
...
* fix: apply field restriction correctly
* refactor: Use Vec::retain
2022-03-03 15:04:05 +00:00
b9ffc5bcbe
Merge pull request #3913 from influxdata/er/feat/rpc_tracing
...
feat: add tracing to all remaining influx RPC calls
2022-03-03 14:41:34 +00:00
de7c46c9bb
feat: add read_window_aggregate tracing
2022-03-03 14:30:27 +00:00
ea32bc366a
feat: add read_group tracing
2022-03-03 14:27:01 +00:00
32baaa1ee7
feat: add tracing to field_columns
2022-03-03 14:27:01 +00:00
787a848bf5
feat: add tracing for tag_values
2022-03-03 14:27:01 +00:00
6a6fbf73ae
feat: add tracing support tag_keys
2022-03-03 14:27:01 +00:00
998e205c2c
feat: trace table_names
2022-03-03 14:27:01 +00:00
301ae886ce
feat: add tracing down to the chunk level ( #3804 )
...
* refactor: wire exectution context to Deduplicator
* feat: example trace to chunk read_filter
* refactor: make execution context required
* refactor: expose metadata API
* refactor: more span context for chunk read_filter
* refactor: fix build
* refactor: push context into result stream
* refactor: make executor optional
2022-03-03 14:27:00 +00:00
208c3673ad
Merge pull request #3901 from influxdata/dom/rustls
...
feat: rustls as TLS backend
2022-03-03 12:11:01 +00:00
04a7a957fe
Merge branch 'main' into dom/rustls
2022-03-03 11:59:40 +00:00
9ed9426bf5
Merge pull request #3902 from influxdata/dom/fix-feature-flags
...
fix: jemalloc feature flag
2022-03-03 11:59:09 +00:00
ab010f9d82
fix: regenerate workspace hack config
2022-03-03 11:55:00 +00:00
c0ebe5a2f1
Merge branch 'main' into dom/fix-feature-flags
2022-03-03 11:48:35 +00:00
6204a9046d
fix: jemalloc feature flag
...
In #2435 we moved the jemalloc metrics around, and missed a few feature
gates to avoid pulling in jemalloc unless it is requested.
2022-03-03 11:48:24 +00:00
02fc20d638
Merge pull request #3897 from influxdata/dom/batch-column-upsert
...
feat: batch column upsert
2022-03-03 11:41:13 +00:00
03ffc03c4b
Merge branch 'main' into dom/batch-column-upsert
2022-03-03 11:29:44 +00:00
8de453edd1
feat: batch column upsert for schema validation
...
Uses the new ColumnRepo::create_or_get_many() catalog method to perform
a bulk upsert of (potentially) new columns to the catalog during schema
validation.
2022-03-03 11:18:29 +00:00
0660b514e3
feat: allow persisting all partitions and/or all tables from CLI ( #3874 )
...
* feat: allow persisting all partitions and/or all tables
* chore: use BTreeSet
* feat: non-zero exit code on error continue
* test: test continue-on-error
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-03 11:17:34 +00:00
da145ffbe4
feat: batch upsert of columns to catalog
...
Adds ColumnRepo::create_or_get_many() to upsert multiple columns in one
round trip to Postgres.
2022-03-03 11:17:30 +00:00
89dba1b7d1
docs: metric decorate macro
...
Some helpful tips for using the metric decorator's decorate! macro when
modifying the catalog traits.
2022-03-03 11:14:57 +00:00
b391cdde5e
docs: remove openssl references
2022-03-03 11:05:20 +00:00
46bb107be4
refactor: use rustls
...
Removes openssl as a dependency, switching to rustls[1] as the TLS
implementation throughout.
It is important to note that this change brings with it a significant
behavioural difference - rustls does not currently support IP SANs in
certificates (instead only supporting fully-qualified names / DNS) and
this will manifest as a failure to connect to IP endpoints over TLS.
This might be a blocker that prevents us using rustls exclusively, but
there's noe asy way to know without trying it. Fortunately the rustls
project has received funding to work on IP SAN support[2].
[1]: https://github.com/rustls/rustls
[2]: https://www.abetterinternet.org/post/preparing-rustls-for-wider-adoption/
2022-03-03 11:05:20 +00:00
f2126c76d3
refactor: address PR feedback ( #3906 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-03 11:04:05 +00:00
4785a7d028
refactor: bump Azure SDK
...
Update to the latest Azure SDK to pick up rustls support.
2022-03-03 11:02:20 +00:00
02a2cfde54
feat: update rskafka ( #3805 ) ( #3910 )
2022-03-03 10:53:47 +00:00
bbeba73345
feat: sync partitions in querier ( #3900 )
...
* feat: sync partitions in querier
* docs: explain per-table partition grouping
2022-03-03 09:28:44 +00:00
ea4a245697
Merge pull request #3786 from influxdata/cn/querier-flight-request
...
feat: Querier flight request to Ingester
2022-03-02 20:40:10 +00:00
caba3e9fd2
Merge branch 'main' into cn/querier-flight-request
2022-03-02 20:30:00 +00:00
806a43eb6c
fix: Temporarily remove ingester end-to-end test case
...
This now needs more setup because without any data, the flight query
request returns an error. The setup is proving to be nontrivial.
2022-03-02 15:29:48 -05:00
3d047073b9
feat: add tracing down to the chunk level ( #3804 )
...
* refactor: wire exectution context to Deduplicator
* feat: example trace to chunk read_filter
* refactor: make execution context required
* refactor: expose metadata API
* refactor: more span context for chunk read_filter
* refactor: fix build
* refactor: push context into result stream
* refactor: make executor optional
2022-03-02 19:08:22 +00:00
3f2a58b47f
refactor: pub use data_types from data_types2
...
So it's clearer which parts of data_types the NG design is using, and
which types can be cleaned up eventually.
2022-03-02 13:55:31 -05:00
2a90841715
refactor: Move IngesterQueryRequest to data_types2
...
So that querier doesn't need to depend on ingester.
2022-03-02 13:52:13 -05:00
8f3e44bf76
refactor: Extract a crate for shared data types in the new design
2022-03-02 12:16:15 -05:00
16d86ed05b
feat: Deserialize metadata to get max_sequencer_number
...
And add an end-to-end test for the flight request to the ingester.
2022-03-02 11:50:47 -05:00
141a6087d0
feat: Querier able to send Flight queries to Ingester
...
Fixes #3773 .
2022-03-02 11:50:45 -05:00
2fd68ea75f
feat: sync tables and schemas in querier ( #3895 )
...
* feat: convert `iox_catalog` schema to `schema::Schema`
* fix: remove leftover println statements
* feat: sync tables and schemas in querier
* feat: `PartitionRepo::list_by_namespace`
* docs: explain `QuerierNamespace` data structs a bit
* refactor: improve variable naming
* test: extend `test_sync_schemas
* fix: do not block forever when namespace is gone
2022-03-02 15:32:03 +00:00
286d5f7b2b
feat: add `success` column to system.queries ( #3891 )
...
* feat: add `success` column to system.queries
* refactor: Remove lifetime from QueryCompletedToken and thread through flight
* test: update test to make incomplete query clearer
* refactor: use better patter to set complete
* fix: logical merge conflict
2022-03-02 15:05:06 +00:00
1d6200a80d
fix: make catalog db migration idempotent again ( #3894 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-02 13:56:42 +00:00
1e9164c3fd
Merge pull request #3898 from influxdata/dom/ingest-metrics
...
feat: http ingest metrics
2022-03-02 13:15:58 +00:00
bd64f55658
feat: http ingest metrics
...
Records LP line count, field count & request body size (decompressed,
byte size) for writes, and request body byte size for deletes.
2022-03-02 13:05:55 +00:00
44cfdc7aca
feat: update to latest rskafka logging ( #3805 ) ( #3896 )
2022-03-02 11:05:03 +00:00
af57664f53
feat: wire up flight into the querier ( #3889 )
2022-03-02 09:20:26 +00:00
936f51013d
test: querier shutdown and background task handling ( #3881 )
...
This is similar to what we already have for the ingester.
2022-03-02 08:59:50 +00:00
5f1095ef7d
Merge pull request #3880 from influxdata/crepererum/querier_cleanup
...
refactor: clean up `querier` a bit
2022-03-01 20:02:47 +00:00
14a3f33424
Merge branch 'main' into crepererum/querier_cleanup
2022-03-01 19:51:38 +00:00
39e39f92a8
fix: remove rskafka from valid options for INFLUXDB_IOX_WRITE_BUFFER_TYPE ( #3890 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-01 17:29:02 +00:00
143311f63f
feat: additional write buffer logging ( #3805 ) ( #3887 )
...
* feat: additional write buffer logging (#3805 )
* fix: assertion direction
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-01 17:08:57 +00:00
63bea75ee1
feat: split release build and deploy ( #3879 )
...
* feat: split release build and deploy
* fix: only build_release on main
* chore: fix build_perf CI
* fix: deploy_release checkout
* fix: update deploy_release image
* fix: deploy_release credentials
* fix: add COMMIT_SHA to env
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-01 16:35:39 +00:00
01f4d3ed3b
fix: check arrow instead of storage gRPC health for SQL REPL ( #3886 )
...
There is no reason a query pod should support the storage API. Note that
some features like the observer mode or `show databases;` still need the
management API. We'll probably need to fix that for NG at some point.
2022-03-01 15:46:02 +00:00
Andrew Lamb
kodiakhq[bot]
Edd Robinson
Dom Dwyer
Raphael Taylor-Davies
Marco Neumann
Carol (Nichols || Goulding)
Marko Mikulicic