455a051653
fix: Remove edit db rules scripts that no longer apply to NG
2022-04-29 16:29:52 -04:00
a4443e4c31
fix: Remove OG gRPC client code and APIs
2022-04-29 16:29:49 -04:00
8245ff83ac
Merge pull request #4482 from influxdata/cn/switch-list-databases
...
fix: Switch `list_database_names` to `get_namespaces`
2022-04-29 19:25:25 +00:00
8e21d062fb
fix: Remove use of OG list databases from sql observer mode
...
Fixes #4478 .
2022-04-29 14:33:48 -04:00
ede51d2529
fix: Remove OG 'show databases' command in the SQL REPL
...
Connects to #4478 .
2022-04-29 14:33:22 -04:00
bc77f87ea9
feat: port influxrpc read_group and read_window_aggregate e2e test to NG ( #4464 )
...
port more
2022-04-29 16:13:32 +00:00
b6af396498
Merge pull request #4477 from influxdata/cn/delete-og-e2e-tests
...
fix: Delete OG end-to-end tests
2022-04-29 15:02:23 +00:00
8436e083f7
Merge branch 'main' into cn/delete-og-e2e-tests
2022-04-29 14:53:47 +00:00
bb7910f689
fix: Remove influxdb_iox unused dev deps
2022-04-29 10:32:40 -04:00
e5cbee9fa8
fix: Delete OG end-to-end tests
2022-04-29 10:32:21 -04:00
4813cc8332
test: Added explain tests for querier. Found and fixed #4468 ( #4469 )
...
* test: Added explain tests for querier. Found and fixed #4468
* chore: cleanup
* chore: Apply suggestions from code review
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-29 14:15:30 +00:00
6381ea60bb
chore: port remaining read_filter influxrpc tests to NG ( #4383 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-29 14:06:50 +00:00
a81fc6f641
Merge pull request #4446 from influxdata/dom/schema-cache-warm
...
feat(router2): pre-warm namespace schema cache
2022-04-29 13:27:50 +00:00
fe3a48b3b1
Merge branch 'main' into dom/schema-cache-warm
2022-04-29 13:19:47 +00:00
1e38426484
docs: Update data_organization_lifecycle.md ( #4456 )
2022-04-29 13:19:26 +00:00
7c7d3fafe9
Merge branch 'main' into dom/schema-cache-warm
2022-04-29 09:11:53 -04:00
0a20086a58
feat: expose catalog timeouts via CLI/env ( #4472 )
...
This is useful for local instances that run against a prod system,
because port forwarding can lead to long connection delays.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-29 11:14:15 +00:00
ed1ad858c0
chore: add some logging to flaky end to end test ( #4465 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-29 11:06:06 +00:00
b23fe9d141
chore(deps): Bump syn from 1.0.91 to 1.0.92 ( #4471 )
...
Bumps [syn](https://github.com/dtolnay/syn ) from 1.0.91 to 1.0.92.
- [Release notes](https://github.com/dtolnay/syn/releases )
- [Commits](https://github.com/dtolnay/syn/compare/1.0.91...1.0.92 )
---
updated-dependencies:
- dependency-name: syn
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-29 10:43:25 +00:00
01a6cf1c2a
chore(deps): Bump http from 0.2.6 to 0.2.7 ( #4470 )
...
Bumps [http](https://github.com/hyperium/http ) from 0.2.6 to 0.2.7.
- [Release notes](https://github.com/hyperium/http/releases )
- [Changelog](https://github.com/hyperium/http/blob/master/CHANGELOG.md )
- [Commits](https://github.com/hyperium/http/compare/v0.2.6...v0.2.7 )
---
updated-dependencies:
- dependency-name: http
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-29 09:56:48 +00:00
8e48fcd620
feat: add remote pull partition ( #4433 )
...
Add lookup of partitions by table id to catalog.
Add API to catalog to return partitions by table id.
Add to client to return partitions by table id.
Add CLI to pull remote schema, partition, and parquet files into a local catalog and object store.
2022-04-28 21:04:27 +00:00
7b6754cc03
chore: Update dependencies via `cargo update` ( #4460 )
...
* chore: Update dependencies via `cargo update`
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-04-28 18:48:51 +00:00
63df3ceb6f
feat: share server processes in end to end test ( #4387 )
...
* feat: share server processes in end to end test
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-28 17:51:14 +00:00
5688bd63a3
docs: document for deduplication and sort key ( #4452 )
...
* docs: document for deduplication and sort key
* chore: Apply suggestions from code review
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Luke Bond <luke.n.bond@gmail.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* chore: Update query/src/provider.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* chore: address review comments
* fix: fix comment
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Luke Bond <luke.n.bond@gmail.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-28 16:44:40 +00:00
e433c6c704
refactor: use with_timeout_panic ( #4461 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-28 15:33:58 +00:00
6eed09a926
test: use "real" ingester in query tests ( #4455 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-28 14:39:31 +00:00
3198d75f2f
Merge pull request #4447 from influxdata/cn/aio
...
feat: Make all-in-one mode easier to use
2022-04-28 14:00:55 +00:00
5c89e7b952
fix: Remove max catalog connection config from all-in-one mode
2022-04-28 09:29:03 -04:00
ff268ace6d
fix: Add more docs on write buffer temp dir behavior
2022-04-28 09:29:02 -04:00
c93c26c503
fix: Add reference to TODO ticket
2022-04-28 09:29:02 -04:00
4597c02daa
fix: Add instructions on specifying host when loading data
2022-04-28 09:29:02 -04:00
a024e07147
fix: Small corrections to some comments
2022-04-28 09:29:02 -04:00
df1afa3481
feat: Log what kind of server in startup/shutdown logs
2022-04-28 09:29:02 -04:00
84683056db
feat: Add more logging for all-in-one mode to make it clearer what components are used
2022-04-28 09:29:02 -04:00
d6d50f83c2
feat: Set different catalog config defaults for all-in-one mode
...
Connects to #4399 .
If `--catalog-dsn` is specified, use that Postgres catalog. If
`--catalog-dsn` is not specified, use an in-memory catalog.
2022-04-28 09:29:02 -04:00
941dd12dd1
feat: Set different write buffer config defaults for all-in-one mode
...
Connects to #4399 .
Only file-based write buffer is supported. If `--data-dir` is specified,
store it there, otherwise store it in a temp directory to be ephemeral
2022-04-28 09:29:02 -04:00
0cfd16263c
refactor: Change run_config to logging_config
...
The only spot this method is used actually wants the logging config
2022-04-28 09:29:01 -04:00
06342f9ed8
feat: Set different ingester defaults for all-in-one mode
...
Connects to #4399 .
Manually flatten the arguments to set different defaults, not allow
setting the partition min/max, but still allow customization of the
other arguments.
2022-04-28 09:29:01 -04:00
f8166f7cf4
feat: Update README instructions for NG
...
Connects to #4399 .
2022-04-28 09:29:01 -04:00
6ec0b401e9
fix: Support for computing series across multiple record batches ( #4444 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-28 13:12:27 +00:00
e13d3433ae
feat: Use datafusion serialization code rather than our own copy of it ( #4421 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-28 13:03:34 +00:00
e41785ac3b
Merge pull request #4458 from influxdata/dom/e2e-metrics
...
test: remove hard-coded metric count
2022-04-28 12:48:24 +00:00
246af0c3ca
test: remove hard-coded metric count
...
Prior to this commit, adding any metric to the catalog (and only the
catalog) would cause the end_to_end_ng_cases::metrics::test_metrics test
to fail due to asserting an exact number of metrics observed.
This commit changes the check condition to a more permisive >= rather
than ==.
2022-04-28 11:28:48 +01:00
420c306caa
chore(deps): Bump tokio from 1.17.0 to 1.18.0 ( #4453 )
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.17.0...tokio-1.18.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 08:21:17 +00:00
b0ba0361ed
Merge branch 'main' into dom/schema-cache-warm
2022-04-27 21:24:57 +00:00
115f007317
refactor: Use DataFusion `Expr` instead of our own custom wrapper for `ValueExpr` ( #4440 )
...
* refactor: Use DataFusion `Expr` instead of custom wrapper for BinaryExprs
* fix: apply code review suggestions
* fix: more code review suggestions
2022-04-27 19:20:15 +00:00
59f6556483
fix: do not create empty batches in ingester ( #4443 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-27 17:52:22 +00:00
a195973cfb
chore(deps): Bump clap from 3.1.11 to 3.1.12 ( #4406 )
...
* chore(deps): Bump clap from 3.1.11 to 3.1.12
Bumps [clap](https://github.com/clap-rs/clap ) from 3.1.11 to 3.1.12.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.1.11...v3.1.12 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* fix: Update tests now that the clap crossed-streams bug has been fixed
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-27 17:43:55 +00:00
d7d4de5144
feat(router2): pre-warm schema cache
...
Fetch all NamespaceSchema from the catalog and pre-cache them during
router initialisation.
This pre-warming happens in 3 queries, and should significantly reduce
the load on the Postgres instance backing the catalog during
deployments/after router crashes/etc.
Pre-warming occurs during init, before the HTTP endpoints are bound, and
therefore before the /health HTTP endpoint can return OK. This ensures
the router pre-warms the cache before it begins receiving traffic.
2022-04-27 17:28:07 +01:00
bb8a19b571
feat(iox_catalog): list_schemas()
...
Adds a function to resolve an atomic snapshot of all NamespaceSchema in
the catalog with minimal query overhead.
2022-04-27 17:23:28 +01:00
Carol (Nichols || Goulding)
kodiakhq[bot]
Andrew Lamb
Nga Tran
YIXIAO SHI
Marco Neumann
dependabot[bot]
Paul Dix
Dom Dwyer