Commit Graph

20 Commits (f76cbb31ac0ac22f124f30cd82a1a0805acc5c1b)

Author SHA1 Message Date
Marco Neumann 351b0d0c15
fix: unknown namespace/table in querier<>ingester flight protocol (#4307)
* fix: return "not found" gRPC error instead of "internal" when ingester does not know table

* fix: properly handle "namespace not found" in ingester queries

* fix: make `initialize_db` work with async code

* test: add custom step for NG tests

* fix: handle "unknown table/namespace" resp. in querier

* docs: explain test setup

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-04-14 12:36:15 +00:00
Andrew Lamb 85f3e696e8
refactor: Use declarative steps to reduce duplication in end to end testing (#4301)
* refactor: Use declarative steps to reduce duplication in end to end testing,

* fix: improve whitespace formatting

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-13 16:24:57 +00:00
kodiakhq[bot] 410b6d6db4
Merge branch 'main' into crepererum/ng_end2end_tests_schema_names 2022-04-13 15:54:28 +00:00
Marco Neumann ab31f78172 test: improve server restarts
- Ensure that the restarted server uses the same log file as the old
  one. This is important for debugging because we print out server logs
  for failed tests.
- Try to be polite when shutting down or restarting servers (i.e. try to
  use SIGTERM).
2022-04-13 17:33:09 +02:00
Marco Neumann 1b2ed9af9d test: allow test framework to restart ingesters 2022-04-13 17:33:09 +02:00
Marco Neumann ef1f252c6c test: use random catalog schema name for NG end2end tests
Otherwise tests leak state to other tests or previous runs.
2022-04-13 17:24:42 +02:00
Andrew Lamb bd4566cbe0
fix: Do not specify a querier address by default (#4289) 2022-04-12 20:52:18 +00:00
Marco Neumann 380cd9bbff
refactor: use a single flight client implementation (#4273)
"end-user -> querier" and "querier -> ingester" should use a single
Flight client implementation. The difference is just the request and
response metadata.

This changes our default Flight client to use protobuf instead of JSON
for the ticket format.
2022-04-12 09:08:25 +00:00
Andrew Lamb 3f5eab7648
feat: allow the querier to talk with multiple ingesters (#4271)
* refactor: Move querier config to clap_blocks

* refactor: Add tests

* refactor: allow multiple addresses

* refactor: Update to use multiple addresses

* fix: bow to clippy

* fix: docstring

* fix: error if address is repeated multiple times

* chore: Add error enum, plumb through

* fix: clippy

* refactor: improve Rust API

* fix: fix test
2022-04-11 18:49:49 +00:00
Andrew Lamb f6e6821276
feat: Add basic Querier <--> Ingester "Service Configuration" (#4259)
* feat: Add basic Querier <--> Ingester "Service Configuration"

* docs: update comments in test

* refactor: cleanup tests a little

* refactor: make trait more consistent

* docs: improve comments in IngesterPartition
2022-04-11 11:50:22 +00:00
Andrew Lamb a30a85e62c
feat: Add get_write_info service (#4227)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-07 19:24:58 +00:00
Andrew Lamb 833c10c083
feat: return write_token from HTTP writes to router2 (#4202)
* feat: return write_token from HTTP writes to router2

* fix: Update router2/src/dml_handlers/instrumentation.rs

Co-authored-by: Dom <dom@itsallbroken.com>

* refactor: Use WriteSummary::default more vigorously

* fix: fix typo and add links to follow on issues

Co-authored-by: Dom <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-02 10:34:51 +00:00
Andrew Lamb 92da65a065
feat: Add end to end tests for querier and schema client (#4178)
* refactor: split up ingester schema test, add mini cluster

* feat: add schema cli test

* feat: add end to end test for querier

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-30 17:07:32 +00:00
Andrew Lamb cabaf7c0e2
refactor: move object store configuration to TestConfig (#4128)
* refactor: move object store configuration to TestConfig

* fix: clippy

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-25 18:19:44 +00:00
Andrew Lamb 196ad50f5a
test: add e2e test for schema CLI (#4132)
* test: add e2e test for schema CLI

* fix: logical confict
2022-03-25 18:11:03 +00:00
Andrew Lamb e222acbb48
refactor: use typed `TestConfig` rather than environment variable names for NG end to end tests (#4126)
* refactor: move environment variable mapping in end to end tests into TestConfig

* fix: clippy

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-24 18:36:47 +00:00
Andrew Lamb 39d9f30f12
refactor: Make `server` an implementation detail (#4122) 2022-03-24 15:58:04 +00:00
Andrew Lamb 5c69a3f43b
chore: Update deps: datafusion, arrow/arrow-flight/parquet to 11, zstd to 0.11 (#4119)
* chore: update datafusion

* chore(deps): Bump arrow from 10.0.0 to 11.0.0

Bumps [arrow](https://github.com/apache/arrow-rs) from 10.0.0 to 11.0.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/10.0.0...11.0.0)

---
updated-dependencies:
- dependency-name: arrow
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): Bump arrow-flight from 10.0.0 to 11.0.0

Bumps [arrow-flight](https://github.com/apache/arrow-rs) from 10.0.0 to 11.0.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/10.0.0...11.0.0)

---
updated-dependencies:
- dependency-name: arrow-flight
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: update parquet to 11.0.0

* fix: error on create schema, test for same

* fix: upgrade zstd

* chore: Run cargo hakari tasks

* fix: fix logical merge conflict

* fix: hakari

* fix: hakari

* fix: update newly introduced dep

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-24 15:27:36 +00:00
Andrew Lamb 1ca9d28fee
fix: remove unecessary test setup (#4127)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-24 15:17:17 +00:00
Andrew Lamb fb75ef7b82
test: add end to end test for all in one mode, restructure fixture (#4114)
* test: add end to end test for all in one mode, restructure fixture

* docs: fix typos and clarify schema requrements
2022-03-24 12:53:25 +00:00