Commit Graph

118 Commits (db7e6335ca2a416cc3120f87ce1441bd4f60b03f)

Author SHA1 Message Date
Carol (Nichols || Goulding) 4f8dd072b3
fix: Translate a test with a predicate of a literal = literal 2023-01-27 10:28:43 -05:00
Carol (Nichols || Goulding) 94f7f015f4
fix: Port a test with a predicate that tag=tag, which is always true 2023-01-27 10:28:43 -05:00
Carol (Nichols || Goulding) 67c430da63
test: Port read_filter query_tests to end-to-end tests 2023-01-27 10:28:43 -05:00
Carol (Nichols || Goulding) 9d490ceb1a
feat: Add a method to create tag expressions ORed together 2023-01-27 10:28:42 -05:00
Carol (Nichols || Goulding) 7b94e545f1
feat: Change combine_predicate to take a logical operator
To enable building "OR" queries
2023-01-27 10:28:42 -05:00
Carol (Nichols || Goulding) b633b8b7a0
feat: Allow building a predicate that ANDs multiple nodes 2023-01-27 10:28:42 -05:00
Carol (Nichols || Goulding) d6bd6d5178
fix: Make regex_predicate function private; it's only used in this impl 2023-01-27 10:28:42 -05:00
Carol (Nichols || Goulding) 31e7925f47
refactor: Extract a function for making a comparison expression node 2023-01-27 10:28:42 -05:00
Carol (Nichols || Goulding) a400e212ec
refactor: Extract a function for making a string value node 2023-01-27 10:28:42 -05:00
Carol (Nichols || Goulding) 39cd34912b
refactor: Extract a function for making a tag ref node 2023-01-27 10:28:42 -05:00
Carol (Nichols || Goulding) c2c8524dd8
refactor: Extract a shared function for tag predicates 2023-01-27 10:28:42 -05:00
Andrew Lamb 5ef9018f7e
refactor: Move sql script files from query_tests and into end to end query tests (#6708)
* refactor: Move sql script files from query_tests and into end to end query tests

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2023-01-26 19:49:21 +00:00
Carol (Nichols || Goulding) 4658510102
fix: For Ingester2, persist a particular namespace on demand and share MiniClusters
This should hopefully help CI from running out of Postgres
connections 😬

The old architecture will still need to be non-shared and persist
everything.
2023-01-25 10:36:56 -05:00
Carol (Nichols || Goulding) af203f7a6d
docs: Explain why the tests set the number of query threads 2023-01-19 16:44:30 -05:00
Carol (Nichols || Goulding) f85c8d916e
fix: Hardcode query threads in old query_tests too 2023-01-19 16:44:30 -05:00
Carol (Nichols || Goulding) dc95438694
feat: Add old ingester and old parquet states into the query_tests2 framework 2023-01-19 16:44:29 -05:00
Andrew Lamb 65c020c9f2
refactor: remove iox_arrow_flight use in `influxdb_iox_client ` and `querier` (#6624)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-19 18:48:23 +00:00
Carol (Nichols || Goulding) d68a35a853
docs: Explain why the tests set the number of query threads
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2023-01-18 13:31:52 -05:00
Carol (Nichols || Goulding) 747bdb452b
fix: Add the standard lints to query_tests2 and fix the new warnings 2023-01-18 10:24:17 -05:00
Carol (Nichols || Goulding) f3b5dcaab7
feat: Reimagining query_tests 2023-01-18 10:24:17 -05:00
Dom f7ff877582
Merge branch 'main' into cn/ingester-persist-tick 2023-01-13 12:31:45 +00:00
Carol (Nichols || Goulding) f56123bf30
test: Allow integration tests that should_panic to pass if TEST_INTEGRATION isn't set 2023-01-12 15:31:34 -05:00
Carol (Nichols || Goulding) b989e0893f
test: Make persist-on-demand test a step test and check the number of parquet files 2023-01-12 11:40:46 -05:00
Carol (Nichols || Goulding) 3a2544a7eb
feat: Define a new gRPC service for ingester persist 2023-01-12 11:03:12 -05:00
Carol (Nichols || Goulding) e9b3efb33d
refactor: Extract a method for making requests to the ingester onto MiniCluster 2023-01-12 11:03:10 -05:00
Carol (Nichols || Goulding) f6dda3f4bb
test: Create shared, non-persisting ingester2 cluster 2023-01-12 10:23:00 -05:00
Carol (Nichols || Goulding) be7c312033
fix: Wait for a particular number of Parquet files, not just any change 2023-01-11 12:11:56 -05:00
Carol (Nichols || Goulding) 7e921e6a23
fix: Make recording num parquet files an explicit test step
To support a case where someone calls WriteLineProtocol twice in
a row to simulate two write requests. The test should be able to
record this state before the two write requests and not twice.
2023-01-11 11:51:56 -05:00
Carol (Nichols || Goulding) 6677ae5c61
test: Record number of Parquet files before a write
Fixes #6506.

Also has the pleasant side effect of making this code simpler and less
hacky-- it now checks the number of Parquet files for the whole
namespace, which is useful in cases where the line protocol writes to
several tables.
2023-01-11 11:51:55 -05:00
Stuart Carnie 66047f4372
feat: InfluxQL learns how to plan some InfluxQL queries (#6520)
* feat: InfluxQL learns how to plan some queries

Also added a means to test the planner and execution

* chore: Update module docs

* chore: Document the planner functions

* chore: Update end_to_end_cases crate

* chore: Clarify why `SLIMIT` and `SOFFSET` return `NotImplemented`

* chore: Address lint issues

* chore: Fix rustdoc link issue

* chore: Remove InfluxQL tests from query_tests crate

Will follow conventions established by @carols10cents when
new query_tests crate is merged.

* chore: `now` field

`now` is a DataFusion built-in scalar function

* chore: remove unused code

* chore: Add additional arithmetic expression tests

* chore: Establish pattern for identifying and tracking InfluxQL issues

* chore: Add tests for case sensitivity issues

* chore: group tests into modules and functions

This avoids mass rewriting of insta snapshots as new
tests are added to each function. When tests are added in the middle,
existing snapshots are renamed (-N+1, -N+2, etc) resulting in
having to review numerous additional snapshots.
2023-01-11 02:50:49 +00:00
dependabot[bot] b49cc2e35e
chore(deps): Bump tokio from 1.24.0 to 1.24.1 (#6545)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.0 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.24.0...tokio-1.24.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-10 09:48:44 +00:00
dependabot[bot] 09746c3aea
chore(deps): Bump assert_cmd from 2.0.7 to 2.0.8
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd) from 2.0.7 to 2.0.8.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases)
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v2.0.7...v2.0.8)

---
updated-dependencies:
- dependency-name: assert_cmd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-10 01:06:02 +00:00
Carol (Nichols || Goulding) c464487bb2
test: Ability to set up a querier2 that doesn't connect to any ingesters 2023-01-04 10:06:57 -05:00
Carol (Nichols || Goulding) 08ceb4ee48
test: Check catalog for new Parquet files to know when data is persisted 2023-01-04 10:06:57 -05:00
Carol (Nichols || Goulding) e49bee0c26
test: Make test ingester2 instances either persist very quickly or not at all 2023-01-04 10:06:57 -05:00
Carol (Nichols || Goulding) afd09d96be
test: Create a Querier2 test server type 2023-01-04 10:06:56 -05:00
Carol (Nichols || Goulding) 96029654ab
test: Add a shared MiniCluster for version 2 services 2023-01-04 10:06:56 -05:00
dependabot[bot] 0aacef3c59
chore(deps): Bump once_cell from 1.16.0 to 1.17.0 (#6473)
* chore(deps): Bump once_cell from 1.16.0 to 1.17.0

Bumps [once_cell](https://github.com/matklad/once_cell) from 1.16.0 to 1.17.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.16.0...v1.17.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Change once_cell version specifier to major.minor for less churn

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
2023-01-02 17:07:15 +00:00
Carol (Nichols || Goulding) 46ff8854ec
fix: Use code backticks around invalid HTML tags in doc strings 2022-12-21 16:36:17 -05:00
Carol (Nichols || Goulding) 72aab99951
fix: Remove needless borrow. Thanks clippy! 2022-12-21 14:32:34 -05:00
Andrew Lamb 9b22ede3f0
refactor: Make arrow flight client return `futures::Streams` (#6438)
* refactor: Make arrow flight client use futures::Streams

* refactor: concision
2022-12-19 17:09:26 +00:00
Andrew Lamb 94c2f94ea1
refactor: Extract common ArrowFlight client into iox_arrow_flight (#6427)
* refactor: Extract common ArrowFlight client into iox_arrow_flight

* chore: Run cargo hakari tasks

* fix: clarify intent of iox_arrow_flight crate

* refactor: Apply suggestions from code review

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>

* fix: loop --> while let

* fix: REmove make_tonic_error in favor of From impl

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-19 11:35:20 +00:00
Carol (Nichols || Goulding) 2a1e540ee3
fix: Rename INFLUXDB_IOX_MODE to INFLUXDB_IOX_RPC_MODE 2022-12-15 14:13:01 -05:00
Carol (Nichols || Goulding) aec98015d7
fix: Remove the rpc_write feature flag and use INFLUXDB_IOX_MODE env var instead
And standardize on ingester2 and router2 for consistency.

Connects to #6402.
2022-12-15 14:06:59 -05:00
Andrew Lamb 47cd6821e1
feat: Document IOx Flight API and add convenience methods (#6392)
* feat: Document IOx Flight API and add convenience methods

* fix: InfluxQL handling

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-13 17:32:37 +00:00
Carol (Nichols || Goulding) c3a7575d46
feat: Enable rpc_write on the inner command if it's enabled for tests
And only run rpc_write specific tests if the feature is enabled when
running the tests.
2022-12-08 17:45:30 -05:00
Carol (Nichols || Goulding) 6014c10866
test: Enable running ingester2/router RPC write servers in e2e tests
Add configuration and server types to be able to create server fixtures
for them.
2022-12-08 17:22:52 -05:00
Carol (Nichols || Goulding) 62db312a8f
feat: Switch to escargot to get more control over running Cargo bins 2022-12-08 15:29:44 -05:00
Marco Neumann 4ded68de62
test: "not found" end2end tests for querier (#6352)
I couldn't find any end2end tests for these cases and I was kinda
worried that our error codes were wrong. Turns out they are correct, but
let's have some nice tests for this behavior.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-12-08 18:17:53 +00:00
Marco Neumann cd6a8a1a82
refactor: DF-driven on-demand mem limit instead of ahead-of-time heuristics (#6313)
* refactor: DF-driven on-demand mem limit instead of ahead-of-time heuristics

Closes #6310.

* refactor: rename and tune default exec mem limits

* fix: ingester2 bits after rebase
2022-12-05 12:38:28 +00:00