Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
influxdb/query_tests
History
Marco Neumann 365a246f8d
refactor: do not run de-dup in ingester for querier requests (#5626) 
* refactor: do not run de-dup in ingester for querier requests

This removes the entire de-dup logic from the inegster for querier
requests. Furthermore, it even removes the entire datafusion execution
from the querier and just dumps the in-memory record batches as quickly
as possible. No filters are applied. Note that even prior to this PR,
we've never applied projections (tracked by #5624).

**Pros:**

- speed up query planning within the querier (since we need the ingester
  response for state reconciling)
- lowered ingester CPU load

**Cons:**

- more querier<>ingester network traffic

Closes #5602.

* test: extend query test case

* fix: ingester tests

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 		2022-09-22 07:33:54 +00:00
..
cases refactor: do not run de-dup in ingester for querier requests (#5626) 2022-09-22 07:33:54 +00:00
generate fix: ensure querier cache is refreshed for partition sort key (#4660) 2022-05-25 10:44:42 +00:00
src refactor: do not run de-dup in ingester for querier requests (#5626) 2022-09-22 07:33:54 +00:00
Cargo.toml chore(deps): Bump once_cell from 1.14.0 to 1.15.0 (#5701) 2022-09-21 05:20:55 +00:00
README.md fix: Remove db crate from query_tests 2022-05-06 11:30:36 -04:00

README.md

This crate contains "integration" tests for the query engine. Specifically, it runs queries against a fully created database instance, records the output, and compares it to expected output.

Some tests simply have their inputs and outputs hard coded into #[test] annotated tests as is Rust's norm.

The tests in src/runner are driven somewhat more dynamically based on input files.

Cookbook: Adding a new Test

How to make a new test:

  1. Add a new file .sql to the cases/in directory
  2. Regenerate file: (cd generate && cargo run)
  3. Run the tests `` cargo test -p query_tests`
  4. You will get a failure message that contains examples of how to update the files

Example output

Possibly helpful commands:
  # See diff
  diff -du "/Users/alamb/Software/influxdb_iox/query_tests/cases/in/pushdown.expected" "/Users/alamb/Software/influxdb_iox/query_tests/cases/out/pushdown.out"
  # Update expected
  cp -f "/Users/alamb/Software/influxdb_iox/query_tests/cases/in/pushdown.out" "/Users/alamb/Software/influxdb_iox/query_tests/cases/out/pushdown.expected"

Cookbook: Adding a new test scenario

Each test can be defined in terms of a "setup" (a set of actions taken to prepare the state of database).

In the future, we envision more fine grained control of these setups (by implementing some of the database commands as IOX_TEST commands), but for now they are hard coded.

The SQL files refer to the setups with a specially formatted comment:

-- IOX_SETUP: OneMeasurementFourChunksWithDuplicates

To add a new setup, follow the pattern in scenario.rs of get_all_setups.