influxdb/query_tests/README.md

46 lines
1.7 KiB
Markdown
Raw Normal View History

This crate contains "integration" tests for the query
engine. Specifically, it runs queries against a fully created `Db`
instance, records the output, and compares it to expected output.
This crate does not (yet) run queries against a fully functioning
server (aka run of `influxdb_iox` binary) but that may be added at
some future time.
Some tests simply have their inputs and outputs hard coded into
`#[test]` annotated tests as is rust's norm
The tests in `src/runner` are driven somewhat more dynamically based on input files
# Cookbook: Adding a new Test
How do you make a new test:
test: ensure that query tests don't rebuild all the time Beforehand: ```text ❯ env CARGO_LOG=cargo::core::compiler::fingerprint=info cargo test -p query_tests [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] stale: changed "/home/mneumann/src/influxdb_iox/query_tests/cases" [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] (vs) "/home/mneumann/src/influxdb_iox/target/debug/build/query_tests-0e8f741dfb84437f/output" [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] FileTime { seconds: 1625474716, nanos: 436081357 } != FileTime { seconds: 1625474752, nanos: 52625167 } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] fingerprint error for query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests)/Test/TargetInner { ..: lib_target("query_tests", ["lib"], "/home/mneumann/src/influxdb_iox/query_tests/src/lib.rs", Edition2018) } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] err: current filesystem status shows we're outdated [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] fingerprint error for query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests)/RunCustomBuild/TargetInner { ..: custom_build_target("build-script-build", "/home/mneumann/src/influxdb_iox/query_tests/build.rs", Edition2018) } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] err: current filesystem status shows we're outdated [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] fingerprint error for query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests)/Build/TargetInner { ..: lib_target("query_tests", ["lib"], "/home/mneumann/src/influxdb_iox/query_tests/src/lib.rs", Edition2018) } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] err: current filesystem status shows we're outdated Compiling query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests) ``` The issue is that both the input and the test output files are located under `cases/`. `build.rs` used `cargo:rerun-if-changed=cases` which per Cargo doc will scan ALL files in that directory. Note that the normal `exclude` directive in `Cargo.toml` does NOT work, see https://github.com/rust-lang/cargo/issues/4587 . So we need to split input and output files into separate directories (`cases/{in,out}`).
2021-07-05 09:48:50 +00:00
1. Add a new file .sql to the `cases/in` directory
2. Regenerate file: `(cd generate && cargo run)`
2. Run the tests `` cargo test -p query_tests`
3. You will get a failure message that contains examples of how to update the files
## Example output
```
Possibly helpful commands:
# See diff
test: ensure that query tests don't rebuild all the time Beforehand: ```text ❯ env CARGO_LOG=cargo::core::compiler::fingerprint=info cargo test -p query_tests [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] stale: changed "/home/mneumann/src/influxdb_iox/query_tests/cases" [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] (vs) "/home/mneumann/src/influxdb_iox/target/debug/build/query_tests-0e8f741dfb84437f/output" [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] FileTime { seconds: 1625474716, nanos: 436081357 } != FileTime { seconds: 1625474752, nanos: 52625167 } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] fingerprint error for query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests)/Test/TargetInner { ..: lib_target("query_tests", ["lib"], "/home/mneumann/src/influxdb_iox/query_tests/src/lib.rs", Edition2018) } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] err: current filesystem status shows we're outdated [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] fingerprint error for query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests)/RunCustomBuild/TargetInner { ..: custom_build_target("build-script-build", "/home/mneumann/src/influxdb_iox/query_tests/build.rs", Edition2018) } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] err: current filesystem status shows we're outdated [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] fingerprint error for query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests)/Build/TargetInner { ..: lib_target("query_tests", ["lib"], "/home/mneumann/src/influxdb_iox/query_tests/src/lib.rs", Edition2018) } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] err: current filesystem status shows we're outdated Compiling query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests) ``` The issue is that both the input and the test output files are located under `cases/`. `build.rs` used `cargo:rerun-if-changed=cases` which per Cargo doc will scan ALL files in that directory. Note that the normal `exclude` directive in `Cargo.toml` does NOT work, see https://github.com/rust-lang/cargo/issues/4587 . So we need to split input and output files into separate directories (`cases/{in,out}`).
2021-07-05 09:48:50 +00:00
diff -du "/Users/alamb/Software/influxdb_iox/query_tests/cases/in/pushdown.expected" "/Users/alamb/Software/influxdb_iox/query_tests/cases/out/pushdown.out"
# Update expected
test: ensure that query tests don't rebuild all the time Beforehand: ```text ❯ env CARGO_LOG=cargo::core::compiler::fingerprint=info cargo test -p query_tests [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] stale: changed "/home/mneumann/src/influxdb_iox/query_tests/cases" [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] (vs) "/home/mneumann/src/influxdb_iox/target/debug/build/query_tests-0e8f741dfb84437f/output" [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] FileTime { seconds: 1625474716, nanos: 436081357 } != FileTime { seconds: 1625474752, nanos: 52625167 } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] fingerprint error for query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests)/Test/TargetInner { ..: lib_target("query_tests", ["lib"], "/home/mneumann/src/influxdb_iox/query_tests/src/lib.rs", Edition2018) } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] err: current filesystem status shows we're outdated [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] fingerprint error for query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests)/RunCustomBuild/TargetInner { ..: custom_build_target("build-script-build", "/home/mneumann/src/influxdb_iox/query_tests/build.rs", Edition2018) } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] err: current filesystem status shows we're outdated [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] fingerprint error for query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests)/Build/TargetInner { ..: lib_target("query_tests", ["lib"], "/home/mneumann/src/influxdb_iox/query_tests/src/lib.rs", Edition2018) } [2021-07-05T08:52:13Z INFO cargo::core::compiler::fingerprint] err: current filesystem status shows we're outdated Compiling query_tests v0.1.0 (/home/mneumann/src/influxdb_iox/query_tests) ``` The issue is that both the input and the test output files are located under `cases/`. `build.rs` used `cargo:rerun-if-changed=cases` which per Cargo doc will scan ALL files in that directory. Note that the normal `exclude` directive in `Cargo.toml` does NOT work, see https://github.com/rust-lang/cargo/issues/4587 . So we need to split input and output files into separate directories (`cases/{in,out}`).
2021-07-05 09:48:50 +00:00
cp -f "/Users/alamb/Software/influxdb_iox/query_tests/cases/in/pushdown.out" "/Users/alamb/Software/influxdb_iox/query_tests/cases/out/pushdown.expected"
```
# Cookbook: Adding a new test scenario
Each test can be defined in terms of a "setup" (a set of actions taken to prepare the state of database)
In the future we envision more fine grained control of these setups (by implementing some of the database commands as IOX_TEST commands) but for now they are hard coded.
The SQL files refer to the setups with a specially formatted comment:
```sql
-- IOX_SETUP: OneMeasurementFourChunksWithDuplicates
```
To add a new setup, follow the pattern in scenario.rs of `get_all_setups`;