* feat: add seconds to generated load files
This adds seconds to the time string portion of the generated files from
load generation runs. Previously, if the generator was run more than once
in the same minute, latter runs would fail because the results files
already exist.
* refactor: make query/write/system graphs optional based on run
Made the analysis tool have optional graphs based on what was actually
generated.
* refactor: change the time string format in generated load files
* refactor: make end common to load generatino tool
Made the --end argument common to both the query and write load generation
runners.
A panic message was also added in the table buffer where unwraps were
causing panics
* refactor: load gen print statements for consistency
* refactor: query/write load gen arg interface
Refactored the argument interface for the query and write load gen
commands to make them easier to unify in a new `full` command.
In summary:
- remove the query sampling interval
- make short-form of --querier-count 'q' instead of 'Q'
- remove the short-form for --query-format
- remove --spec-path in favour of --querier-spec and --writer-spec
for specifying spec path of the `query` and `write` loads, resp.
* feat: produce error on 0s sampling interval
* refactor: split out query/write command configs
Refactored the query and write command clap configurations to make
them composable for the full command
* refactor: expose query and write runner for composability
Refactored the query and write runners so that they can be
composed into the full runner.
* feat: add the full load generator sub-command
Implement a new sub-command for the load generator: full
This runs both the query and write loads simultaneously, and exposes
the unified CLI of the two commands, respectively.
* chore: cargo update to fix audit
* feat: report system stats in load generator
Added the mechanism to report system stats during load generation. The
following stats are saved in a CSV file:
- cpu_usage
- disk_written_bytes
- disk_read_bytes
- memory
- virtual_memory
This only works when running the load generator against a local instance
of influxdb3, i.e., one that is running on your machine.
Generating system stats is done by passing the --system-stats flag to the
load generator.
* feat: add new clap args for results gen
Added the results_dir and configuration_name args
to the common load generator config which will be
used in generating the results directory structure.
* feat: load gen results directory structure
Write and query load generation runners will now setup files in a
results directory, using a specific structure. Users of the load tool
can specify a `results_dir` to save these results, or the tool will
pick a `results` folder in the current directory, by default.
Results will be saved in files using the following path convention:
results/<s>/<c>/<write|query|system>_<time>.csv
- <s>: spec name
- <c>: configuration name, specified by user with the `config-name`
arg, or by default, will use the revision SHA of the running server
- <write|query|system>: which kind of results file
- <time>: a timestamp in the form 'YYYY-MM-DD-HH-MM'
The setup code was unified for both write and query commands, in
preparation for the creation of a system stats file, as well as for
the capability to run both query and write at the same time, however,
those remain unimplemented as of this commit.
* feat: /ping API support on influxdb3_client::Client
Implement the query load generator. The design follows that of the existing write load generator.
A QuerySpec is defined that will be used by the query command to generate a set of queriers to perform queries against a running server in parallel.
* feat: initial load generator implementation
This adds a load generator as a new crate. Initially it only generates write load, but the scaffolding is there to add a query load generator to complement the write load tool.
This could have been added as a subcommand to the influxdb3 program, but I thought it best to have it separate for now.
It's fairly light on tests and error handling given its an internal tooling CLI. I've added only something very basic to test the line protocol generation and run the actual write command by hand.
I included pretty detailed instructions and some runnable examples.
* refactor: address PR feedback
Trevor Hilton
Paul Dix