* Strategies to improve query performance: operations, number and size of parquet files
Fixes#5108
- Add query performance strategies to optimize-queries
* Apply suggestions from code review
* Apply suggestions from code review
* Update content/influxdb/cloud-dedicated/query-data/execute-queries/optimize-queries.md
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* chore(v3): WIP query perf
Fixes#5108
* chore(v3): WIP: Query performance
* chore(v3): WIP: Explain the EXPLAIN report and indicators of query expense, performance problems.
* WIP: optimize queries - how to read a query plan, operators
* WIP: Read a query plan example 2
* WIP: moved how to read a query plan to its own page.
* WIP(v3): operators
* chore(v3): WIP add query plan info from DataFusion slides and @NGA-TRAN
* chore(v3): WIP read a query plan - explain tree format and reorganize
* WIP: query plan
- Adds Query Plan reference
- Completes Analyze a Query Plan, pending cleanup, continue at :471
- Added image from Nga's blog
- Updates EXPLAIN doc
- TODO: Create public docs for
https://github.com/influxdata/docs.influxdata.io/blob/main/content/operations/specifications/iox_runbooks/slow-queries.md
* chore(spelling): Vale config changes
- Add vale to package.json and use Yarn to manage the binary. You can use `npx vale` to run manually.
- Move InfluxData spelling ignore list into the style.
- Reorganize custom (product) spelling lists to comply with Vale 3.x
- Add InfluxDB v3 terms
* chore(spelling): Vale config changes
- Add vale to package.json and use Yarn to manage the binary. You can use `npx vale` to run manually.
- Move InfluxData spelling ignore list into the style.
- Reorganize custom (product) spelling lists to comply with Vale 3.x
- Add InfluxDB v3 terms
* chore(v3): Reorg of query troubleshooting and optimizing docs
- Adds query-data/troubleshoot-and-optimize
- Splits optimize docs into troubleshoot and optimize docs
- Moves Flight response doc to flight-responses.md
* chore: Fixes broken links, typos, missing content, etc.
- Fixes various errors and style violations reported by Vale.
- Fixes broken links and missing content in glossaries.
- Fixes missing and extraneous whitespace.
* Apply suggestions from code review
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/analyze-query-plan.md
* chore(clustered): Query plan: Apply review suggestions
Co-Authored-By: Nga Tran <20850014+NGA-TRAN@users.noreply.github.com>
* feature(v3): Analyze a query plan:
- Apply code formatting to plan implementor names
- Simplify some points
- Add links
* add query plan html diagram (#5365)
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/analyze-query-plan.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/optimize-queries.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/trace.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/trace.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/trace.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-serverless/query-data/troubleshoot-and-optimize/analyze-query-plan.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* fix(v3): finish the EXPLAIN descriptions and examples
* chore(tests): Setup a python venv in test containers
* fix(ci): Vale vocab
* fix(v3): Punctuation typo
* chore(ci): Update README
* fix(v3): Apply review suggestions and capitalization
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* fix(v3): Add note to optimize page and revise troubleshoot
* fix(v3): optimize-queries link
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Nga Tran <20850014+NGA-TRAN@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* chore(ci): Install Telegraf to test container
* chore(v3): test telegraf config and startup:
- creates the config file for the test run. TODO: remove the duplication and use the content for test setup.
- fix placeholder name
* fix(v3): make test pass for curl API example
* chore(ci): make tests pass for write. Add dependencies to Dockerfile.
* chore(tests): update test build
- Compose metadata: Add profile to test service. We can decouple services later and apply the profile to each. Rename. Pass the image name to the container.
- Fetch the parse_yaml.sh script during docker build, so we only fetch when necessary. Track the script in git. This is still a little janky, but better.
- test.sh: Use `HEAD` for diff (duh). Go back to using docker compose up so we're rebuilding only as needed and reusing the container - much faster, but the output isn't as pretty as `run`.
- pytest.ini, requirements.txt: Use pytest-dotenv to load env.products variables.
* fix(write): update JS instructions and pass tests.
* chore(ci): Install influx v2 CLI into the Docker test container.
* chore(ci): move .env file loading to pytest and run product tests separately with the proper .env file:
- Moves .env configs to be loaded by the test runner instead of by Docker. I'd prefer to ignore any product divergence and have them run interchangeably, but the pros outweigh the cons for now.
- Add example for passing custom commands in the compose file.
* fix(v3): replace curl example with v1 and v2, simplify JS example, pass tests.
feat(v3): add v1 write, test, update text:
- adds Serverless v1 /write example with note about autogen behavior
- adds link from v1 examples to Migrate guide
- adds description of autogen behavior to v1-compat guide
- updates "powered by..." text
- adapts examples to make them more testable, asserts `/write` status code
- replace "shared" examples (only used once) with inline to make them testable
fix(v3): promote migrate link
* Update content/influxdb/cloud-dedicated/get-started/write.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* chore(v3): Write: fix wording for curl success (#5156)
---------
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
- Bad Dockerfile merge.
- Add condition to abort if there are no files to test.
- Revert search path to `.`. If we need to change it, we can do it in pytest.ini.
* nest reference docs menu in v2 and cloud
* move migrate data into write data
* nest notebooks under tools
* nest templates under tools
* nest telegraf configs under tools
* cleanup weights and titles of tools sections
* reorg org and bucket management under admin
* add backup/restore to admin, clean up related links
* restructure security sections under admin
* Apply suggestions from code review
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* updated serverless cli docs to address pr feedback
* Update content/influxdb/cloud/admin/organizations/delete-org.md
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* port v2 install changes to new structure
* address pr feedback and ported missing content to serverless
---------
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* chore(ci): Docker install gpg and wget for test image.
* chore(ci): add a config (temporary) for v2 tests.
* chore(ci): tests: replace latest-patch tag with environment variable.
* chore(ci): test runner
- test.sh diffs against origin/master
- test.sh builds the image for each run and then removes the container after the run.
- tesh.sh adds examples for running and debugging to remind myself.
- compose.yaml adds a bind mount to copy docs-v2/data into docs-v2/test.
- Dockerfile: simplify path use
- run-tests.sh: make default run less verbose, add support for a verbose option (and future options).
- run-tests.sh: generates a .env.products file from data/products.yml and sources the file variables as environment variables to use in test runs.
- run-tests.sh: adds some misc. test setup.
- requirements.txt: return to using the influxdb3-python release
* chore(v2): Update and test v2 OSS and CLI install instructions:
- Update download URLs and paths for all distributions.
- Update and fix install instructions, make consistent between browser downloading and shell
- Make testable.
* chore(v2): Ignore parse_yaml.sh download.
- Adds a Docker image, scripts, and configuration that install and run pytest-codeblocks on Python and shell code samples in Markdown.
- Copied from https://github.com/jstirnaman/docs-codeblocks/
- TODO: The script copies files into a tmp directory configured as a bind mount for the Docker container. We might want to replace this with the experimental `watch` feature.
- TODO: Use a secret to configure the token. As is, values subbed from the dotenv file are exposed in your Docker container history.
Jason Stirnaman
Scott Anderson