Done:
- Refactors and improves pytest runs--smaller scripts, leaner Dockerfile and containers, sensitive .env files are kept in a bind mount and not added to any container.
WIP:
- Test accepts a list of files, otherwise tests staged files.
- Reconfigures the product-specific linting to behave more as intended
- Add product-specific styles with Branding configurations
- Merges various spelling rules into InfluxDataDocs vocabulary for simplicity
- Removes redundant vocabs and terms lists
- Reconfigures vale.sh to lint staged and unstaged paths, using product-specific Vale configs, and report all suggestions, warnings, and errors--intended for "cleanup projects" beyond routine pre-commit checks.
- Updates CONTRIBUTING.md doc, explains how to install dependencies and how to ignore pre-commit hooks
- Removes reliance on npm/npx
- Rename folder, line protocol -> line-protocol
- Update clustered doc
- Replace bucket/measurement with database/table
- Revert vale JSON output to line output so that husky appropriately fails the commit.
- Update custom vocab terms
- Add Vale to BasedOnStyles to enable the custom vocabs--they need access to Vale's **Avoid** action.
- Configures lint-staged to run vale on staged files and use a custom config with custom vocab for specific directories.
- .md files in other directories use the project-level .vale.ini
- Adds the write-good package for style checking
* Downsampling InfluxDB2 data with Quix
* Minor updates
* Adjust weight
* WIP - Added downsample with Quix
* Update menu
* Update menu for v2 downsampling with Flux and Quix
* v2 downsampling page ordering
* Add Quix downsample example to clustered
* Add Quix downsample example to cloud
* Update content/influxdb/cloud-dedicated/process-data/downsample/downsample-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud/process-data/common-tasks/downsample/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud/process-data/common-tasks/downsample/downsample-data.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud/process-data/common-tasks/downsample/downsample-data.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud/process-data/common-tasks/downsample/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud/process-data/common-tasks/downsample/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/clustered/process-data/downsample/_index.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Updates based on review comments
* Update Cloud downsampling with Quix page title
* Task/influx quix updates (#3)
Adding alternative downsampling examples - Using Quix.
This PR includes changes to:
Dedicated
Cloud(TSM)
OSS
Clustered
Quix is collaborating with Influxdata to act as a pure Python alternative to Kapacitor+Flux.
The audience is folks who:
Know Python already and/or don't want to learn a DSL, and possibly those who want to process data more "Pythonically" rather than mixing SQL syntax and Python.
FYI: Quix is both a runtime platform (Quix Cloud) and data processor (Quix Streams Python library). The aim of the articles is to show how downsampling can be done in a continuous real-time process using pure Python.
[x] Signed the InfluxData CLA
(if necessary)
[x] Rebased/mergeableAdding alternative downsampling examples - Using Quix.
* Update content/influxdb/cloud-serverless/process-data/downsample/quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-dedicated/process-data/downsample/downsample.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-serverless/process-data/downsample/quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update layouts/shortcodes/html-diagram/influxdb-v3-quix-downsample-pipeline.html
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update layouts/shortcodes/html-diagram/influxdb-v3-quix-downsample-pipeline.html
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update layouts/shortcodes/html-diagram/influxdb-v3-quix-downsample-pipeline.html
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Review updates (#4)
Updates from review. thanks.
* Update content/influxdb/cloud-dedicated/process-data/downsample/downsample-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/v2/tools/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/v2/tools/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/v2/tools/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/v2/tools/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/v2/tools/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/v2/tools/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/clustered/process-data/downsample/quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/clustered/process-data/downsample/quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/clustered/process-data/downsample/client-libraries.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-serverless/process-data/downsample/quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud/process-data/common-tasks/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud/process-data/common-tasks/downsample-data-quix.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Updates from review (#5)
Updates from review
---------
Co-authored-by: steverosam <steve@quix.ai>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* add infomrmation about sql timezone support
* Apply suggestions from code review
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* fixed typos
---------
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* Adding alternative version of downsampling procedure that uses the Quix Python library
* adding Jupyter Notebook link
* Apply suggestions from code review
* Rename downsample_quix.md to downsample-quix.md
---------
Co-authored-by: Merlin <merlin@quix.io>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Fix shortcut
* chore(serverless): partition and file limits for free and PAYG (closes#5375)
* chore(serverless): partition and file limits for free and PAYG (closes#5375)
* added influxql query support for influxctl
* removed sql tag from task-based influxctl query guide
* Influxctl v2.8.0 (#5418)
* Release influxctl v2.8.0
* Update change logs
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
---------
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Add influxctl write timeout and new insecure TLS config (#5419)
* add timeout flag to write, update config with insecure tls
* fixed link
* Add 'enable-system-tables' flag to `influxctl query` (#5415)
* add enable system tables flag to influxctl
* Apply suggestions from code review
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
---------
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* Query system tables with influxctl (#5414)
* outline for querying system tables
* finalized system table query examples
* updated system table validity date
* Apply suggestions from code review
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* Apply suggestions from code review
---------
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
---------
Co-authored-by: Joshua Powers <powersj@fastmail.com>
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* feat(dedicated): Parameterized queries for SQL and InfluxQL using the native Flight RPC API
* feat(v3): Parameterized queries for SQL and InfluxQL
* feat(v3): Parameterized queries for SQL and InfluxQL
* feat(v3): Dedicated: Parameterized queries for SQL and InfluxQL
* feat(v3): Serverless: Parameterized queries for SQL and InfluxQL
* feat(v3): Clustered: Parameterized queries for SQL and InfluxQL
* feat(v3): Cleanup parameterized queries
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-dedicated/reference/influxql/time-and-timezone.md
* Update content/influxdb/cloud-dedicated/reference/influxql/time-and-timezone.md
* Update content/influxdb/cloud-dedicated/reference/influxql/time-and-timezone.md
* Influxctl v2.7.1 (#5395)
* Release influxctl v2.7.1
* Update change logs
* feat(v3): Parameterized queries: Add references to time and data types pages
* feat(v3): InfluxQL parameterized queries data types
* feat(v3): InfluxQL parameterized queries data types
* feat(v3): Note that you can't parameterize identifiers in queries
* feat(v3): Parameterized queries restrictions
---------
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Joshua Powers <powersj@fastmail.com>
* update v2 and cloud (tsm) grafana docs
* fixed bug where clicking tab would not update url
* updated v1 and enterprise grafana docs, added grafana video to v3
* Strategies to improve query performance: operations, number and size of parquet files
Fixes#5108
- Add query performance strategies to optimize-queries
* Apply suggestions from code review
* Apply suggestions from code review
* Update content/influxdb/cloud-dedicated/query-data/execute-queries/optimize-queries.md
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* chore(v3): WIP query perf
Fixes#5108
* chore(v3): WIP: Query performance
* chore(v3): WIP: Explain the EXPLAIN report and indicators of query expense, performance problems.
* WIP: optimize queries - how to read a query plan, operators
* WIP: Read a query plan example 2
* WIP: moved how to read a query plan to its own page.
* WIP(v3): operators
* chore(v3): WIP add query plan info from DataFusion slides and @NGA-TRAN
* chore(v3): WIP read a query plan - explain tree format and reorganize
* WIP: query plan
- Adds Query Plan reference
- Completes Analyze a Query Plan, pending cleanup, continue at :471
- Added image from Nga's blog
- Updates EXPLAIN doc
- TODO: Create public docs for
https://github.com/influxdata/docs.influxdata.io/blob/main/content/operations/specifications/iox_runbooks/slow-queries.md
* chore(spelling): Vale config changes
- Add vale to package.json and use Yarn to manage the binary. You can use `npx vale` to run manually.
- Move InfluxData spelling ignore list into the style.
- Reorganize custom (product) spelling lists to comply with Vale 3.x
- Add InfluxDB v3 terms
* chore(spelling): Vale config changes
- Add vale to package.json and use Yarn to manage the binary. You can use `npx vale` to run manually.
- Move InfluxData spelling ignore list into the style.
- Reorganize custom (product) spelling lists to comply with Vale 3.x
- Add InfluxDB v3 terms
* chore(v3): Reorg of query troubleshooting and optimizing docs
- Adds query-data/troubleshoot-and-optimize
- Splits optimize docs into troubleshoot and optimize docs
- Moves Flight response doc to flight-responses.md
* chore: Fixes broken links, typos, missing content, etc.
- Fixes various errors and style violations reported by Vale.
- Fixes broken links and missing content in glossaries.
- Fixes missing and extraneous whitespace.
* Apply suggestions from code review
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/analyze-query-plan.md
* chore(clustered): Query plan: Apply review suggestions
Co-Authored-By: Nga Tran <20850014+NGA-TRAN@users.noreply.github.com>
* feature(v3): Analyze a query plan:
- Apply code formatting to plan implementor names
- Simplify some points
- Add links
* add query plan html diagram (#5365)
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/analyze-query-plan.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/optimize-queries.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/trace.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/trace.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/trace.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-serverless/query-data/troubleshoot-and-optimize/analyze-query-plan.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* fix(v3): finish the EXPLAIN descriptions and examples
* chore(tests): Setup a python venv in test containers
* fix(ci): Vale vocab
* fix(v3): Punctuation typo
* chore(ci): Update README
* fix(v3): Apply review suggestions and capitalization
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* fix(v3): Add note to optimize page and revise troubleshoot
* fix(v3): optimize-queries link
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: Nga Tran <20850014+NGA-TRAN@users.noreply.github.com>
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* updates sql functions added upstream
* Apply suggestions from code review
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* port changes to serverless and clustered
---------
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* updated sql time and date functions
* update sql casting docs with new TIMESTAMP behavior, closes#5331
* Apply suggestions from code review
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* added format_n argument to to_timestamp* functions
---------
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* add note to show tag values docs about including a FROM clause, closes#5321
* Update content/influxdb/cloud-dedicated/reference/influxql/show.md
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* port changes to serverless and clustered
---------
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* Updated grafana.md to include the new InfluxDB data source SQL setup
* updates to new grafana docs
* Apply suggestions from code review
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
* ported grafana docs to other v3 products
* minor updates to grafana files
---------
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
Co-authored-by: Scott Anderson <scott@influxdata.com>
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
Fixes#5271
- Updates Go samples that use the client library to use the same Arrow package version (v13) used in the client library
- Updates Go FlightSQL examples to use the latest Arrow package (v14).
* chore(v3): preprocess writes (Telegraf, etc)
* feat: optimize writes using Telegraf
* WIP: dedup plugin
* WIP: Dedup
* chore(v3): Writes: Preprocess data using Telegraf: Dedup example
(#5155)
- Add example using processors.dedup in Telegraf.
- The plugin or configuration doesn't currently work as expected. Investigating...
* chore(v3): Write data: pre-process data using Telegraf
- Add Telegraf examples that pre-process data to avoid schema conflicts and reduce payload (bandwith, memory, etc.) when writing data.
- For Serverless. Will port to Dedicated and Clustered after approval.
- Doesn't include Dedup yet - I'm waiting on help to make it work (see branch https://github.com/influxdata/docs-v2/compare/issue5155-preprocess-with-dedup)
* Update content/influxdb/cloud-serverless/write-data/best-practices/optimize-writes.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* chore(test): Test harness setup, use golang base image
- Use golang base image for ease of Go environment setup. Installing Python separately is easier than Go.
- Add flag to bypass python warning about using system python and pip.
- Remove redundant env.products from pytest.ini; we already load it into environment variables.
* chore(v3): Optimize writes: Use Telegraf
Closes Telegraf guide: how to use the Converter processor to coerce data before writing #5155
- Adds use cases to Dedicated and Clustered docs
- Adds example for Execd processor plugin to all
- Clean up and pass tests
- Remove shared code sample
* fix(v3): Optimize writes: Placeholders and indent fixes
* Update content/influxdb/cloud-dedicated/write-data/best-practices/optimize-writes.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/cloud-serverless/write-data/best-practices/optimize-writes.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Update content/influxdb/clustered/write-data/best-practices/optimize-writes.md
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
---------
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* feature(v3): Write: partial writes write good lines and reject bad lines #5160
- Documents the proposed partial writes feature for Serverless and Dedicated.
- Updates Troubleshoot writes and API docs.
* chore(v3): update write status codes (#5160)
* Update content/influxdb/cloud-serverless/write-data/troubleshoot.md
* chore(v3): cleanup partial write descriptions in API ref and troubleshooting guide
* feature(v3): Serverless write API: writes good lines, rejects bad lines #5160
- Documents the proposed partial writes feature for Serverless and Dedicated.
- If some points are written, returns 201 status code.
- Updates Troubleshoot writes and API docs.
* feature(v3): Serverless write API: writes good lines, rejects bad lines #5160
- Documents the proposed partial writes feature for Serverless and Dedicated.
- If some points are written, returns 201 status code.
- Updates Troubleshoot writes and API docs.
* fix: Serverless cleanup
* fix: Serverless write API: restore 204 and explain 201 and 400 status codes.
* Apply suggestions from code review
* fix: Serverless write API: 201 superseded by 204 to maintain v2 compat.
- Documents https://github.com/influxdata/idpe/pull/18398.
* fix: Serverless write API: 201 superseded by 204 to maintain v2 compat.
- Documents https://github.com/influxdata/idpe/pull/18398.
* Update content/influxdb/cloud-serverless/write-data/troubleshoot.md
* Update content/influxdb/cloud-serverless/write-data/troubleshoot.md
* Update content/influxdb/cloud-serverless/write-data/troubleshoot.md
* fix(v2): series keys in OSS
closes#5226
- Fix v2 series keys definitions
- Remove field key from OSS description
- Update glossary and schema-design.
* fix(v3): rejected points reason
closes Fix series definitions and reasons for rejected points #5226
- Replaces series with partition in reasons for rejected points.
* chore(v3): DBRP: Update Serverless API ref (#5234)
* chore(ci): Tests: substitute placeholders in JSON body parameters
* chore(v3): DBRP: copy page from influxql to admin/dbrps.
* chore(ci): Trying to prevent "a Influx..."
* chore(ci): Allow autogen
* chore(serverless): DBRP glossary entry
* chore(v2): add default DBRP note
* chore(v3): use product-name shortcode
* chore(v3): use product-name shortcode, clarify retention period and mapping
* chore(serverless): Create Bucket:
- add missing example for Data Explorer
- move Retention Period syntax and define retentionRules syntax
- setup for tests
* chore(serverless): Create Bucket:
- add missing example for Data Explorer
- move Retention Period syntax and define retentionRules syntax
- setup for tests
* chore(serverless): Update bucket: alias to match other products
* chore(v3): Serverless for prototyping: update with DBRP behavior
* Serverless auto-generated buckets have 3-day default retention period
Fixes#5234
- v1-compat guide:
- passes tests
- explains DBRPs (default, automatic, and management)
- migrate DBRP content from query-data/influxql/dbrp
- v2-compat guide:
- fix TOC
- add list code example
- rename influxdb-v1-api to api/v1-http, migrate params from v1-compat guide, and explain DBRP in context
- add write-data/api/v1-http and migrate content from v1-compat guide
- move optimize and troubleshooting up to /query-data
- add Related items support for paths containing http
- substitute placeholders in tests
* chore(test): update configuration in README
* fix(serverless): Optimize queries parent
* fix(serverless): create-bucket cleanup
* fix: cleanup
* fix: close tag
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* fix(v3): permission to create dbrp-mapped bucket
---------
Co-authored-by: Scott Anderson <sanderson@users.noreply.github.com>
* update mentions of IOx to v3
* Apply suggestions from code review
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
---------
Co-authored-by: Jason Stirnaman <stirnamanj@gmail.com>
Jason Stirnaman
Scott Anderson
Steve
Scott Anderson
Abdullah Sabaa Allil
Merlin
Dennis German
Andreas Deininger
Jay Clifford
Andrew Lamb
Gary Fowler