From 5da04427275ea899aad81882f9a0b668cfe4d381 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Tue, 27 May 2025 10:33:12 -0500 Subject: [PATCH 1/3] chore(qol): Build custom CoPilot instructions from CONTRIBUTING.md: - Add a script and pre-commit command to generate a CoPilot instructions file from CONTRIBUTING.md guidelines. - Add custom instructions for CoPilot to help with influxdb3 placeholders --- .../instructions/contributing.instructions.md | 1705 +++++++++++++++++ ...nfluxdb3-code-placeholders.instructions.md | 89 + CONTRIBUTING.md | 2 +- build-scripts/build-copilot-instructions.js | 76 + compose.yaml | 3 + lefthook.yml | 5 +- package.json | 1 + 7 files changed, 1879 insertions(+), 2 deletions(-) create mode 100644 .github/instructions/contributing.instructions.md create mode 100644 .github/instructions/influxdb3-code-placeholders.instructions.md create mode 100644 build-scripts/build-copilot-instructions.js diff --git a/.github/instructions/contributing.instructions.md b/.github/instructions/contributing.instructions.md new file mode 100644 index 000000000..4fb3b1efe --- /dev/null +++ b/.github/instructions/contributing.instructions.md @@ -0,0 +1,1705 @@ +--- +applyTo: "content/**/*.md, layouts/**/*.html" +--- + +# GitHub Copilot Instructions for InfluxData Documentation + +## Purpose and scope + +GitHub Copilot should help document InfluxData products +by creating clear, accurate technical content with proper +code examples, frontmatter, shortcodes, and formatting. + +# Contributing to InfluxData Documentation + +### Sign the InfluxData CLA + +The InfluxData Contributor License Agreement (CLA) is part of the legal framework +for the open source ecosystem that protects both you and InfluxData. +To make substantial contributions to InfluxData documentation, first sign the InfluxData CLA. +What constitutes a "substantial" change is at the discretion of InfluxData documentation maintainers. + +[Sign the InfluxData CLA](https://www.influxdata.com/legal/cla/) + +_**Note:** Typo and broken link fixes are greatly appreciated and do not require signing the CLA._ + +_If you're new to contributing or you're looking for an easy update, see [`docs-v2` good-first-issues](https://github.com/influxdata/docs-v2/issues?q=is%3Aissue+is%3Aopen+label%3Agood-first-issue)._ + +## Make suggested updates + +### Fork and clone InfluxData Documentation Repository + +[Fork this repository](https://help.github.com/articles/fork-a-repo/) and +[clone it](https://help.github.com/articles/cloning-a-repository/) to your local machine. + +## Install project dependencies + +docs-v2 automatically runs format (Markdown, JS, and CSS) linting and code block tests for staged files that you try to commit. + +For the linting and tests to run, you need to install Docker and Node.js +dependencies. + +\_**Note:** +The git pre-commit and pre-push hooks are configured to run linting and tests automatically +when you commit or push changes. +We strongly recommend letting them run, but you can skip them +(and avoid installing related dependencies) +by including the `--no-verify` flag with your commit--for example, enter the following command in your terminal: + +```sh +git commit -m "" --no-verify +``` + +### Install Node.js dependencies + +To install dependencies listed in package.json: + +1. Install [Node.js](https://nodejs.org/en) for your system. +2. Install [Yarn](https://yarnpkg.com/getting-started/install) for your system. +3. Run `yarn` to install dependencies (including Hugo). +4. Install the Yarn package manager and run `yarn` to install project dependencies. + +`package.json` contains dependencies used in `/assets/js` JavaScript code and +dev dependencies used in pre-commit hooks for linting, syntax-checking, and testing. + +Dev dependencies include: + +- [Lefthook](https://github.com/evilmartians/lefthook): configures and +manages git pre-commit and pre-push hooks for linting and testing Markdown content. +- [prettier](https://prettier.io/docs/en/): formats code, including Markdown, according to style rules for consistency +- [Cypress]: e2e testing for UI elements and URLs in content + +### Install Docker + +docs-v2 includes Docker configurations (`compose.yaml` and Dockerfiles) for running the Vale style linter and tests for code blocks (Shell, Bash, and Python) in Markdown files. + +Install [Docker](https://docs.docker.com/get-docker/) for your system. + +#### Build the test dependency image + +After you have installed Docker, run the following command to build the test +dependency image, `influxdata:docs-pytest`. +The tests defined in `compose.yaml` use the dependencies and execution +environment from this image. + +```bash +docker build -t influxdata/docs-pytest:latest -f Dockerfile.pytest . +``` + +### Run the documentation locally (optional) + +To run the documentation locally, follow the instructions provided in the README. + +### Install Visual Studio Code extensions + +If you use Microsoft Visual Studio (VS) Code, you can install extensions +to help you navigate, check, and edit files. + +docs-v2 contains a `./.vscode/settings.json` that configures the following extensions: + +- Comment Anchors: recognizes tags (for example, `//SOURCE`) and makes links and filepaths clickable in comments. +- Vale: shows linter errors and suggestions in the editor. +- YAML Schemas: validates frontmatter attributes. + +### Make your changes + +Make your suggested changes being sure to follow the [style and formatting guidelines](#style--formatting) outline below. + +## Lint and test your changes + +`package.json` contains scripts for running tests and linting. + +### Automatic pre-commit checks + +docs-v2 uses Lefthook to manage Git hooks that run during pre-commit and pre-push. The hooks run the scripts defined in `package.json` to lint Markdown and test code blocks. +When you try to commit changes (`git commit`), Git runs +the commands configured in `lefthook.yml` which pass your **staged** files to Vale, +Prettier, Cypress (for UI tests and link-checking), and Pytest (for testing Python and shell code in code blocks). + +### Skip pre-commit hooks + +**We strongly recommend running linting and tests**, but you can skip them +(and avoid installing dependencies) +by including the `LEFTHOOK=0` environment variable or the `--no-verify` flag with +your commit--for example: + +```sh +git commit -m "" --no-verify +``` + +```sh +LEFTHOOK=0 git commit +``` + +### Set up test scripts and credentials + +Tests for code blocks require your InfluxDB credentials and other typical +InfluxDB configuration. + +To set up your docs-v2 instance to run tests locally, do the following: + +1. **Set executable permissions on test scripts** in `./test/src`: + + ```sh + chmod +x ./test/src/*.sh + ``` + +2. **Create credentials for tests**: + + - Create databases, buckets, and tokens for the product(s) you're testing. + - If you don't have access to a Clustered instance, you can use your +Cloud Dedicated instance for testing in most cases. To avoid conflicts when + running tests, create separate Cloud Dedicated and Clustered databases. + +1. **Create .env.test**: Copy the `./test/env.test.example` file into each + product directory to test and rename the file as `.env.test`--for example: + + ```sh + ./content/influxdb/cloud-dedicated/.env.test + ``` + +2. Inside each product's `.env.test` file, assign your InfluxDB credentials to + environment variables: + + - Include the usual `INFLUX_` environment variables + - In + `cloud-dedicated/.env.test` and `clustered/.env.test` files, also define the + following variables: + + - `ACCOUNT_ID`, `CLUSTER_ID`: You can find these values in your `influxctl` + `config.toml` configuration file. + - `MANAGEMENT_TOKEN`: Use the `influxctl management create` command to generate + a long-lived management token to authenticate Management API requests + + See the substitution + patterns in `./test/src/prepare-content.sh` for the full list of variables you may need to define in your `.env.test` files. + +3. For influxctl commands to run in tests, move or copy your `config.toml` file + to the `./test` directory. + +> [!Warning] +> +> - The database you configure in `.env.test` and any written data may +be deleted during test runs. +> - Don't add your `.env.test` files to Git. To prevent accidentally adding credentials to the docs-v2 repo, +Git is configured to ignore `.env*` files. Consider backing them up on your local machine in case of accidental deletion. + +#### Test shell and python code blocks + +[pytest-codeblocks](https://github.com/nschloe/pytest-codeblocks/tree/main) extracts code from python and shell Markdown code blocks and executes assertions for the code. +If you don't assert a value (using a Python `assert` statement), `--codeblocks` considers a non-zero exit code to be a failure. + +**Note**: `pytest --codeblocks` uses Python's `subprocess.run()` to execute shell code. + +You can use this to test CLI and interpreter commands, regardless of programming +language, as long as they return standard exit codes. + +To make the documented output of a code block testable, precede it with the +`` tag and **omit the code block language +descriptor**--for example, in your Markdown file: + +##### Example markdown + +```python +print("Hello, world!") +``` + + + +The next code block is treated as an assertion. +If successful, the output is the following: + +``` +Hello, world! +``` + +For commands, such as `influxctl` CLI commands, that require launching an +OAuth URL in a browser, wrap the command in a subshell and redirect the output +to `/shared/urls.txt` in the container--for example: + +```sh +# Test the preceding command outside of the code block. +# influxctl authentication requires TTY interaction-- +# output the auth URL to a file that the host can open. +script -c "influxctl user list " \ + /dev/null > /shared/urls.txt +``` + +You probably don't want to display this syntax in the docs, which unfortunately +means you'd need to include the test block separately from the displayed code +block. +To hide it from users, wrap the code block inside an HTML comment. +pytest-codeblocks will still collect and run the code block. + +##### Mark tests to skip + +pytest-codeblocks has features for skipping tests and marking blocks as failed. +To learn more, see the pytest-codeblocks README and tests. + +#### Troubleshoot tests + +### Pytest collected 0 items + +Potential reasons: + +- See the test discovery options in `pytest.ini`. +- For Python code blocks, use the following delimiter: + + ```python + # Codeblocks runs this block. + ``` + + `pytest --codeblocks` ignores code blocks that use the following: + + ```py + # Codeblocks ignores this block. + ``` + +### Vale style linting + +docs-v2 includes Vale writing style linter configurations to enforce documentation writing style rules, guidelines, branding, and vocabulary terms. + +To run Vale, use the Vale extension for your editor or the included Docker configuration. +For example, the following command runs Vale in a container and lints `*.md` (Markdown) files in the path `./content/influxdb/cloud-dedicated/write-data/` using the specified configuration for `cloud-dedicated`: + +```sh +docker compose run -T vale --config=content/influxdb/cloud-dedicated/.vale.ini --minAlertLevel=error content/influxdb/cloud-dedicated/write-data/**/*.md +``` + +The output contains error-level style alerts for the Markdown content. + +**Note**: We strongly recommend running Vale, but it's not included in the +docs-v2 pre-commit hooks](#automatic-pre-commit-checks) for now. +You can include it in your own Git hooks. + +If a file contains style, spelling, or punctuation problems, +the Vale linter can raise one of the following alert levels: + +- **Error**: + - Problems that can cause content to render incorrectly + - Violations of branding guidelines or trademark guidelines + - Rejected vocabulary terms +- **Warning**: General style guide rules and best practices +- **Suggestion**: Style preferences that may require refactoring or updates to an exceptions list + +### Integrate Vale with your editor + +To integrate Vale with VSCode: + +1. Install the [Vale VSCode](https://marketplace.visualstudio.com/items?itemName=ChrisChinchilla.vale-vscode) extension. +2. In the extension settings, set the `Vale:Vale CLI:Path` value to the path of your Vale binary (`${workspaceFolder}/node_modules/.bin/vale` for Yarn-installed Vale). + +To use with an editor other than VSCode, see the [Vale integration guide](https://vale.sh/docs/integrations/guide/). + +### Configure style rules + +`/.ci/vale/styles/` contains configuration files for the custom `InfluxDataDocs` style. + +The easiest way to add accepted or rejected spellings is to enter your terms (or regular expression patterns) into the Vocabulary files at `.ci/vale/styles/config/vocabularies`. + +To add accepted/rejected terms for specific products, configure a style for the product and include a `Branding.yml` configuration. As an example, see `content/influxdb/cloud-dedicated/.vale.ini` and `.ci/vale/styles/Cloud-Dedicated/Branding.yml`. + +To learn more about configuration and rules, see [Vale configuration](https://vale.sh/docs/topics/config). + +### Submit a pull request + +Push your changes up to your forked repository, then [create a new pull request](https://help.github.com/articles/creating-a-pull-request/). + +## Style & Formatting + +### Markdown + +Most docs-v2 documentation content uses [Markdown](https://en.wikipedia.org/wiki/Markdown). + +_Some parts of the documentation, such as `./api-docs`, contain Markdown within YAML and rely on additional tooling._ + +### Semantic line feeds + +Use [semantic line feeds](http://rhodesmill.org/brandon/2012/one-sentence-per-line/). +Separating each sentence with a new line makes it easy to parse diffs with the human eye. + +**Diff without semantic line feeds:** + +```diff +-Data is taking off. This data is time series. You need a database that specializes in time series. You should check out InfluxDB. ++Data is taking off. This data is time series. You need a database that specializes in time series. You need InfluxDB. +``` + +**Diff with semantic line feeds:** + +```diff +Data is taking off. +This data is time series. +You need a database that specializes in time series. +-You should check out InfluxDB. ++You need InfluxDB. +``` + +### Article headings + +Use only h2-h6 headings in markdown content. +h1 headings act as the page title and are populated automatically from the `title` frontmatter. +h2-h6 headings act as section headings. + +### Image naming conventions + +Save images using the following naming format: `project/version-context-description.png`. +For example, `influxdb/2-0-visualizations-line-graph.png` or `influxdb/2-0-tasks-add-new.png`. +Specify a version other than 2.0 only if the image is specific to that version. + +## Page frontmatter + +Every documentation page includes frontmatter which specifies information about the page. +Frontmatter populates variables in page templates and the site's navigation menu. + +```yaml +title: # Title of the page used in the page's h1 +seotitle: # Page title used in the html title and used in search engine results +list_title: # Title used in article lists generated using the {{< children >}} shortcode +description: # Page description displayed in search engine results +menu: + influxdb_2_0: + name: # Article name that only appears in the left nav + parent: # Specifies a parent group and nests navigation items +weight: # Determines sort order in both the nav tree and in article lists +draft: # If true, will not render page on build +product/v2.x/tags: # Tags specific to each version (replace product and .x" with the appropriate product and minor version ) +related: # Creates links to specific internal and external content at the bottom of the page + - /path/to/related/article + - https://external-link.com, This is an external link +external_url: # Used in children shortcode type="list" for page links that are external +list_image: # Image included with article descriptions in children type="articles" shortcode +list_note: # Used in children shortcode type="list" to add a small note next to listed links +list_code_example: # Code example included with article descriptions in children type="articles" shortcode +list_query_example:# Code examples included with article descriptions in children type="articles" shortcode, + # References to examples in data/query_examples +canonical: # Path to canonical page, overrides auto-gen'd canonical URL +v2: # Path to v2 equivalent page +prepend: # Prepend markdown content to an article (especially powerful with cascade) + block: # (Optional) Wrap content in a block style (note, warn, cloud) + content: # Content to prepend to article +append: # Append markdown content to an article (especially powerful with cascade) + block: # (Optional) Wrap content in a block style (note, warn, cloud) + content: # Content to append to article +metadata: [] # List of metadata messages to include under the page h1 +updated_in: # Product and version the referenced feature was updated in (displayed as a unique metadata) +source: # Specify a file to pull page content from (typically in /content/shared/) +``` + +### Title usage + +##### `title` + +The `title` frontmatter populates each page's HTML `h1` heading tag. +It shouldn't be overly long, but should set the context for users coming from outside sources. + +##### `seotitle` + +The `seotitle` frontmatter populates each page's HTML `title` attribute. +Search engines use this in search results (not the page's h1) and therefore it should be keyword optimized. + +##### `list_title` + +The `list_title` frontmatter determines an article title when in a list generated +by the [`{{< children >}}` shortcode](#generate-a-list-of-children-articles). + +##### `menu > name` + +The `name` attribute under the `menu` frontmatter determines the text used in each page's link in the site navigation. +It should be short and assume the context of its parent if it has one. + +#### Page Weights + +To ensure pages are sorted both by weight and their depth in the directory +structure, pages should be weighted in "levels." +All top level pages are weighted 1-99. +The next level is 101-199. +Then 201-299 and so on. + +_**Note:** `_index.md` files should be weighted one level up from the other `.md` files in the same directory._ + +### Related content + +Use the `related` frontmatter to include links to specific articles at the bottom of an article. + +- If the page exists inside of this documentation, just include the path to the page. + It will automatically detect the title of the page. +- If the page exists inside of this documentation, but you want to customize the link text, + include the path to the page followed by a comma, and then the custom link text. + The path and custom text must be in that order and separated by a comma and a space. +- If the page exists outside of this documentation, include the full URL and a title for the link. + The link and title must be in that order and separated by a comma and a space. + +```yaml +related: + - /v2.0/write-data/quick-start + - /v2.0/write-data/quick-start, This is custom text for an internal link + - https://influxdata.com, This is an external link +``` + +### Canonical URLs + +Search engines use canonical URLs to accurately rank pages with similar or identical content. +The `canonical` HTML meta tag identifies which page should be used as the source of truth. + +By default, canonical URLs are automatically generated for each page in the InfluxData +documentation using the latest version of the current product and the current path. + +Use the `canonical` frontmatter to override the auto-generated canonical URL. + +_**Note:** The `canonical` frontmatter supports the [`{{< latest >}}` shortcode](#latest-links)._ + +```yaml +canonical: /path/to/canonical/doc/ + +# OR + +canonical: /{{< latest "influxdb" "v2" >}}/path/to/canonical/doc/ +``` + +### v2 equivalent documentation + +To display a notice on a 1.x page that links to an equivalent 2.0 page, +add the following frontmatter to the 1.x page: + +```yaml +v2: /influxdb/v2.0/get-started/ +``` + +### Prepend and append content to a page + +Use the `prepend` and `append` frontmatter to add content to the top or bottom of a page. +Each has the following fields: + +```yaml +append: | + > [!Note] + > #### This is example markdown content + > This is just an example note block that gets appended to the article. +``` + +Use this frontmatter with [cascade](#cascade) to add the same content to +all children pages as well. + +```yaml +cascade: + append: | + > [!Note] + > #### This is example markdown content + > This is just an example note block that gets appended to the article. +``` + +### Cascade + +To automatically apply frontmatter to a page and all of its children, use the +[`cascade` frontmatter](https://gohugo.io/content-management/front-matter/#front-matter-cascade) +built in into Hugo. + +```yaml +title: Example page +description: Example description +cascade: + layout: custom-layout +``` + +`cascade` applies the frontmatter to all children unless the child already includes +those frontmatter keys. Frontmatter defined on the page overrides frontmatter +"cascaded" from a parent. + +## Use shared content in a page + +Use the `source` frontmatter to specify a shared file to use to populate the +page content. Shared files are typically stored in the `/content/shared` directory. + +When building shared content, use the `show-in` and `hide-in` shortcodes to show +or hide blocks of content based on the current InfluxDB product/version. +For more information, see [show-in](#show-in) and [hide-in](#hide-in). + +## Shortcodes + +### Notes and warnings + +Shortcodes are available for formatting notes and warnings in each article: + +```md +{{% note %}} +Insert note markdown content here. +{{% /note %}} + +{{% warn %}} +Insert warning markdown content here. +{{% /warn %}} +``` + +### Product data + +Display the full product name and version name for the current page--for example: + +- InfluxDB 3 Core +- InfluxDB 3 Cloud Dedicated + +```md +{{% product-name %}} +``` + +Display the short version name (part of the key used in `products.yml`) from the current page URL--for example: + +- `/influxdb3/core` returns `core` + +```md +{{% product-key %}} +``` + +#### Enterprise name + +The name used to refer to InfluxData's enterprise offering is subject to change. +To facilitate easy updates in the future, use the `enterprise-name` shortcode +when referencing the enterprise product. +This shortcode accepts a `"short"` parameter which uses the "short-name". + +``` +This is content that references {{< enterprise-name >}}. +This is content that references {{< enterprise-name "short" >}}. +``` + +Product names are stored in `data/products.yml`. + +#### Enterprise link + +References to InfluxDB Enterprise are often accompanied with a link to a page where +visitors can get more information about the Enterprise offering. +This link is subject to change. +Use the `enterprise-link` shortcode when including links to more information about +InfluxDB Enterprise. + +``` +Find more info [here][{{< enterprise-link >}}] +``` + +### Latest patch version + +Use the `{{< latest-patch >}}` shortcode to add the latest patch version of a product. +By default, this shortcode parses the product and minor version from the URL. +To specify a specific product and minor version, use the `product` and `version` arguments. +Easier to maintain being you update the version number in the `data/products.yml` file instead of updating individual links and code examples. + +```md +{{< latest-patch >}} + +{{< latest-patch product="telegraf" >}} + +{{< latest-patch product="chronograf" version="1.7" >}} +``` + +### Latest influx CLI version + +Use the `{{< latest-patch cli=true >}}` shortcode to add the latest version of the `influx` +CLI supported by the minor version of InfluxDB. +By default, this shortcode parses the minor version from the URL. +To specify a specific minor version, use the `version` argument. +Maintain CLI version numbers in the `data/products.yml` file instead of updating individual links and code examples. + +```md +{{< latest-patch cli=true >}} + +{{< latest-cli version="2.1" >}} +``` + +### API endpoint + +Use the `{{< api-endpoint >}}` shortcode to generate a code block that contains +a colored request method, a specified API endpoint, and an optional link to +the API reference documentation. +Provide the following arguments: + +- **method**: HTTP request method (get, post, patch, put, or delete) +- **endpoint**: API endpoint +- **api-ref**: Link the endpoint to a specific place in the API documentation +- **influxdb_host**: Specify which InfluxDB product host to use + _if the `endpoint` contains the `influxdb/host` shortcode_. + Uses the current InfluxDB product as default. + Supports the following product values: + + - oss + - cloud + - serverless + - dedicated + - clustered + +```md +{{< api-endpoint method="get" endpoint="/api/v2/tasks" api-ref="/influxdb/cloud/api/#operation/GetTasks">}} +``` + +```md +{{< api-endpoint method="get" endpoint="{{< influxdb/host >}}/api/v2/tasks" influxdb_host="cloud">}} +``` + +### Tabbed Content + +To create "tabbed" content (content that is changed by a users' selection), use the following three shortcodes in combination: + +`{{< tabs-wrapper >}}` +This shortcode creates a wrapper or container for the tabbed content. +All UI interactions are limited to the scope of each container. +If you have more than one "group" of tabbed content in a page, each needs its own `tabs-wrapper`. +This shortcode must be closed with `{{< /tabs-wrapper >}}`. + +**Note**: The `<` and `>` characters used in this shortcode indicate that the contents should be processed as HTML. + +`{{% tabs %}}` +This shortcode creates a container for buttons that control the display of tabbed content. +It should contain simple markdown links with anonymous anchors (`#`). +The link text is used as the button text. +This shortcode must be closed with `{{% /tabs %}}`. + +**Note**: The `%` characters used in this shortcode indicate that the contents should be processed as Markdown. + +The `{{% tabs %}}` shortcode has an optional `style` argument that lets you +assign CSS classes to the tags HTML container. The following classes are available: + +- **small**: Tab buttons are smaller and don't scale to fit the width. +- **even-wrap**: Prevents uneven tab widths when tabs are forced to wrap. + +`{{% tab-content %}}` +This shortcode creates a container for a content block. +Each content block in the tab group needs to be wrapped in this shortcode. +**The number of `tab-content` blocks must match the number of links provided in the `tabs` shortcode** +This shortcode must be closed with `{{% /tab-content %}}`. + +**Note**: The `%` characters used in this shortcode indicate that the contents should be processed as Markdown. + +#### Example tabbed content group + +```md +{{< tabs-wrapper >}} + +{{% tabs %}} +[Button text for tab 1](#) +[Button text for tab 2](#) +{{% /tabs %}} + +{{% tab-content %}} +Markdown content for tab 1. +{{% /tab-content %}} + +{{% tab-content %}} +Markdown content for tab 2. +{{% /tab-content %}} + +{{< /tabs-wrapper >}} +``` + +#### Tabbed code blocks + +Shortcodes are also available for tabbed code blocks primarily used to give users +the option to choose between different languages and syntax. +The shortcode structure is the same as above, but the shortcode names are different: + +`{{< code-tabs-wrapper >}}` +`{{% code-tabs %}}` +`{{% code-tab-content %}}` + +````md +{{< code-tabs-wrapper >}} + +{{% code-tabs %}} +[Flux](#) +[InfluxQL](#) +{{% /code-tabs %}} + +{{% code-tab-content %}} + +```js +data = from(bucket: "example-bucket") + |> range(start: -15m) + |> filter(fn: (r) => + r._measurement == "mem" and + r._field == "used_percent" + ) +``` + +{{% /code-tab-content %}} + +{{% code-tab-content %}} + +```sql +SELECT "used_percent" +FROM "telegraf"."autogen"."mem" +WHERE time > now() - 15m +``` + +{{% /code-tab-content %}} + +{{< /code-tabs-wrapper >}} +```` + +#### Link to tabbed content + +To link to tabbed content, click on the tab and use the URL parameter shown. +It will have the form `?t=`, plus a string. +For example: + +``` +[Windows installation](/influxdb/v2.0/install/?t=Windows) +``` + +### Required elements + +Use the `{{< req >}}` shortcode to identify required elements in documentation with +orange text and/or asterisks. By default, the shortcode outputs the text, "Required," but +you can customize the text by passing a string argument with the shortcode. + +```md +{{< req >}} +``` + +**Output:** Required + +```md +{{< req "This is Required" >}} +``` + +**Output:** This is required + +If using other named arguments like `key` or `color`, use the `text` argument to +customize the text of the required message. + +```md +{{< req text="Required if ..." color="blue" type="key" >}} +``` + +#### Required elements in a list + +When identifying required elements in a list, use `{{< req type="key" >}}` to generate +a "\* Required" key before the list. For required elements in the list, include +{{< req "\*" >}} before the text of the list item. For example: + +```md +{{< req type="key" >}} + +- {{< req "\*" >}} **This element is required** +- {{< req "\*" >}} **This element is also required** +- **This element is NOT required** +``` + +#### Change color of required text + +Use the `color` argument to change the color of required text. +The following colors are available: + +- blue +- green +- magenta + +```md +{{< req color="magenta" text="This is required" >}} +``` + +### Page navigation buttons + +Use the `{{< page-nav >}}` shortcode to add page navigation buttons to a page. +These are useful for guiding users through a set of docs that should be read in sequential order. +The shortcode has the following parameters: + +- **prev:** path of the previous document _(optional)_ +- **next:** path of the next document _(optional)_ +- **prevText:** override the button text linking to the previous document _(optional)_ +- **nextText:** override the button text linking to the next document _(optional)_ +- **keepTab:** include the currently selected tab in the button link _(optional)_ + +The shortcode generates buttons that link to both the previous and next documents. +By default, the shortcode uses either the `list_title` or the `title` of the linked +document, but you can use `prevText` and `nextText` to override button text. + +```md + + +{{ page-nav prev="/path/to/prev/" next="/path/to/next" >}} + + + +{{ page-nav prev="/path/to/prev/" prevText="Previous" next="/path/to/next" nextText="Next" >}} + + + +{{ page-nav prev="/path/to/prev/" next="/path/to/next" keepTab=true>}} +``` + +### Keybinds + +Use the `{{< keybind >}}` shortcode to include OS-specific keybindings/hotkeys. +The following parameters are available: + +- mac +- linux +- win +- all +- other + +```md + + +{{< keybind mac="⇧⌘P" other="Ctrl+Shift+P" >}} + + + +{{< keybind all="Ctrl+Shift+P" >}} + + + +{{< keybind mac="⇧⌘P" linux="Ctrl+Shift+P" win="Ctrl+Shift+Alt+P" >}} +``` + +### Diagrams + +Use the `{{< diagram >}}` shortcode to dynamically build diagrams. +The shortcode uses [mermaid.js](https://github.com/mermaid-js/mermaid) to convert +simple text into SVG diagrams. +For information about the syntax, see the [mermaid.js documentation](https://mermaid-js.github.io/mermaid/#/). + +```md +{{< diagram >}} +flowchart TB +This --> That +That --> There +{{< /diagram >}} +``` + +### File system diagrams + +Use the `{{< filesystem-diagram >}}` shortcode to create a styled file system +diagram using a Markdown unordered list. + +##### Example filesystem diagram shortcode + +```md +{{< filesystem-diagram >}} + +- Dir1/ +- Dir2/ + - ChildDir/ + - Child + - Child +- Dir3/ + {{< /filesystem-diagram >}} +``` + +### High-resolution images + +In many cases, screenshots included in the docs are taken from high-resolution (retina) screens. +Because of this, the actual pixel dimension is 2x larger than it needs to be and is rendered 2x bigger than it should be. +The following shortcode automatically sets a fixed width on the image using half of its actual pixel dimension. +This preserves the detail of the image and renders it at a size where there should be little to no "blur" +cause by browser image resizing. + +```html +{{< img-hd src="/path/to/image" alt="Alternate title" />}} +``` + +###### Notes + +- This should only be used on screenshots takes from high-resolution screens. +- The `src` should be relative to the `static` directory. +- Image widths are limited to the width of the article content container and will scale accordingly, + even with the `width` explicitly set. + +### Truncated content blocks + +In some cases, it may be appropriate to shorten or truncate blocks of content. +Use cases include long examples of output data or tall images. +The following shortcode truncates blocks of content and allows users to opt into +to seeing the full content block. + +```md +{{% truncate %}} +Truncated markdown content here. +{{% /truncate %}} +``` + +### Expandable accordion content blocks + +Use the `{{% expand "Item label" %}}` shortcode to create expandable, accordion-style content blocks. +Each expandable block needs a label that users can click to expand or collapse the content block. +Pass the label as a string to the shortcode. + +```md +{{% expand "Label 1" %}} +Markdown content associated with label 1. +{{% /expand %}} + +{{% expand "Label 2" %}} +Markdown content associated with label 2. +{{% /expand %}} + +{{% expand "Label 3" %}} +Markdown content associated with label 3. +{{% /expand %}} +``` + +Use the optional `{{< expand-wrapper >}}` shortcode around a group of `{{% expand %}}` +shortcodes to ensure proper spacing around the expandable elements: + +```md +{{< expand-wrapper >}} +{{% expand "Label 1" %}} +Markdown content associated with label 1. +{{% /expand %}} + +{{% expand "Label 2" %}} +Markdown content associated with label 2. +{{% /expand %}} +{{< /expand-wrapper >}} +``` + +### Captions + +Use the `{{% caption %}}` shortcode to add captions to images and code blocks. +Captions are styled with a smaller font size, italic text, slight transparency, +and appear directly under the previous image or code block. + +```md +{{% caption %}} +Markdown content for the caption. +{{% /caption %}} +``` + +### Generate a list of children articles + +Section landing pages often contain just a list of articles with links and descriptions for each. +This can be cumbersome to maintain as content is added. +To automate the listing of articles in a section, use the `{{< children >}}` shortcode. + +```md +{{< children >}} +``` + +The children shortcode can also be used to list only "section" articles (those with their own children), +or only "page" articles (those with no children) using the `show` argument: + +```md +{{< children show="sections" >}} + + + +{{< children show="pages" >}} +``` + +_By default, it displays both sections and pages._ + +Use the `type` argument to specify the format of the children list. + +```md +{{< children type="functions" >}} +``` + +The following list types are available: + +- **articles:** lists article titles as headers with the description or summary + of the article as a paragraph. Article headers link to the articles. +- **list:** lists children article links in an unordered list. +- **anchored-list:** lists anchored children article links in an unordered list + meant to act as a page navigation and link to children header. +- **functions:** a special use-case designed for listing Flux functions. + +#### Include a "Read more" link + +To include a "Read more" link with each child summary, set `readmore=true`. +_Only the `articles` list type supports "Read more" links._ + +```md +{{< children readmore=true >}} +``` + +#### Include a horizontal rule + +To include a horizontal rule after each child summary, set `hr=true`. +_Only the `articles` list type supports horizontal rules._ + +```md +{{< children hr=true >}} +``` + +#### Include a code example with a child summary + +Use the `list_code_example` frontmatter to provide a code example with an article +in an articles list. + +````yaml +list_code_example: | + ```sh + This is a code example + ``` +```` + +#### Organize and include native code examples + +To include text from a file in `/shared/text/`, use the +`{{< get-shared-text >}}` shortcode and provide the relative path and filename. + +This is useful for maintaining and referencing sample code variants in their +native file formats. + +1. Store code examples in their native formats at `/shared/text/`. + +```md +/shared/text/example1/example.js +/shared/text/example1/example.py +``` + +2. Include the files--for example, in code tabs: + + ````md + {{% code-tabs-wrapper %}} + {{% code-tabs %}} + [Javascript](#js) + [Python](#py) + {{% /code-tabs %}} + {{% code-tab-content %}} + + ```js + {{< get-shared-text "example1/example.js" >}} + ``` + + {{% /code-tab-content %}} + {{% code-tab-content %}} + + ```py + {{< get-shared-text "example1/example.py" >}} + ``` + + {{% /code-tab-content %}} + {{% /code-tabs-wrapper %}} + ```` + +#### Include specific files from the same directory + +To include the text from one file in another file in the same +directory, use the `{{< get-leaf-text >}}` shortcode. +The directory that contains both files must be a +Hugo [_Leaf Bundle_](https://gohugo.io/content-management/page-bundles/#leaf-bundles), +a directory that doesn't have any child directories. + +In the following example, `api` is a leaf bundle. `content` isn't. + +```md +content +| +|--- api +| query.pdmc +| query.sh +| \_index.md +``` + +##### query.pdmc + +```md +# Query examples +``` + +##### query.sh + +```md +curl https://localhost:8086/query +``` + +To include `query.sh` and `query.pdmc` in `api/_index.md`, use the following code: + +````md +{{< get-leaf-text "query.pdmc" >}} + +# Curl example + +```sh +{{< get-leaf-text "query.sh" >}} +``` +```` + +Avoid using the following file extensions when naming included text files since Hugo interprets these as markup languages: +`.ad`, `.adoc`, `.asciidoc`, `.htm`, `.html`, `.markdown`, `.md`, `.mdown`, `.mmark`, `.pandoc`, `.pdc`, `.org`, or `.rst`. + +#### Reference a query example in children + +To include a query example with the children in your list, update `data/query_examples.yml` +with the example code, input, and output, and use the `list_query_example` +frontmatter to reference the corresponding example. + +```yaml +list_query_example: cumulative_sum +``` + +#### Children frontmatter + +Each children list `type` uses [frontmatter properties](#page-frontmatter) when generating the list of articles. +The following table shows which children types use which frontmatter properties: + +| Frontmatter | articles | list | functions | +| :------------------- | :------: | :--: | :-------: | +| `list_title` | ✓ | ✓ | ✓ | +| `description` | ✓ | | | +| `external_url` | ✓ | ✓ | | +| `list_image` | ✓ | | | +| `list_note` | | ✓ | | +| `list_code_example` | ✓ | | | +| `list_query_example` | ✓ | | | + +### Authentication token link + +Use the `{{% token-link "" "%}}` shortcode to +automatically generate links to token management documentation. The shortcode +accepts two _optional_ arguments: + +- **descriptor**: An optional token descriptor +- **link_append**: An optional path to append to the token management link path, + `///admin/tokens/`. + +```md +{{% token-link "database" "resource/" }} + + +[database token](/influxdb3/enterprise/admin/tokens/resource/) +``` + +InfluxDB 3 Enterprise and InfluxDB 3 Core support different kinds of tokens. +The shortcode has a blacklist of token descriptors for each that will prevent +unsupported descriptors from appearing in the rendered output based on the +current product. + +### Inline icons + +The `icon` shortcode allows you to inject icons in paragraph text. +It's meant to clarify references to specific elements in the InfluxDB user interface. +This shortcode supports Clockface (the UI) v2 and v3. +Specify the version to use as the second argument. The default version is `v3`. + +``` +{{< icon "icon-name" "v2" >}} +``` + +Below is a list of available icons (some are aliases): + +- add-cell +- add-label +- alert +- calendar +- chat +- checkmark +- clone +- cloud +- cog +- config +- copy +- dashboard +- dashboards +- data-explorer +- delete +- download +- duplicate +- edit +- expand +- export +- eye +- eye-closed +- eye-open +- feedback +- fullscreen +- gear +- graph +- hide +- influx +- influx-icon +- nav-admin +- nav-config +- nav-configuration +- nav-dashboards +- nav-data-explorer +- nav-organizations +- nav-orgs +- nav-tasks +- note +- notebook +- notebooks +- org +- orgs +- pause +- pencil +- play +- plus +- refresh +- remove +- replay +- save-as +- search +- settings +- tasks +- toggle +- trash +- trashcan +- triangle +- view +- wrench +- x + +### InfluxDB UI left navigation icons + +In many cases, documentation references an item in the left nav of the InfluxDB UI. +Provide a visual example of the navigation item using the `nav-icon` shortcode. +This shortcode supports Clockface (the UI) v2 and v3. +Specify the version to use as the second argument. The default version is `v3`. + +``` +{{< nav-icon "tasks" "v2" >}} +``` + +The following case insensitive values are supported: + +- admin, influx +- data-explorer, data explorer +- notebooks, books +- dashboards +- tasks +- monitor, alerts, bell +- cloud, usage +- data, load data, load-data +- settings +- feedback + +### Flexbox-formatted content blocks + +CSS Flexbox formatting lets you create columns in article content that adjust and +flow based on the viewable width. +In article content, this helps if you have narrow tables that could be displayed +side-by-side, rather than stacked vertically. +Use the `{{< flex >}}` shortcode to create the Flexbox wrapper. +Use the `{{% flex-content %}}` shortcode to identify each column content block. + +```md +{{< flex >}} +{{% flex-content %}} +Column 1 +{{% /flex-content %}} +{{% flex-content %}} +Column 2 +{{% /flex-content %}} +{{< /flex >}} +``` + +`{{% flex-content %}}` has an optional width argument that determines the maximum +width of the column. + +```md +{{% flex-content "half" %}} +``` + +The following options are available: + +- half _(Default)_ +- third +- quarter + +### Tooltips + +Use the `{{< tooltip >}}` shortcode to add tooltips to text. +The **first** argument is the text shown in the tooltip. +The **second** argument is the highlighted text that triggers the tooltip. + +```md +I like {{< tooltip "Butterflies are awesome!" "butterflies" >}}. +``` + +The rendered output is "I like butterflies" with "butterflies" highlighted. +When you hover over "butterflies," a tooltip appears with the text: "Butterflies are awesome!" + +### Flux sample data tables + +The Flux `sample` package provides basic sample datasets that can be used to +illustrate how Flux functions work. To quickly display one of the raw sample +datasets, use the `{{% flux/sample %}}` shortcode. + +The `flux/sample` shortcode has the following arguments that can be specified +by name or positionally. + +#### set + +Sample dataset to output. Use either `set` argument name or provide the set +as the first argument. The following sets are available: + +- float +- int +- uint +- string +- bool +- numericBool + +#### includeNull + +Specify whether or not to include _null_ values in the dataset. +Use either `includeNull` argument name or provide the boolean value as the second argument. + +#### includeRange + +Specify whether or not to include time range columns (`_start` and `_stop`) in the dataset. +This is only recommended when showing how functions that require a time range +(such as `window()`) operate on input data. +Use either `includeRange` argument name or provide the boolean value as the third argument. + +##### Example Flux sample data shortcodes + +```md + + +{{% flux/sample %}} + + + +{{% flux/sample set="string" includeNull=false %}} + + + +{{% flux/sample "int" true %}} + + + + +{{% flux/sample set="int" includeNull=true includeRange=true %}} +{{% flux/sample "int" true true %}} +``` + +### Duplicate OSS content in Cloud + +Docs for InfluxDB OSS and InfluxDB Cloud share a majority of content. +To prevent duplication of content between versions, use the following shortcodes: + +- `{{< duplicate-oss >}}` +- `{{% oss-only %}}` +- `{{% cloud-only %}}` + +#### duplicate-oss + +The `{{< duplicate-oss >}}` shortcode copies the page content of the file located +at the identical file path in the most recent InfluxDB OSS version. +The Cloud version of this markdown file should contain the frontmatter required +for all pages, but the body content should just be the `{{< duplicate-oss >}}` shortcode. + +#### oss-only + +Wrap content that should only appear in the OSS version of the doc with the `{{% oss-only %}}` shortcode. +Use the shortcode on both inline and content blocks: + +```md +{{% oss-only %}}This is inline content that only renders in the InfluxDB OSS docs{{% /oss-only %}} + +{{% oss-only %}} + +This is a multi-paragraph content block that spans multiple paragraphs and will +only render in the InfluxDB OSS documentation. + +**Note:** Notice the blank newline after the opening short-code tag. +This is necessary to get the first sentence/paragraph to render correctly. + +{{% /oss-only %}} + +- {{% oss-only %}}This is a list item that will only render in InfluxDB OSS docs.{{% /oss-only %}} +- {{% oss-only %}} + + This is a multi-paragraph list item that will only render in the InfluxDB OSS docs. + + **Note:** Notice shortcode is _inside_ of the line item. + There also must be blank newline after the opening short-code tag. + This is necessary to get the first sentence/paragraph to render correctly. + + {{% /oss-only %}} + +1. Step 1 +2. {{% oss-only %}}This is a list item that will only render in InfluxDB OSS docs.{{% /oss-only %}} +3. {{% oss-only %}} + + This is a list item that contains multiple paragraphs or nested list items and will only render in the InfluxDB OSS docs. + + **Note:** Notice shortcode is _inside_ of the line item. + There also must be blank newline after the opening short-code tag. + This is necessary to get the first sentence/paragraph to render correctly. + + {{% /oss-only %}} +``` + +#### cloud-only + +Wrap content that should only appear in the Cloud version of the doc with the `{{% cloud-only %}}` shortcode. +Use the shortcode on both inline and content blocks: + +```md +{{% cloud-only %}}This is inline content that only renders in the InfluxDB Cloud docs{{% /cloud-only %}} + +{{% cloud-only %}} + +This is a multi-paragraph content block that spans multiple paragraphs and will +only render in the InfluxDB Cloud documentation. + +**Note:** Notice the blank newline after the opening short-code tag. +This is necessary to get the first sentence/paragraph to render correctly. + +{{% /cloud-only %}} + +- {{% cloud-only %}}This is a list item that will only render in InfluxDB Cloud docs.{{% /cloud-only %}} +- {{% cloud-only %}} + + This is a list item that contains multiple paragraphs or nested list items and will only render in the InfluxDB Cloud docs. + + **Note:** Notice shortcode is _inside_ of the line item. + There also must be blank newline after the opening short-code tag. + This is necessary to get the first sentence/paragraph to render correctly. + + {{% /cloud-only %}} + +1. Step 1 +2. {{% cloud-only %}}This is a list item that will only render in InfluxDB Cloud docs.{{% /cloud-only %}} +3. {{% cloud-only %}} + + This is a multi-paragraph list item that will only render in the InfluxDB Cloud docs. + + **Note:** Notice shortcode is _inside_ of the line item. + There also must be blank newline after the opening short-code tag. + This is necessary to get the first sentence/paragraph to render correctly. + + {{% /cloud-only %}} +``` + +### Show or hide content blocks in shared content + +The `source` frontmatter lets you source page content from another file and is +used to share content across InfluxDB products. Within the shared content, you +can use the `show-in` and `hide-in` shortcodes to conditionally show or hide +content blocks based on the InfluxDB "version." Valid "versions" include: + +- v2 +- cloud +- cloud-serverless +- cloud-dedicated +- clustered +- core +- enterprise + +#### show-in + +The `show-in` shortcode accepts a comma-delimited string of InfluxDB "versions" +to show the content block in. The version is the second level of the page +path--for example: `/influxdb//...`. + +```md +{{% show-in "core,enterprise" %}} + +This content will appear in pages in the InfluxDB 3 Core and InfluxDB 3 Enterprise +documentation, but not any other InfluxDB documentation this content is shared in. + +{{% /show-in %}} +``` + +#### hide-in + +The `hide-in` shortcode accepts a comma-delimited string of InfluxDB "versions" +to hide the content block in. The version is the second level of the page +path--for example: `/influxdb//...`. + +```md +{{% hide-in "core,enterprise" %}} + +This content will not appear in pages in the InfluxDB 3 Core and InfluxDB 3 +Enterprise documentation, but will in all other InfluxDB documentation this +content is shared in. + +{{% /hide-in %}} +``` + +### All-Caps + +Clockface v3 introduces many buttons with text formatted as all-caps. +Use the `{{< caps >}}` shortcode to format text to match those buttons. + +```md +Click {{< caps >}}Add Data{{< /caps >}} +``` + +### Code callouts + +Use the `{{< code-callout >}}` shortcode to highlight and emphasize a specific +piece of code (for example, a variable, placeholder, or value) in a code block. +Provide the string to highlight in the code block. +Include a syntax for the codeblock to properly style the called out code. + +````md +{{< code-callout "03a2bbf46249a000" >}} + +```sh +http://localhost:8086/orgs/03a2bbf46249a000/... +``` + +{{< /code-callout >}} +```` + +### InfluxDB University banners + +Use the `{{< influxdbu >}}` shortcode to add an InfluxDB University banner that +points to the InfluxDB University site or a specific course. +Use the default banner template, a predefined course template, or fully customize +the content of the banner. + +```html + +{{< influxdbu >}} + + +{{< influxdbu "influxdb-101" >}} + + +{{< influxdbu title="Course title" summary="Short course summary." action="Take +the course" link="https://university.influxdata.com/" >}} +``` + +#### Course templates + +Use one of the following course templates: + +- influxdb-101 +- telegraf-102 +- flux-103 + +#### Custom banner content + +Use the following shortcode parameters to customize the content of the InfluxDB +University banner: + +- **title**: Course or banner title +- **summary**: Short description shown under the title +- **action**: Text of the button +- **link**: URL the button links to + +### Reference content + +The InfluxDB documentation is "task-based," meaning content primarily focuses on +what a user is **doing**, not what they are **using**. +However, there is a need to document tools and other things that don't necessarily +fit in the task-based style. +This is referred to as "reference content." + +Reference content is styled just as the rest of the InfluxDB documentation. +The only difference is the `menu` reference in the page's frontmatter. +When defining the menu for reference content, use the following pattern: + +```yaml +# Pattern +menu: + ___ref: + # ... + +# Example +menu: + influxdb_2_0_ref: + # ... +``` + +## InfluxDB URLs + +When a user selects an InfluxDB product and region, example URLs in code blocks +throughout the documentation are updated to match their product and region. +InfluxDB URLs are configured in `/data/influxdb_urls.yml`. + +By default, the InfluxDB URL replaced inside of code blocks is `http://localhost:8086`. +Use this URL in all code examples that should be updated with a selected provider and region. + +For example: + +```` +```sh +# This URL will get updated +http://localhost:8086 + +# This URL will NOT get updated +http://example.com +``` +```` + +If the user selects the **US West (Oregon)** region, all occurrences of `http://localhost:8086` +in code blocks will get updated to `https://us-west-2-1.aws.cloud2.influxdata.com`. + +### Exempt URLs from getting updated + +To exempt a code block from being updated, include the `{{< keep-url >}}` shortcode +just before the code block. + +```` +{{< keep-url >}} +``` +// This URL won't get updated +http://localhost:8086 +``` +```` + +### Code examples only supported in InfluxDB Cloud + +Some functionality is only supported in InfluxDB Cloud and code examples should +only use InfluxDB Cloud URLs. In these cases, use `https://cloud2.influxdata.com` +as the placeholder in the code block. It will get updated on page load and when +users select a Cloud region in the URL select modal. + +```` +```sh +# This URL will get updated +https://cloud2.influxdata.com +``` +```` + +### Automatically populate InfluxDB host placeholder + +The InfluxDB host placeholder that gets replaced by custom domains differs +between each InfluxDB product/version. +Use the `influxdb/host` shortcode to automatically render the correct +host placeholder value for the current product. You can also pass a single +argument to specify a specific InfluxDB product to use. +Supported argument values: + +- oss +- cloud +- cloud-serverless +- cloud-dedicated +- clustered +- core +- enterprise + +``` +{{< influxdb/host >}} + +{{< influxdb/host "serverless" >}} +``` + +### User-populated placeholders + +Use the `code-placeholders` shortcode to format placeholders +as text fields that users can populate with their own values. +The shortcode takes a regular expression for matching placeholder names. +Use the `code-placeholder-key` shortcode to format the placeholder names in +text that describes the placeholder--for example: + +``` +{{% code-placeholders "DATABASE_NAME|USERNAME|PASSWORD_OR_TOKEN|API_TOKEN|exampleuser@influxdata.com" %}} +```sh +curl --request POST http://localhost:8086/write?db=DATABASE_NAME \ + --header "Authorization: Token API_TOKEN" \ + --data-binary @path/to/line-protocol.txt +``` +{{% /code-placeholders %}} + +Replace the following: + +- {{% code-placeholder-key %}}`DATABASE_NAME` and `RETENTION_POLICY`{{% /code-placeholder-key %}}: the [database and retention policy mapping (DBRP)](/influxdb/v2/reference/api/influxdb-1x/dbrp/) for the InfluxDB v2 bucket that you want to write to +- {{% code-placeholder-key %}}`USERNAME`{{% /code-placeholder-key %}}: your [InfluxDB 1.x username](/influxdb/v2/reference/api/influxdb-1x/#manage-credentials) +- {{% code-placeholder-key %}}`PASSWORD_OR_TOKEN`{{% /code-placeholder-key %}}: your [InfluxDB 1.x password or InfluxDB API token](/influxdb/v2/reference/api/influxdb-1x/#manage-credentials) +- {{% code-placeholder-key %}}`API_TOKEN`{{% /code-placeholder-key %}}: your [InfluxDB API token](/influxdb/v2/admin/tokens/) +``` + +## InfluxDB API documentation + +InfluxData uses [Redoc](https://github.com/Redocly/redoc) to generate the full +InfluxDB API documentation when documentation is deployed. +Redoc generates HTML documentation using the InfluxDB `swagger.yml`. +For more information about generating InfluxDB API documentation, see the +[API Documentation README](https://github.com/influxdata/docs-v2/tree/master/api-docs#readme). diff --git a/.github/instructions/influxdb3-code-placeholders.instructions.md b/.github/instructions/influxdb3-code-placeholders.instructions.md new file mode 100644 index 000000000..583ef705a --- /dev/null +++ b/.github/instructions/influxdb3-code-placeholders.instructions.md @@ -0,0 +1,89 @@ +--- +mode: 'edit' +applyTo: "content/{influxdb3/core,influxdb3/enterprise,shared/influxdb3*}/**" +--- +## Best Practices + +- Use UPPERCASE for placeholders to make them easily identifiable +- Don't use pronouns in placeholders (e.g., "your", "this") +- List placeholders in the same order they appear in the code +- Provide clear descriptions including: +- - Expected data type or format +- - Purpose of the value +- - Any constraints or requirements +- Mark optional placeholders as "Optional:" in their descriptions +- Placeholder key descriptions should fit the context of the code snippet +- Include examples for complex formats + +## Writing Placeholder Descriptions + +Descriptions should follow consistent patterns: + +1. **Admin Authentication tokens**: + - Recommended: "a {{% token-link "admin" %}} for your {{< product-name >}} instance" + - Avoid: "your token", "the token", "an authorization token" +2. **Database resource tokens**: + - Recommended: "your {{% token-link "database" %}}"{{% show-in "enterprise" %}} with permissions on the specified database{{% /show-in %}} + - Avoid: "your token", "the token", "an authorization token" +3. **Database names**: + - Recommended: "the name of the database to [action]" + - Avoid: "your database", "the database name" +4. **Conditional content**: + - Use `{{% show-in "enterprise" %}}` for content specific to enterprise versions + - Example: "your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to query the specified database{{% /show-in %}}" + +## Common placeholders for InfluxDB 3 + +- `AUTH_TOKEN`: your {{% token-link %}} +- `DATABASE_NAME`: the database to use +- `TABLE_NAME`: Name of the table/measurement to query or write to +- `NODE_ID`: Node ID for a specific node in a cluster +- `CLUSTER_ID`: Cluster ID for a specific cluster +- `HOST`: InfluxDB server hostname or URL +- `PORT`: InfluxDB server port (typically 8181) +- `QUERY`: SQL or InfluxQL query string +- `LINE_PROTOCOL`: Line protocol data for writes +- `PLUGIN_FILENAME`: Name of plugin file to use +- `CACHE_NAME`: Name for a new or existing cache + +## Hugo shortcodes in Markdown + +- `{{% code-placeholders "PLACEHOLDER1|PLACEHOLDER2" %}}`: Use this shortcode to define placeholders in code snippets. +- `{{% /code-placeholders %}}`: End the shortcode. +- `{{% code-placeholder-key %}}`: Use this shortcode to define a specific placeholder key. +- `{{% /code-placeholder-key %}}`: End the specific placeholder key shortcode. + +## Language-Specific Placeholder Formatting + +- **Bash/Shell**: Use uppercase variables with no quotes or prefix + ```bash + --database DATABASE_NAME + ``` +- Python: Use string literals with quotes + ```python + database_name='DATABASE_NAME' + ``` +- JSON: Use key-value pairs with quotes + ```json + { + "database": "DATABASE_NAME" + } + ``` + +## Real-World Examples from Documentation + +### InfluxDB CLI Commands +This pattern appears frequently in CLI documentation: + +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} +```bash +influxdb3 write \ + --database DATABASE_NAME \ + --token AUTH_TOKEN \ + --precision ns +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +{{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to write to +{{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with write permissions on the specified database{{% /show-in %}} \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ecf5c9cd4..479578424 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,6 @@ # Contributing to InfluxData Documentation -## Sign the InfluxData CLA +### Sign the InfluxData CLA The InfluxData Contributor License Agreement (CLA) is part of the legal framework for the open source ecosystem that protects both you and InfluxData. diff --git a/build-scripts/build-copilot-instructions.js b/build-scripts/build-copilot-instructions.js new file mode 100644 index 000000000..0d089e2c1 --- /dev/null +++ b/build-scripts/build-copilot-instructions.js @@ -0,0 +1,76 @@ +#!/usr/bin/env node + +/** + * Script to generate GitHub Copilot instructions + * for InfluxData documentation. + */ +import fs from 'fs'; +import path from 'path'; +import process from 'process'; +import { execSync } from 'child_process'; + +// Get the current file path and directory +export { buildContributingInstructions }; + +(async () => { + try { + await buildContributingInstructions(); + } catch (error) { + console.error('Error generating Copilot instructions:', error); + } +})(); + +/** Build instructions from CONTRIBUTING.md + * This script reads CONTRIBUTING.md, formats it appropriately, + * and saves it to .github/instructions/contributing.instructions.md + */ +function buildContributingInstructions() { + // Paths + const contributingPath = path.join(process.cwd(), 'CONTRIBUTING.md'); + const instructionsDir = path.join(process.cwd(), '.github', 'instructions'); + const instructionsPath = path.join( + instructionsDir, + 'contributing.instructions.md' + ); + + // Ensure the instructions directory exists + if (!fs.existsSync(instructionsDir)) { + fs.mkdirSync(instructionsDir, { recursive: true }); + } + + // Read the CONTRIBUTING.md file + let content = fs.readFileSync(contributingPath, 'utf8'); + + // Format the content for Copilot instructions with applyTo attribute + content = `--- +applyTo: "content/**/*.md, layouts/**/*.html" +--- + +# GitHub Copilot Instructions for InfluxData Documentation + +## Purpose and scope + +GitHub Copilot should help document InfluxData products +by creating clear, accurate technical content with proper +code examples, frontmatter, shortcodes, and formatting. + +${content}`; + + // Write the formatted content to the instructions file + fs.writeFileSync(instructionsPath, content); + + console.log(`✅ Generated Copilot instructions at ${instructionsPath}`); + + // Add the file to git if it has changed + try { + const gitStatus = execSync( + `git status --porcelain "${instructionsPath}"` + ).toString(); + if (gitStatus.trim()) { + execSync(`git add "${instructionsPath}"`); + console.log('✅ Added instructions file to git staging'); + } + } catch (error) { + console.warn('⚠️ Could not add instructions file to git:', error.message); + } +} diff --git a/compose.yaml b/compose.yaml index a2338c3b1..fe0293615 100644 --- a/compose.yaml +++ b/compose.yaml @@ -449,6 +449,9 @@ services: - type: bind source: ./content target: /app/content + - type: bind + source: ./CONTRIBUTING.md + target: /app/CONTRIBUTING.md volumes: test-content: cloud-tmp: diff --git a/lefthook.yml b/lefthook.yml index e813fa67e..907378b5a 100644 --- a/lefthook.yml +++ b/lefthook.yml @@ -5,10 +5,13 @@ pre-commit: parallel: true commands: + build-copilot-instructions: + glob: "CONTRIBUTING.md" + run: yarn build:copilot-instructions # Report linting warnings and errors, don't output files to stdout lint-markdown: tags: lint - glob: 'content/*.md' + glob: "{CONTRIBUTING.md,content/*.md}" run: | docker compose run --rm --name remark-lint remark-lint '{staged_files}' cloud-lint: diff --git a/package.json b/package.json index c3da669e5..7df0d28bc 100644 --- a/package.json +++ b/package.json @@ -41,6 +41,7 @@ }, "scripts": { "build:pytest:image": "docker build -t influxdata/docs-pytest:latest -f Dockerfile.pytest .", + "build:copilot-instructions": "node ./build-scripts/build-copilot-instructions.js", "lint": "LEFTHOOK_EXCLUDE=test lefthook run pre-commit && lefthook run pre-push", "pre-commit": "lefthook run pre-commit", "test": "echo \"Run 'yarn test:e2e', 'yarn test:links', 'yarn test:codeblocks:all' or a specific test command. e2e and links test commands can take a glob of file paths to test. Some commands run automatically during the git pre-commit and pre-push hooks.\" && exit 0", From cbd62a8f87c4295073d445826fe3dd3505c0afcc Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Tue, 27 May 2025 14:46:46 -0500 Subject: [PATCH 2/3] chore(mono): Core and Enterprise: add auth to Get Started examples (part of #6080) - Make it clear that a token is required when auth is activated. - Add --token for CLI examples in Get Started. - Add Auth header for API examples in Get Started. --- content/shared/v3-core-get-started/_index.md | 359 +++++++++++---- .../v3-enterprise-get-started/_index.md | 424 +++++++++++++----- 2 files changed, 579 insertions(+), 204 deletions(-) diff --git a/content/shared/v3-core-get-started/_index.md b/content/shared/v3-core-get-started/_index.md index f4e160ab4..7ebcfcd14 100644 --- a/content/shared/v3-core-get-started/_index.md +++ b/content/shared/v3-core-get-started/_index.md @@ -134,10 +134,9 @@ source ~/.zshrc #### Start InfluxDB -To start your InfluxDB instance, use the `influxdb3 serve` command -and provide the following: +To start your InfluxDB instance, use the `influxdb3 serve` command and provide the following: -- `--object-store`: Specifies the type of Object store to use. +`--object-store`: Specifies the type of object store to use. InfluxDB supports the following: local file system (`file`), `memory`, S3 (and compatible services like Ceph or Minio) (`s3`), Google Cloud Storage (`google`), and Azure Blob Storage (`azure`). @@ -148,6 +147,8 @@ and provide the following: This forms the final part of the storage path: `/`. In a multi-node setup, this ID is used to reference specific nodes. +The following examples show how to start {{% product-name %}} with different object store configurations. + > [!Note] > #### Diskless architecture > @@ -155,15 +156,13 @@ and provide the following: > storage alone, eliminating the need for locally attached disks. > {{% product-name %}} can also work with only local disk storage when needed. -The following examples show how to start InfluxDB 3 with different object store configurations: -```bash -# Memory object store -# Stores data in RAM; doesn't persist data -influxdb3 serve \ ---node-id host01 \ ---object-store memory -``` +##### Filesystem object store + +Store data in a specified directory on the local filesystem. +This is the default object store type. + +Replace the following with your values: ```bash # Filesystem object store @@ -174,15 +173,11 @@ influxdb3 serve \ --data-dir ~/.influxdb3 ``` -To run the [Docker image](/influxdb3/core/install/#docker-image) and persist data to the filesystem, mount a volume for the Object store-for example, pass the following options: +To run the [Docker image](/influxdb3/version/install/#docker-image) and persist data to the filesystem, mount a volume for the object store-for example, pass the following options: - `-v /path/on/host:/path/in/container`: Mounts a directory from your filesystem to the container - `--object-store file --data-dir /path/in/container`: Uses the mount for server storage -> [!Note] -> -> The {{% product-name %}} Docker image exposes port `8181`, the `influxdb3` server default for HTTP connections. -> To map the exposed port to a different port when running a container, see the Docker guide for [Publishing and exposing ports](https://docs.docker.com/get-started/docker-concepts/running-containers/publishing-ports/). ```bash @@ -197,13 +192,24 @@ docker run -it \ --data-dir /path/in/container ``` +> [!Note] +> +> The {{% product-name %}} Docker image exposes port `8181`, the `influxdb3` server default for HTTP connections. +> To map the exposed port to a different port when running a container, see the Docker guide for [Publishing and exposing ports](https://docs.docker.com/get-started/docker-concepts/running-containers/publishing-ports/). + +##### S3 object store + +Store data in an S3-compatible object store. +This is useful for production deployments that require high availability and durability. +Provide your bucket name and credentials to access the S3 object store. + ```bash # S3 object store (default is the us-east-1 region) -# Specify the Object store type and associated options +# Specify the object store type and associated options influxdb3 serve \ --node-id host01 \ --object-store s3 \ - --bucket BUCKET \ + --bucket OBJECT_STORE_BUCKET \ --aws-access-key AWS_ACCESS_KEY_ID \ --aws-secret-access-key AWS_SECRET_ACCESS_KEY ``` @@ -215,14 +221,27 @@ influxdb3 serve \ influxdb3 serve \ --node-id host01 \ --object-store s3 \ - --bucket BUCKET \ + --bucket OBJECT_STORE_BUCKET \ --aws-access-key-id AWS_ACCESS_KEY_ID \ --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ --aws-endpoint ENDPOINT \ --aws-allow-http ``` -For more information about server options, use the CLI help: +#### Memory object store + +Store data in RAM without persisting it on shutdown. +It's useful for rapid testing and development. + +```bash +# Memory object store +# Stores data in RAM; doesn't persist data +influxdb3 serve \ +--node-id host01 \ +--object-store memory +``` + +For more information about server options, use the CLI help or view the [InfluxDB 3 CLI reference](/influxdb3/version/reference/cli/serve/): ```bash influxdb3 serve --help @@ -232,7 +251,7 @@ influxdb3 serve --help After you have [started the server](#start-influxdb), you can create and manage tokens using the `influxdb3` CLI or the HTTP API. {{% product-name %}} uses token-based authentication and authorization which is enabled by default when you start the server. -With authentication enabled, you must provide a token to access server actions. +With authentication enabled, you must provide a token with `influxdb3` CLI commands and HTTP API requests. An {{% product-name %}} instance can have one _admin token_, which grants access to all CLI actions and API endpoints. When you create a token, InfluxDB 3 returns a token string in plain text @@ -282,8 +301,54 @@ Replace {{% code-placeholder-key %}}`CONTAINER_NAME`{{% /code-placeholder-key %} {{< /code-tabs-wrapper >}} The command returns a token string that you can use to authenticate CLI commands and API requests. +Store your token in a secure location, as you cannot retrieve it from the database later. -For more information, see how to [Manage admin tokens](/influxdb3/version/admin/tokens/admin/). +For more information about tokens, see how to [Manage admin tokens](/influxdb3/version/admin/tokens/admin/). + +#### Use tokens to authorize CLI commands and API requests + +With authentication enabled (the default), {{% product-name %}} requires a +token for all `influxdb3` CLI commands and HTTP API requests. + +In the following examples, replace {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}} with your {{% token-link "admin" %}} string. + +{{< tabs-wrapper >}} +{{% tabs %}} +[CLI](#cli-use-a-token) +[HTTP API](#api-use-a-token) +{{% /tabs %}} +{{% tab-content %}} +For `influxdb3` to use your token automatically, assign it your + token to the `INFLUXDB3_AUTH_TOKEN` environment variable: + +{{% code-placeholders "AUTH_TOKEN" %}} +```bash +# Set the environment variable for future CLI commands +export INFLUXDB3_AUTH_TOKEN=AUTH_TOKEN +``` +{{% /code-placeholders %}} + +Or to authenticate a single `influxdb3` CLI command and override `$INFLUXDB3_AUTH_TOKEN`, include the `--token` option: + +{{% code-placeholders "AUTH_TOKEN" %}} +```bash +# Use the --token option for a single command +influxdb3 show databases --token AUTH_TOKEN +``` +{{% /code-placeholders %}} +{{% /tab-content %}} +{{% tab-content %}} +To authenticate HTTP API requests, include `Bearer ` in the `Authorization` header value: + +{{% code-placeholders "AUTH_TOKEN" %}} +```bash +# Include the token in the Authorization HTTP request header +curl "http://{{< influxdb/host >}}/api/v3/configure/database" \ + --header "Authorization: Bearer AUTH_TOKEN" +``` +{{% /code-placeholders %}} +{{% /tab-content %}} +{{< /tabs-wrapper >}} ### Data model @@ -373,24 +438,18 @@ Use the `influxdb3 write` command to write data to a database. In the code samples, replace the following placeholders with your values: -- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: The name of the [database](/influxdb3/version/admin/databases/) to write to. -{{% show-in "core" %}} -- {{% code-placeholder-key %}}`TOKEN`{{% /code-placeholder-key %}}: A [token](/influxdb3/version/admin/tokens/) for your {{% product-name %}} server. -{{% /show-in %}} -{{% show-in "enterprise" %}} -- {{% code-placeholder-key %}}`TOKEN`{{% /code-placeholder-key %}}: A [token](/influxdb3/version/admin/tokens/) - with permission to write to the specified database. -{{% /show-in %}} +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the [database](/influxdb3/version/admin/databases/) to write to. +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to write to the specified database{{% /show-in %}} ##### Write data via stdin Pass data as quoted line protocol via standard input (stdin)--for example: -{{% code-placeholders "DATABASE_NAME|TOKEN" %}} +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash influxdb3 write \ --database DATABASE_NAME \ - --token TOKEN \ + --token AUTH_TOKEN \ --precision ns \ --accept-partial \ 'cpu,host=Alpha,region=us-west,application=webserver val=1i,usage_percent=20.5,status="OK" @@ -408,17 +467,21 @@ Pass the `--file` option to write line protocol you have saved to a file--for ex [sample line protocol](#write-data-in-line-protocol-syntax) to a file named `server_data` and then enter the following command: -{{% code-placeholders "DATABASE_NAME|TOKEN" %}} +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash influxdb3 write \ --database DATABASE_NAME \ - --token TOKEN \ + --token AUTH_TOKEN \ --precision ns \ --accept-partial \ - --file server_data + --file path/to/server_data ``` {{% /code-placeholders %}} +Replace the following placeholders with your values: +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the [database](/influxdb3/version/admin/databases/) to write to. +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to write to the specified database{{% /show-in %}} + ### Write data using the HTTP API {{% product-name %}} provides three write API endpoints that respond to HTTP `POST` requests. @@ -450,7 +513,7 @@ and supports the following parameters: - `?precision=`: Specify the precision of the timestamp. The default is nanosecond precision. - request body: The line protocol data to write. -For more information about the parameters, see [Write data](/influxdb3/core/write-data/). +For more information about the parameters, see [Write data](/influxdb3/version/write-data/). ##### Example: write data using the /api/v3 HTTP API @@ -463,6 +526,7 @@ With `accept_partial=true` (default): ```bash curl -v "http://{{< influxdb/host >}}/api/v3/write_lp?db=sensors&precision=auto" \ + --header 'Authorization: Bearer apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0==' \ --data-raw 'home,room=Sunroom temp=96 home,room=Sunroom temp="hi"' ``` @@ -493,6 +557,7 @@ With `accept_partial=false`: ```bash curl -v "http://{{< influxdb/host >}}/api/v3/write_lp?db=sensors&precision=auto&accept_partial=false" \ + --header 'Authorization: Bearer apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0==' \ --data-raw 'home,room=Sunroom temp=96 home,room=Sunroom temp="hi"' ``` @@ -546,17 +611,17 @@ The `/write` InfluxDB v1 compatibility endpoint provides backwards compatibility #### Write responses -By default, InfluxDB acknowledges writes after flushing the WAL file to the Object store (occurring every second). +By default, InfluxDB acknowledges writes after flushing the WAL file to the object store (occurring every second). For high write throughput, you can send multiple concurrent write requests. #### Use no_sync for immediate write responses To reduce the latency of writes, use the `no_sync` write option, which acknowledges writes _before_ WAL persistence completes. -When `no_sync=true`, InfluxDB validates the data, writes the data to the WAL, and then immediately responds to the client, without waiting for persistence to the Object store. +When `no_sync=true`, InfluxDB validates the data, writes the data to the WAL, and then immediately responds to the client, without waiting for persistence to the object store. Using `no_sync=true` is best when prioritizing high-throughput writes over absolute durability. -- Default behavior (`no_sync=false`): Waits for data to be written to the Object store before acknowledging the write. Reduces the risk of data loss, but increases the latency of the response. +- Default behavior (`no_sync=false`): Waits for data to be written to the object store before acknowledging the write. Reduces the risk of data loss, but increases the latency of the response. - With `no_sync=true`: Reduces write latency, but increases the risk of data loss in case of a crash before WAL persistence. ##### Immediate write using the HTTP API @@ -565,30 +630,27 @@ The `no_sync` parameter controls when writes are acknowledged--for example: ```bash curl "http://{{< influxdb/host >}}/api/v3/write_lp?db=sensors&precision=auto&no_sync=true" \ + --header 'Authorization: Bearer apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0==' \ --data-raw "home,room=Sunroom temp=96" ``` -##### Immediate write using the influxdb3 CLI - -The `no_sync` CLI option controls when writes are acknowledged--for example: - -```bash -influxdb3 write \ - --bucket mydb \ - --org my_org \ - --token my-token \ - --no-sync -``` - ### Create a database or table To create a database without writing data, use the `create` subcommand--for example: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash -influxdb3 create database mydb +influxdb3 create database DATABASE_NAME \ + --token AUTH_TOKEN ``` +{{% /code-placeholders %}} -To learn more about a subcommand, use the `-h, --help` flag: +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to create +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: the {{% token-link "admin" %}} for your {{% product-name %}} server + +To learn more about a subcommand, use the `-h, --help` flag or view the [InfluxDB 3 CLI reference](/influxdb3/version/reference/cli/create/): ```bash influxdb3 create -h @@ -596,7 +658,7 @@ influxdb3 create -h ### Query data -InfluxDB 3 now supports native SQL for querying, in addition to InfluxQL, an +InfluxDB 3 supports native SQL for querying, in addition to InfluxQL, an SQL-like language customized for time series queries. {{< product-name >}} limits @@ -662,12 +724,20 @@ $ influxdb3 query --database servers "SELECT DISTINCT usage_percent, time FROM c To query using InfluxQL, enter the `influxdb3 query` subcommand and specify `influxql` in the language option--for example: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash influxdb3 query \ - --database servers \ + --database DATABASE_NAME \ + --token \ --language influxql \ "SELECT DISTINCT usage_percent FROM cpu WHERE time >= now() - 1d" ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to query +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to query the specified database{{% /show-in %}} ### Query using the API @@ -683,18 +753,35 @@ Use the `format` parameter to specify the response format: `pretty`, `jsonl`, `p The following example sends an HTTP `GET` request with a URL-encoded SQL query: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash -curl -v "http://{{< influxdb/host >}}/api/v3/query_sql?db=servers&q=select+*+from+cpu+limit+5" +curl -G "http://{{< influxdb/host >}}/api/v3/query_sql" \ + --header 'Authorization: Bearer AUTH_TOKEN' \ + --data-urlencode "db=DATABASE_NAME" \ + --data-urlencode "q=select * from cpu limit 5" ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to query +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to query the specified database{{% /show-in %}} ##### Example: Query passing JSON parameters The following example sends an HTTP `POST` request with parameters in a JSON payload: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash curl http://{{< influxdb/host >}}/api/v3/query_sql \ - --data '{"db": "server", "q": "select * from cpu limit 5"}' + --data '{"db": "DATABASE_NAME", "q": "select * from cpu limit 5"}' ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to query +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to query the specified database{{% /show-in %}} ### Query using the Python client @@ -709,24 +796,35 @@ pip install influxdb3-python From here, you can connect to your database with the client library using just the **host** and **database name: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```python from influxdb_client_3 import InfluxDBClient3 client = InfluxDBClient3( + token='AUTH_TOKEN', host='http://{{< influxdb/host >}}', - database='servers' + database='DATABASE_NAME' ) ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to query +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to query the specified database{{% /show-in %}} The following example shows how to query using SQL, and then -use PyArrow to explore the schema and process results: +use PyArrow to explore the schema and process results. +To authorize the query, the example retrieves the {{% token-link "database" %}} +from the `INFLUXDB3_AUTH_TOKEN` environment variable. ```python from influxdb_client_3 import InfluxDBClient3 +import os client = InfluxDBClient3( + token=os.environ.get('INFLUXDB3_AUTH_TOKEN'), host='http://{{< influxdb/host >}}', - database='servers' ) @@ -776,12 +874,22 @@ visualization of your time series data. {{% product-name %}} supports a **last-n values cache** which stores the last N values in a series or column hierarchy in memory. This gives the database the ability to answer these kinds of queries in under 10 milliseconds. You can use the `influxdb3` CLI to [create a last value cache](/influxdb3/version/reference/cli/influxdb3/create/last_cache/). +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN|TABLE_NAME|CACHE_NAME" %}} ```bash influxdb3 create last_cache \ - -d \ - -t \ - [CACHE_NAME] + --token AUTH_TOKEN + --database DATABASE_NAME \ + --table TABLE_NAME \ + CACHE_NAME ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to create the last values cache in +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: the name of the table to create the last values cache in +- {{% code-placeholder-key %}}`CACHE_NAME`{{% /code-placeholder-key %}}: Optionally, a name for the new cache Consider the following `cpu` sample table: @@ -797,6 +905,7 @@ The following command creates a last value cache named `cpuCache`: ```bash influxdb3 create last_cache \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ --database servers \ --table cpu \ --key-columns host,application \ @@ -808,10 +917,11 @@ _You can create a last values cache per time series, but be mindful of high card #### Query a last values cache -To use the LVC, call it using the `last_cache()` function in your query--for example: +To query data from the LVC, use the [`last_cache()`](influxdb3/version/reference/sql/functions/cache/#last_cache) function in your query--for example: ```bash influxdb3 query \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ --database servers \ "SELECT * FROM last_cache('cpu', 'cpuCache') WHERE host = 'Bravo';" ``` @@ -819,32 +929,53 @@ influxdb3 query \ > [!Note] > #### Only works with SQL > -> The Last values cache only works with SQL, not InfluxQL; SQL is the default language. +> The last values cache only works with SQL, not InfluxQL; SQL is the default language. -#### Delete a Last values cache +#### Delete a last values cache Use the `influxdb3` CLI to [delete a last values cache](/influxdb3/version/reference/cli/influxdb3/delete/last_cache/) +{{% code-placeholders "DATABASE_NAME|TABLE_NAME|CACHE_NAME" %}} ```bash influxdb3 delete last_cache \ - --database \ - --table
\ - --cache-name + --token AUTH_TOKEN \ + --database DATABASE_NAME \ + --table TABLE \ + --cache-name CACHE_NAME ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to delete the last values cache from +- {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: the name of the table to delete the last values cache from +- {{% code-placeholder-key %}}`CACHE_NAME`{{% /code-placeholder-key %}}: the name of the last values cache to delete ### Distinct values cache -Similar to the Last values cache, the database can cache in RAM the distinct values for a single column in a table or a hierarchy of columns. This is useful for fast metadata lookups, which can return in under 30 milliseconds. Many of the options are similar to the last value cache. +Similar to the [last values cache](#last-values-cache), the database can cache in RAM the distinct values for a single column in a table or a hierarchy of columns. +This is useful for fast metadata lookups, which can return in under 30 milliseconds. +Many of the options are similar to the last value cache. You can use the `influxdb3` CLI to [create a distinct values cache](/influxdb3/version/reference/cli/influxdb3/create/distinct_cache/). +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN|TABLE_NAME|CACHE_NAME" %}} ```bash influxdb3 create distinct_cache \ - --database \ - --table
\ - --columns \ - [CACHE_NAME] + --token AUTH_TOKEN \ + --database DATABASE_NAME \ + --table TABLE \ + --columns COLUMNS \ + CACHE_NAME ``` +{{% /code-placeholders %}} +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to create the last values cache in +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: the name of the table to create the distinct values cache in +- {{% code-placeholder-key %}}`CACHE_NAME`{{% /code-placeholder-key %}}: Optionally, a name for the new cache Consider the following `cpu` sample table: @@ -860,6 +991,7 @@ The following command creates a distinct values cache named `cpuDistinctCache`: ```bash influxdb3 create distinct_cache \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ --database servers \ --table cpu \ --columns host,application \ @@ -868,10 +1000,11 @@ influxdb3 create distinct_cache \ #### Query a distinct values cache -To use the distinct values cache, call it using the `distinct_cache()` function in your query--for example: +To query data from the distinct values cache, use the [`distinct_cache()`](/influxdb3/version/reference/sql/functions/cache/#distinct_cache) function in your query--for example: ```bash influxdb3 query \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ --database servers \ "SELECT * FROM distinct_cache('cpu', 'cpuDistinctCache')" ``` @@ -885,19 +1018,28 @@ influxdb3 query \ Use the `influxdb3` CLI to [delete a distinct values cache](/influxdb3/version/reference/cli/influxdb3/delete/distinct_cache/) +{{% code-placeholders "DATABASE_NAME|TABLE_NAME|CACHE_NAME" %}} ```bash influxdb3 delete distinct_cache \ - --database \ - --table
\ - --cache-name + --token AUTH_TOKEN \ + --database DATABASE_NAME \ + --table TABLE \ + --cache-name CACHE_NAME ``` +{{% /code-placeholders %}} -### Python plugins and the Processing engine +Replace the following placeholders with your values: +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to delete the distinct values cache from +- {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: the name of the table to delete the distinct values cache from +- {{% code-placeholder-key %}}`CACHE_NAME`{{% /code-placeholder-key %}}: the name of the distinct values cache to delete -The InfluxDB 3 Processing engine is an embedded Python VM for running code inside the database to process and transform data. +### Python plugins and the processing engine -To activate the Processing engine, pass the `--plugin-dir ` option when starting the {{% product-name %}} server. -`PLUGIN_DIR` is your filesystem location for storing [plugin](#plugin) files for the Processing engine to run. +The InfluxDB 3 processing engine is an embedded Python VM for running code inside the database to process and transform data. + +To activate the processing engine, pass the `--plugin-dir ` option when starting the {{% product-name %}} server. +`PLUGIN_DIR` is your filesystem location for storing [plugin](#plugin) files for the processing engine to run. #### Plugin @@ -1011,13 +1153,24 @@ To test a plugin, do the following: - `--lp` or `--file`: The line protocol to test - Optional: `--input-arguments`: A comma-delimited list of `=` arguments for your plugin code - ```bash - influxdb3 test wal_plugin \ - --lp \ - --input-arguments "arg1=foo,arg2=bar" \ - --database \ - - ``` +{{% code-placeholders "INPUT_LINE_PROTOCOL|INPUT_ARGS|DATABASE_NAME|AUTH_TOKEN|PLUGIN_FILENAME" %}} +```bash +influxdb3 test wal_plugin \ +--lp INPUT_LINE_PROTOCOL \ +--input-arguments INPUT_ARGS \ +--database DATABASE_NAME \ +--token AUTH_TOKEN \ +PLUGIN_FILENAME +``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`INPUT_LINE_PROTOCOL`{{% /code-placeholder-key %}}: the line protocol to test +- Optional: {{% code-placeholder-key %}}`INPUT_ARGS`{{% /code-placeholder-key %}}: a comma-delimited list of `=` arguments for your plugin code--for example, `arg1=hello,arg2=world` +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to test against +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: the {{% token-link "admin" %}} for your {{% product-name %}} server +- {{% code-placeholder-key %}}`PLUGIN_FILENAME`{{% /code-placeholder-key %}}: the name of the plugin file to test The command runs the plugin code with the test data, yields the data to the plugin code, and then responds with the plugin result. You can quickly see how the plugin behaves, what data it would have written to the database, and any errors. @@ -1042,7 +1195,8 @@ trigger: # Test a plugin influxdb3 test wal_plugin \ --lp "my_measure,tag1=asdf f1=1.0 123" \ - --database mydb \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ + --database sensors \ --input-arguments "arg1=hello,arg2=world" \ test.py ``` @@ -1050,7 +1204,8 @@ influxdb3 test wal_plugin \ ```bash # Create a trigger that runs the plugin influxdb3 create trigger \ - -d mydb \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ + --database sensors \ --plugin test_plugin \ --trigger-spec "table:foo" \ --trigger-arguments "arg1=hello,arg2=world" \ @@ -1060,8 +1215,28 @@ influxdb3 create trigger \ After you have created a plugin and trigger, enter the following command to enable the trigger and have it run the plugin as you write data: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN|TRIGGER_NAME" %}} ```bash -influxdb3 enable trigger --database mydb trigger1 +influxdb3 enable trigger \ + --token AUTH_TOKEN \ + --database DATABASE_NAME \ + TRIGGER_NAME +``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to enable the trigger in +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`TRIGGER_NAME`{{% /code-placeholder-key %}}: the name of the trigger to enable + +For example, to enable the trigger named `trigger1` in the `sensors` database: + +```bash +influxdb3 enable trigger \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ + --database sensors + trigger1 ``` For more information, see [Python plugins and the Processing engine](/influxdb3/version/plugins/). diff --git a/content/shared/v3-enterprise-get-started/_index.md b/content/shared/v3-enterprise-get-started/_index.md index 7262570f5..c200058a4 100644 --- a/content/shared/v3-enterprise-get-started/_index.md +++ b/content/shared/v3-enterprise-get-started/_index.md @@ -133,21 +133,20 @@ source ~/.zshrc #### Start InfluxDB -To start your InfluxDB instance, use the `influxdb3 serve` command -and provide the following: +To start your InfluxDB instance, use the `influxdb3 serve` command and provide the following: -- `--object-store`: Specifies the type of Object store to use. + +- `--object-store`: Specifies the type of object store to use. InfluxDB supports the following: local file system (`file`), `memory`, S3 (and compatible services like Ceph or Minio) (`s3`), Google Cloud Storage (`google`), and Azure Blob Storage (`azure`). The default is `file`. Depending on the object store type, you may need to provide additional options for your object store configuration. -- `--cluster-id`: A string identifier that determines part of the storage path hierarchy. All nodes within the same cluster share this identifier. The storage path follows the pattern `//`. In a multi-node setup, this ID is used to reference the entire cluster. - `--node-id`: A string identifier that distinguishes individual server instances within the cluster. This forms the final part of the storage path: `//`. In a multi-node setup, this ID is used to reference specific nodes. +- `--cluster-id`: A string identifier that determines part of the storage path hierarchy. All nodes within the same cluster share this identifier. The storage path follows the pattern `//`. In a multi-node setup, this ID is used to reference the entire cluster. -> [!Note] -> The combined path structure `//` ensures proper organization of data in your object store, allowing for clean separation between clusters and individual nodes. +The following examples show how to start {{% product-name %}} with different object store configurations. > [!Note] > #### Diskless architecture @@ -156,16 +155,15 @@ and provide the following: > storage alone, eliminating the need for locally attached disks. > {{% product-name %}} can also work with only local disk storage when needed. -The following examples show how to start InfluxDB 3 with different object store configurations: +> [!Note] +> The combined path structure `//` ensures proper organization of data in your object store, allowing for clean separation between clusters and individual nodes. -```bash -# Memory object store -# Stores data in RAM; doesn't persist data -influxdb3 serve \ ---node-id host01 \ ---cluster-id cluster01 \ ---object-store memory -``` +##### Filesystem object store + +Store data in a specified directory on the local filesystem. +This is the default object store type. + +Replace the following with your values: ```bash # Filesystem object store @@ -177,15 +175,12 @@ influxdb3 serve \ --data-dir ~/.influxdb3 ``` -To run the [Docker image](/influxdb3/enterprise/install/#docker-image) and persist data to the filesystem, mount a volume for the Object store-for example, pass the following options: +To run the [Docker image](/influxdb3/version/install/#docker-image) and persist data to the filesystem, mount a volume for the object store-for example, pass the following options: - `-v /path/on/host:/path/in/container`: Mounts a directory from your filesystem to the container - `--object-store file --data-dir /path/in/container`: Uses the mount for server storage -> [!Note] -> -> The {{% product-name %}} Docker image exposes port `8181`, the `influxdb3` server default for HTTP connections. -> To map the exposed port to a different port when running a container, see the Docker guide for [Publishing and exposing ports](https://docs.docker.com/get-started/docker-concepts/running-containers/publishing-ports/). + ```bash @@ -201,15 +196,28 @@ docker run -it \ --data-dir /path/in/container ``` + + +> [!Note] +> +> The {{% product-name %}} Docker image exposes port `8181`, the `influxdb3` server default for HTTP connections. +> To map the exposed port to a different port when running a container, see the Docker guide for [Publishing and exposing ports](https://docs.docker.com/get-started/docker-concepts/running-containers/publishing-ports/). + +##### S3 object store + +Store data in an S3-compatible object store. +This is useful for production deployments that require high availability and durability. +Provide your bucket name and credentials to access the S3 object store. + ```bash # S3 object store (default is the us-east-1 region) -# Specify the Object store type and associated options +# Specify the object store type and associated options influxdb3 serve \ --node-id host01 \ --cluster-id cluster01 \ --object-store s3 \ - --bucket BUCKET \ - --aws-access-key-id AWS_ACCESS_KEY_ID \ + --bucket OBJECT_STORE_BUCKET \ + --aws-access-key AWS_ACCESS_KEY_ID \ --aws-secret-access-key AWS_SECRET_ACCESS_KEY ``` @@ -221,14 +229,28 @@ influxdb3 serve \ --node-id host01 \ --cluster-id cluster01 \ --object-store s3 \ - --bucket BUCKET \ + --bucket OBJECT_STORE_BUCKET \ --aws-access-key-id AWS_ACCESS_KEY_ID \ --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ --aws-endpoint ENDPOINT \ --aws-allow-http ``` -For more information about server options, use the CLI help: +#### Memory object store + +Store data in RAM without persisting it on shutdown. +It's useful for rapid testing and development. + +```bash +# Memory object store +# Stores data in RAM; doesn't persist data +influxdb3 serve \ +--node-id host01 \ +--cluster-id cluster01 \ +--object-store memory +``` + +For more information about server options, use the CLI help or view the [InfluxDB 3 CLI reference](/influxdb3/version/reference/cli/serve/): ```bash influxdb3 serve --help @@ -236,7 +258,7 @@ influxdb3 serve --help #### Licensing -When first starting a new instance, InfluxDB prompts you to select a license type. +When first starting a new instance, {{% product-name %}} prompts you to select a license type. InfluxDB 3 Enterprise licenses authorize the use of the InfluxDB 3 Enterprise software and apply to a single cluster. Licenses are primarily based on the number of CPUs InfluxDB can use, but there are other limitations depending on the license type. The following InfluxDB 3 Enterprise license types are available: @@ -250,7 +272,7 @@ You can learn more on managing your InfluxDB 3 Enterprise license on the [Manage After you have [started the server](#start-influxdb), you can create and manage tokens using the `influxdb3` CLI or the HTTP API. {{% product-name %}} uses token-based authentication and authorization which is enabled by default when you start the server. -With authentication enabled, you must provide a token to access server actions. +With authentication enabled, you must provide a token with `influxdb3` CLI commands and HTTP API requests. {{% product-name %}} supports the following types of tokens: - **admin token**: Grants access to all CLI actions and API endpoints. A server can have one admin token. @@ -312,11 +334,12 @@ Replace {{% code-placeholder-key %}}`CONTAINER_NAME`{{% /code-placeholder-key %} {{< /code-tabs-wrapper >}} The command returns a token string that you can use to authenticate CLI commands and API requests. +Store your token in a secure location, as you cannot retrieve it from the database later. + +For more information about tokens, see how to [Manage admin tokens](/influxdb3/version/admin/tokens/admin/). After you have created an admin token, you can use it to create database tokens and system tokens. -For more information, see how to [Manage admin tokens](/influxdb3/version/admin/tokens/admin/). - #### Create a database token To create a database token, use the `influxdb3 create token` subcommand and pass the following: @@ -394,21 +417,59 @@ For more information, see how to [Manage resource tokens](/influxdb3/version/adm #### Use tokens to authorize CLI commands and API requests -- To authenticate `influxdb3` CLI commands, use the `--token` option or assign your - token to the `INFLUXDB3_AUTH_TOKEN` environment variable for `influxdb3` to use it automatically. -- To authenticate HTTP API requests, include `Bearer ` in the `Authorization` header value--for example: +#### Use tokens to authorize CLI commands and API requests -{{% code-placeholders "SYSTEM_TOKEN" %}} +With authentication enabled (the default), {{% product-name %}} requires a +token for all `influxdb3` CLI commands and HTTP API requests. - ```bash - curl "http://{{< influxdb/host >}}/health" \ - --header "Authorization: Bearer SYSTEM_TOKEN" - ``` +In the following examples, replace {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}} with your {{% token-link "admin" %}} string. + +{{< tabs-wrapper >}} +{{% tabs %}} +[CLI](#cli-use-a-token) +[HTTP API](#api-use-a-token) +{{% /tabs %}} +{{% tab-content %}} +For `influxdb3` to use your token automatically, assign it your + token to the `INFLUXDB3_AUTH_TOKEN` environment variable: + +{{% code-placeholders "AUTH_TOKEN" %}} +```bash +# Set the environment variable for future CLI commands +export INFLUXDB3_AUTH_TOKEN=AUTH_TOKEN +``` {{% /code-placeholders %}} -Replace the following: +Or to authenticate a single `influxdb3` CLI command and override `$INFLUXDB3_AUTH_TOKEN`, include the `--token` option: -In your command, replace {{% code-placeholder-key %}}`SYSTEM_TOKEN`{{% /code-placeholder-key %}}: System token that grants access to system endpoints (`/health`, `/metrics`, etc.) +{{% code-placeholders "AUTH_TOKEN" %}} +```bash +# Use the --token option for a single command +influxdb3 show databases --token AUTH_TOKEN +``` +{{% /code-placeholders %}} +{{% /tab-content %}} +{{% tab-content %}} +To authenticate HTTP API requests, include `Bearer ` in the `Authorization` header value: + +{{% code-placeholders "AUTH_TOKEN" %}} +```bash +# Include the token in the Authorization HTTP request header +curl "http://{{< influxdb/host >}}/api/v3/configure/database" \ + --header "Authorization: Bearer AUTH_TOKEN" +``` +{{% /code-placeholders %}} + +{{% code-placeholders "SYSTEM_TOKEN" %}} +```bash +curl "http://{{< influxdb/host >}}/health" \ + --header "Authorization: Bearer SYSTEM_TOKEN" +``` +{{% /code-placeholders %}} + +Replace {{% code-placeholder-key %}}`SYSTEM_TOKEN`{{% /code-placeholder-key %}} with the system token string that grants access to system endpoints (`/health`, `/metrics`) +{{% /tab-content %}} +{{< /tabs-wrapper >}} ### Data model @@ -504,11 +565,11 @@ In the code samples, replace the following placeholders with your values: Pass data as quoted line protocol via standard input (stdin)--for example: -{{% code-placeholders "DATABASE_NAME|TOKEN" %}} +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash influxdb3 write \ --database DATABASE_NAME \ - --token TOKEN \ + --token AUTH_TOKEN \ --precision ns \ --accept-partial \ 'cpu,host=Alpha,region=us-west,application=webserver val=1i,usage_percent=20.5,status="OK" @@ -526,17 +587,21 @@ Pass the `--file` option to write line protocol you have saved to a file--for ex [sample line protocol](#write-data-in-line-protocol-syntax) to a file named `server_data` and then enter the following command: -{{% code-placeholders "DATABASE_NAME|TOKEN" %}} +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash influxdb3 write \ --database DATABASE_NAME \ - --token TOKEN \ + --token AUTH_TOKEN \ --precision ns \ --accept-partial \ - --file server_data + --file path/to/server_data ``` {{% /code-placeholders %}} +Replace the following placeholders with your values: +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the [database](/influxdb3/version/admin/databases/) to write to. +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to write to the specified database{{% /show-in %}} + ### Write data using the HTTP API {{% product-name %}} provides three write API endpoints that respond to HTTP `POST` requests. @@ -568,7 +633,7 @@ and supports the following parameters: - `?precision=`: Specify the precision of the timestamp. The default is nanosecond precision. - request body: The line protocol data to write. -For more information about the parameters, see [Write data](/influxdb3/core/write-data/). +For more information about the parameters, see [Write data](/influxdb3/version/write-data/). ##### Example: write data using the /api/v3 HTTP API @@ -581,6 +646,7 @@ With `accept_partial=true` (default): ```bash curl -v "http://{{< influxdb/host >}}/api/v3/write_lp?db=sensors&precision=auto" \ + --header 'Authorization: Bearer apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0==' \ --data-raw 'home,room=Sunroom temp=96 home,room=Sunroom temp="hi"' ``` @@ -611,6 +677,7 @@ With `accept_partial=false`: ```bash curl -v "http://{{< influxdb/host >}}/api/v3/write_lp?db=sensors&precision=auto&accept_partial=false" \ + --header 'Authorization: Bearer apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0==' \ --data-raw 'home,room=Sunroom temp=96 home,room=Sunroom temp="hi"' ``` @@ -664,17 +731,17 @@ The `/write` InfluxDB v1 compatibility endpoint provides backwards compatibility #### Write responses -By default, InfluxDB acknowledges writes after flushing the WAL file to the Object store (occurring every second). +By default, InfluxDB acknowledges writes after flushing the WAL file to the object store (occurring every second). For high write throughput, you can send multiple concurrent write requests. #### Use no_sync for immediate write responses To reduce the latency of writes, use the `no_sync` write option, which acknowledges writes _before_ WAL persistence completes. -When `no_sync=true`, InfluxDB validates the data, writes the data to the WAL, and then immediately responds to the client, without waiting for persistence to the Object store. +When `no_sync=true`, InfluxDB validates the data, writes the data to the WAL, and then immediately responds to the client, without waiting for persistence to the object store. Using `no_sync=true` is best when prioritizing high-throughput writes over absolute durability. -- Default behavior (`no_sync=false`): Waits for data to be written to the Object store before acknowledging the write. Reduces the risk of data loss, but increases the latency of the response. +- Default behavior (`no_sync=false`): Waits for data to be written to the object store before acknowledging the write. Reduces the risk of data loss, but increases the latency of the response. - With `no_sync=true`: Reduces write latency, but increases the risk of data loss in case of a crash before WAL persistence. ##### Immediate write using the HTTP API @@ -683,30 +750,27 @@ The `no_sync` parameter controls when writes are acknowledged--for example: ```bash curl "http://{{< influxdb/host >}}/api/v3/write_lp?db=sensors&precision=auto&no_sync=true" \ + --header 'Authorization: Bearer apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0==' \ --data-raw "home,room=Sunroom temp=96" ``` -##### Immediate write using the influxdb3 CLI - -The `no_sync` CLI option controls when writes are acknowledged--for example: - -```bash -influxdb3 write \ - --bucket mydb \ - --org my_org \ - --token my-token \ - --no-sync -``` - ### Create a database or table To create a database without writing data, use the `create` subcommand--for example: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash -influxdb3 create database mydb +influxdb3 create database DATABASE_NAME \ + --token AUTH_TOKEN ``` +{{% /code-placeholders %}} -To learn more about a subcommand, use the `-h, --help` flag: +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to create +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: the {{% token-link "admin" %}} for your {{% product-name %}} server + +To learn more about a subcommand, use the `-h, --help` flag or view the [InfluxDB 3 CLI reference](/influxdb3/version/reference/cli/create/): ```bash influxdb3 create -h @@ -774,12 +838,20 @@ $ influxdb3 query --database servers "SELECT DISTINCT usage_percent, time FROM c To query using InfluxQL, enter the `influxdb3 query` subcommand and specify `influxql` in the language option--for example: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash influxdb3 query \ - --database servers \ + --database DATABASE_NAME \ + --token \ --language influxql \ "SELECT DISTINCT usage_percent FROM cpu WHERE time >= now() - 1d" ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to query +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to query the specified database{{% /show-in %}} ### Query using the API @@ -795,18 +867,35 @@ Use the `format` parameter to specify the response format: `pretty`, `jsonl`, `p The following example sends an HTTP `GET` request with a URL-encoded SQL query: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash -curl -v "http://{{< influxdb/host >}}/api/v3/query_sql?db=servers&q=select+*+from+cpu+limit+5" +curl -G "http://{{< influxdb/host >}}/api/v3/query_sql" \ + --header 'Authorization: Bearer AUTH_TOKEN' \ + --data-urlencode "db=DATABASE_NAME" \ + --data-urlencode "q=select * from cpu limit 5" ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to query +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to query the specified database{{% /show-in %}} ##### Example: Query passing JSON parameters The following example sends an HTTP `POST` request with parameters in a JSON payload: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```bash curl http://{{< influxdb/host >}}/api/v3/query_sql \ - --data '{"db": "server", "q": "select * from cpu limit 5"}' + --data '{"db": "DATABASE_NAME", "q": "select * from cpu limit 5"}' ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to query +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to query the specified database{{% /show-in %}} ### Query using the Python client @@ -821,24 +910,35 @@ pip install influxdb3-python From here, you can connect to your database with the client library using just the **host** and **database name: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN" %}} ```python from influxdb_client_3 import InfluxDBClient3 client = InfluxDBClient3( + token='AUTH_TOKEN', host='http://{{< influxdb/host >}}', - database='servers' + database='DATABASE_NAME' ) ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to query +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to query the specified database{{% /show-in %}} The following example shows how to query using SQL, and then -use PyArrow to explore the schema and process results: +use PyArrow to explore the schema and process results. +To authorize the query, the example retrieves the {{% token-link "database" %}} +from the `INFLUXDB3_AUTH_TOKEN` environment variable. ```python from influxdb_client_3 import InfluxDBClient3 +import os client = InfluxDBClient3( + token=os.environ.get('INFLUXDB3_AUTH_TOKEN'), host='http://{{< influxdb/host >}}', - database='servers' ) @@ -883,18 +983,27 @@ Set your expected database connection details on the Settings page. From there, you can query data, browser your database schema, and do basic visualization of your time series data. - ### Last values cache {{% product-name %}} supports a **last-n values cache** which stores the last N values in a series or column hierarchy in memory. This gives the database the ability to answer these kinds of queries in under 10 milliseconds. -You can use the `influxdb3` CLI to create a last value cache. +You can use the `influxdb3` CLI to [create a last value cache](/influxdb3/version/reference/cli/influxdb3/create/last_cache/). +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN|TABLE_NAME|CACHE_NAME" %}} ```bash influxdb3 create last_cache \ - -d \ - -t
\ - [CACHE_NAME] + --token AUTH_TOKEN + --database DATABASE_NAME \ + --table TABLE_NAME \ + CACHE_NAME ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to create the last values cache in +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: the name of the table to create the last values cache in +- {{% code-placeholder-key %}}`CACHE_NAME`{{% /code-placeholder-key %}}: Optionally, a name for the new cache Consider the following `cpu` sample table: @@ -910,6 +1019,7 @@ The following command creates a last value cache named `cpuCache`: ```bash influxdb3 create last_cache \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ --database servers \ --table cpu \ --key-columns host,application \ @@ -921,10 +1031,11 @@ _You can create a last values cache per time series, but be mindful of high card #### Query a last values cache -To use the LVC, call it using the `last_cache()` function in your query--for example: +To query data from the LVC, use the [`last_cache()`](influxdb3/version/reference/sql/functions/cache/#last_cache) function in your query--for example: ```bash influxdb3 query \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ --database servers \ "SELECT * FROM last_cache('cpu', 'cpuCache') WHERE host = 'Bravo';" ``` @@ -932,32 +1043,53 @@ influxdb3 query \ > [!Note] > #### Only works with SQL > -> The Last values cache only works with SQL, not InfluxQL; SQL is the default language. +> The last values cache only works with SQL, not InfluxQL; SQL is the default language. -#### Delete a Last values cache +#### Delete a last values cache Use the `influxdb3` CLI to [delete a last values cache](/influxdb3/version/reference/cli/influxdb3/delete/last_cache/) +{{% code-placeholders "DATABASE_NAME|TABLE_NAME|CACHE_NAME" %}} ```bash influxdb3 delete last_cache \ - --database \ - --table
\ - --cache-name + --token AUTH_TOKEN \ + --database DATABASE_NAME \ + --table TABLE \ + --cache-name CACHE_NAME ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to delete the last values cache from +- {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: the name of the table to delete the last values cache from +- {{% code-placeholder-key %}}`CACHE_NAME`{{% /code-placeholder-key %}}: the name of the last values cache to delete ### Distinct values cache -Similar to the Last values cache, the database can cache in RAM the distinct values for a single column in a table or a hierarchy of columns. This is useful for fast metadata lookups, which can return in under 30 milliseconds. Many of the options are similar to the last value cache. +Similar to the [last values cache](#last-values-cache), the database can cache in RAM the distinct values for a single column in a table or a hierarchy of columns. +This is useful for fast metadata lookups, which can return in under 30 milliseconds. +Many of the options are similar to the last value cache. You can use the `influxdb3` CLI to [create a distinct values cache](/influxdb3/version/reference/cli/influxdb3/create/distinct_cache/). +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN|TABLE_NAME|CACHE_NAME" %}} ```bash influxdb3 create distinct_cache \ - --database \ - --table
\ - --columns \ - [CACHE_NAME] + --token AUTH_TOKEN \ + --database DATABASE_NAME \ + --table TABLE \ + --columns COLUMNS \ + CACHE_NAME ``` +{{% /code-placeholders %}} +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to create the last values cache in +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: the name of the table to create the distinct values cache in +- {{% code-placeholder-key %}}`CACHE_NAME`{{% /code-placeholder-key %}}: Optionally, a name for the new cache Consider the following `cpu` sample table: @@ -973,6 +1105,7 @@ The following command creates a distinct values cache named `cpuDistinctCache`: ```bash influxdb3 create distinct_cache \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ --database servers \ --table cpu \ --columns host,application \ @@ -985,6 +1118,7 @@ To use the distinct values cache, call it using the `distinct_cache()` function ```bash influxdb3 query \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ --database servers \ "SELECT * FROM distinct_cache('cpu', 'cpuDistinctCache')" ``` @@ -998,19 +1132,28 @@ influxdb3 query \ Use the `influxdb3` CLI to [delete a distinct values cache](/influxdb3/version/reference/cli/influxdb3/delete/distinct_cache/) +{{% code-placeholders "DATABASE_NAME|TABLE_NAME|CACHE_NAME" %}} ```bash influxdb3 delete distinct_cache \ - --database \ - --table
\ - --cache-name + --token AUTH_TOKEN \ + --database DATABASE_NAME \ + --table TABLE \ + --cache-name CACHE_NAME ``` +{{% /code-placeholders %}} -### Python plugins and the Processing engine +Replace the following placeholders with your values: +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to delete the distinct values cache from +- {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: the name of the table to delete the distinct values cache from +- {{% code-placeholder-key %}}`CACHE_NAME`{{% /code-placeholder-key %}}: the name of the distinct values cache to delete -The InfluxDB 3 Processing engine is an embedded Python VM for running code inside the database to process and transform data. +### Python plugins and the processing engine -To activate the Processing engine, pass the `--plugin-dir ` option when starting the {{% product-name %}} server. -`PLUGIN_DIR` is your filesystem location for storing [plugin](#plugin) files for the Processing engine to run. +The InfluxDB 3 processing engine is an embedded Python VM for running code inside the database to process and transform data. + +To activate the processing engine, pass the `--plugin-dir ` option when starting the {{% product-name %}} server. +`PLUGIN_DIR` is your filesystem location for storing [plugin](#plugin) files for the processing engine to run. #### Plugin @@ -1027,7 +1170,7 @@ InfluxDB 3 provides the following types of triggers, each with specific trigger- - **On WAL flush**: Sends a batch of written data (for a specific table or all tables) to a plugin (by default, every second). - **On Schedule**: Executes a plugin on a user-configured schedule (using a crontab or a duration); useful for data collection and deadman monitoring. -- **On Request**: Binds a plugin to a custom HTTP API endpoint at `/api/v3/engine/`. +- **On Request**: Binds a plugin to a custom HTTP API endpoint at `/api/v3/engine/`. The plugin receives the HTTP request headers and content, and can then parse, process, and send the data into the database or to third-party services. ### Test, create, and trigger plugin code @@ -1124,13 +1267,24 @@ To test a plugin, do the following: - `--lp` or `--file`: The line protocol to test - Optional: `--input-arguments`: A comma-delimited list of `=` arguments for your plugin code - ```bash - influxdb3 test wal_plugin \ - --lp \ - --input-arguments "arg1=foo,arg2=bar" \ - --database \ - - ``` +{{% code-placeholders "INPUT_LINE_PROTOCOL|INPUT_ARGS|DATABASE_NAME|AUTH_TOKEN|PLUGIN_FILENAME" %}} +```bash +influxdb3 test wal_plugin \ +--lp INPUT_LINE_PROTOCOL \ +--input-arguments INPUT_ARGS \ +--database DATABASE_NAME \ +--token AUTH_TOKEN \ +PLUGIN_FILENAME +``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`INPUT_LINE_PROTOCOL`{{% /code-placeholder-key %}}: the line protocol to test +- Optional: {{% code-placeholder-key %}}`INPUT_ARGS`{{% /code-placeholder-key %}}: a comma-delimited list of `=` arguments for your plugin code--for example, `arg1=hello,arg2=world` +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to test against +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: the {{% token-link "admin" %}} for your {{% product-name %}} server +- {{% code-placeholder-key %}}`PLUGIN_FILENAME`{{% /code-placeholder-key %}}: the name of the plugin file to test The command runs the plugin code with the test data, yields the data to the plugin code, and then responds with the plugin result. You can quickly see how the plugin behaves, what data it would have written to the database, and any errors. @@ -1155,7 +1309,8 @@ trigger: # Test a plugin influxdb3 test wal_plugin \ --lp "my_measure,tag1=asdf f1=1.0 123" \ - --database mydb \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ + --database sensors \ --input-arguments "arg1=hello,arg2=world" \ test.py ``` @@ -1163,7 +1318,8 @@ influxdb3 test wal_plugin \ ```bash # Create a trigger that runs the plugin influxdb3 create trigger \ - -d mydb \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ + --database sensors \ --plugin test_plugin \ --trigger-spec "table:foo" \ --trigger-arguments "arg1=hello,arg2=world" \ @@ -1173,8 +1329,28 @@ influxdb3 create trigger \ After you have created a plugin and trigger, enter the following command to enable the trigger and have it run the plugin as you write data: +{{% code-placeholders "DATABASE_NAME|AUTH_TOKEN|TRIGGER_NAME" %}} ```bash -influxdb3 enable trigger --database mydb trigger1 +influxdb3 enable trigger \ + --token AUTH_TOKEN \ + --database DATABASE_NAME \ + TRIGGER_NAME +``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to enable the trigger in +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`TRIGGER_NAME`{{% /code-placeholder-key %}}: the name of the trigger to enable + +For example, to enable the trigger named `trigger1` in the `sensors` database: + +```bash +influxdb3 enable trigger \ + --token apiv3_0xxx0o0XxXxx00Xxxx000xXXxoo0== \ + --database sensors + trigger1 ``` For more information, see [Python plugins and the Processing engine](/influxdb3/version/plugins/). @@ -1225,6 +1401,7 @@ influxdb3 serve \ --http-bind {{< influxdb/host >}} \ --aws-access-key-id \ --aws-secret-access-key +``` ```bash ## NODE 2 @@ -1241,8 +1418,8 @@ influxdb3 serve \ --object-store s3 \ --bucket influxdb-3-enterprise-storage \ --http-bind localhost:8282 \ - --aws-access-key-id \ - --aws-secret-access-key + --aws-access-key-id AWS_ACCESS_KEY_ID \ + --aws-secret-access-key AWS_SECRET_ACCESS_KEY ``` After the nodes have started, querying either node returns data for both nodes, and _NODE 1_ runs compaction. @@ -1442,28 +1619,43 @@ You can use the default port `8181` for any write or query, without changing any > > When running multiple local instances for testing or separate nodes in production, specifying the host ensures writes and queries are routed to the correct instance. +{{% code-placeholders "(http://localhost:8585)|AUTH_TOKEN|DATABASE_NAME|QUERY" %}} ```bash -# Example variables on a query +# Example querying a specific host # HTTP-bound Port: 8585 -influxdb3 query http://localhost:8585 --database "" +influxdb3 query \ + --host http://localhost:8585 + --token AUTH_TOKEN \ + --database DATABASE_NAME "QUERY" ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`http://localhost:8585`{{% /code-placeholder-key %}}: the host and port of the node to query +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "database" %}}{{% show-in "enterprise" %}} with permission to query the specified database{{% /show-in %}} +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to query +- {{% code-placeholder-key %}}`QUERY`{{% /code-placeholder-key %}}: the SQL or InfluxQL query to run against the database ### File index settings To accelerate performance on specific queries, you can define non-primary keys to index on, which helps improve performance for single-series queries. -This feature is only available in Enterprise and is not available in Core. +This feature is only available in {{% product-name %}} and is not available in Core. #### Create a file index +{{% code-placeholders "AUTH_TOKEN|DATABASE|TABLE|COLUMNS" %}} + ```bash # Example variables on a query # HTTP-bound Port: 8585 influxdb3 create file_index \ --host http://localhost:8585 \ - --database \ - --table
\ - + --token AUTH_TOKEN \ + --database DATABASE_NAME \ + --table TABLE_NAME \ + COLUMNS ``` #### Delete a file index @@ -1471,6 +1663,14 @@ influxdb3 create file_index \ ```bash influxdb3 delete file_index \ --host http://localhost:8585 \ - --database \ - --table
\ + --database DATABASE_NAME \ + --table TABLE_NAME \ ``` +{{% /code-placeholders %}} + +Replace the following placeholders with your values: + +- {{% code-placeholder-key %}}`AUTH_TOKEN`{{% /code-placeholder-key %}}: your {{% token-link "admin" %}} +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of the database to create the file index in +- {{% code-placeholder-key %}}`TABLE_NAME`{{% /code-placeholder-key %}}: the name of the table to create the file index in +- {{% code-placeholder-key %}}`COLUMNS`{{% /code-placeholder-key %}}: a comma-separated list of columns to index on, for example, `host,application` \ No newline at end of file From dc533920f1b58409bd323e04f19e7d7d37be9a40 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Tue, 27 May 2025 15:15:54 -0500 Subject: [PATCH 3/3] add information about column order in monolith (#6094) --- .../best-practices/optimize-writes.md | 3 +- .../best-practices/schema-design.md | 3 +- .../best-practices/optimize-writes.md | 3 +- .../best-practices/schema-design.md | 3 +- .../best-practices/optimize-writes.md | 30 ++++++++++++++++++- .../best-practices/schema-design.md | 25 ++++++++++++++++ 6 files changed, 62 insertions(+), 5 deletions(-) diff --git a/content/influxdb3/core/write-data/best-practices/optimize-writes.md b/content/influxdb3/core/write-data/best-practices/optimize-writes.md index fb4b43eb1..be2bb6b5f 100644 --- a/content/influxdb3/core/write-data/best-practices/optimize-writes.md +++ b/content/influxdb3/core/write-data/best-practices/optimize-writes.md @@ -16,5 +16,6 @@ source: /shared/influxdb3-write-guides/best-practices/optimize-writes.md --- diff --git a/content/influxdb3/core/write-data/best-practices/schema-design.md b/content/influxdb3/core/write-data/best-practices/schema-design.md index f6b24a7e6..4cbb8a9ec 100644 --- a/content/influxdb3/core/write-data/best-practices/schema-design.md +++ b/content/influxdb3/core/write-data/best-practices/schema-design.md @@ -16,5 +16,6 @@ source: /shared/influxdb3-write-guides/best-practices/schema-design.md --- diff --git a/content/influxdb3/enterprise/write-data/best-practices/optimize-writes.md b/content/influxdb3/enterprise/write-data/best-practices/optimize-writes.md index 42966bfa2..88847adeb 100644 --- a/content/influxdb3/enterprise/write-data/best-practices/optimize-writes.md +++ b/content/influxdb3/enterprise/write-data/best-practices/optimize-writes.md @@ -16,5 +16,6 @@ source: /shared/influxdb3-write-guides/best-practices/optimize-writes.md --- diff --git a/content/influxdb3/enterprise/write-data/best-practices/schema-design.md b/content/influxdb3/enterprise/write-data/best-practices/schema-design.md index e9451b2f3..ed7c9d4b2 100644 --- a/content/influxdb3/enterprise/write-data/best-practices/schema-design.md +++ b/content/influxdb3/enterprise/write-data/best-practices/schema-design.md @@ -16,5 +16,6 @@ source: /shared/influxdb3-write-guides/best-practices/schema-design.md --- diff --git a/content/shared/influxdb3-write-guides/best-practices/optimize-writes.md b/content/shared/influxdb3-write-guides/best-practices/optimize-writes.md index aad783214..6abfdd5c7 100644 --- a/content/shared/influxdb3-write-guides/best-practices/optimize-writes.md +++ b/content/shared/influxdb3-write-guides/best-practices/optimize-writes.md @@ -3,7 +3,8 @@ Use these tips to optimize performance and system overhead when writing data to {{< product-name >}}. - [Batch writes](#batch-writes) -- [Sort tags by key](#sort-tags-by-key) +{{% hide-in "enterprise,core" %}}- [Sort tags by key](#sort-tags-by-key){{% /hide-in %}} +{{% show-in "enterprise,core" %}}- [On first write, sort tags by query priority](#on-first-write-sort-tags-by-query-priority){{% /show-in %}} - [Use the coarsest time precision possible](#use-the-coarsest-time-precision-possible) - [Use gzip compression](#use-gzip-compression) - [Enable gzip compression in Telegraf](#enable-gzip-compression-in-telegraf) @@ -34,6 +35,8 @@ Write data in batches to minimize network overhead when writing data to InfluxDB > The optimal batch size is 10,000 lines of line protocol or 10 MBs, whichever > threshold is met first. +{{% hide-in "enterprise,core" %}} + ## Sort tags by key Before writing data points to InfluxDB, sort tags by key in lexicographic order. @@ -49,6 +52,31 @@ measurement,tagC=therefore,tagE=am,tagA=i,tagD=i,tagB=think fieldKey=fieldValue measurement,tagA=i,tagB=think,tagC=therefore,tagD=i,tagE=am fieldKey=fieldValue 1562020262 ``` +{{% /hide-in %}} + +{{% show-in "enterprise,core" %}} + +## On first write, sort tags by query priority + +The first write to a table in {{% product-name %}} determines the physical column +order in storage, and that order has a direct impact on query performance. +Columns that appear earlier are typically faster to filter and access during +query execution. + +Sort your tags by query priority when performing the initial write to a table. +Place the most commonly queried tags first—those you frequently use in `WHERE` +clauses or joins—followed by less frequently queried ones. For example, if most +of your queries filter by `region` and then by `host`, structure your first +write so that `region` comes before `host`. + +> [!Important] +> Column order is determined on the first write and cannot be changed afterward. +> Tags added after the first write are added last in the column sort order. +> Plan your schema with your query workload in mind to ensure the best long-term +> performance. + +{{% /show-in %}} + ## Use the coarsest time precision possible {{< product-name >}} supports up to nanosecond timestamp precision. However, diff --git a/content/shared/influxdb3-write-guides/best-practices/schema-design.md b/content/shared/influxdb3-write-guides/best-practices/schema-design.md index d17ad27fb..5d6dc40f3 100644 --- a/content/shared/influxdb3-write-guides/best-practices/schema-design.md +++ b/content/shared/influxdb3-write-guides/best-practices/schema-design.md @@ -9,6 +9,7 @@ for simpler and more performant queries. - [Do not use duplicate names for tags and fields](#do-not-use-duplicate-names-for-tags-and-fields) - [Maximum number of columns per table](#maximum-number-of-columns-per-table) - [Design for performance](#design-for-performance) + {{% show-in "enterprise,core" %}}- [Sort tags by query priority](#sort-tags-by-query-priority){{% /show-in %}} - [Avoid wide schemas](#avoid-wide-schemas) - [Avoid sparse schemas](#avoid-sparse-schemas) - [Table schemas should be homogenous](#table-schemas-should-be-homogenous) @@ -135,11 +136,35 @@ the performance of queries against that table. The following guidelines help to optimize query performance: +{{% show-in "enterprise,core" %}}- [Sort tags by query priority](#sort-tags-by-query-priority){{% /show-in %}} - [Avoid wide schemas](#avoid-wide-schemas) - [Avoid sparse schemas](#avoid-sparse-schemas) - [Table schemas should be homogenous](#table-schemas-should-be-homogenous) - [Use the best data type for your data](#use-the-best-data-type-for-your-data) +{{% show-in "enterprise,core" %}} + +### Sort tags by query priority + +The first write to a table in {{% product-name %}} determines the physical column +order in storage, and that order has a direct impact on query performance. +Columns that appear earlier are typically faster to filter and access during +query execution. + +Sort your tags by query priority when performing the initial write to a table. +Place the most commonly queried tags first—those you frequently use in `WHERE` +clauses or joins—followed by less frequently queried ones. For example, if most +of your queries filter by `region` and then by `host`, structure your first +write so that `region` comes before `host`. + +> [!Important] +> Column order is determined on the first write and cannot be changed afterward. +> Tags added after the first write are added last in the column sort order. +> Plan your schema with your query workload in mind to ensure the best long-term +> performance. + +{{% /show-in %}} + ### Avoid wide schemas A wide schema refers to a schema with a large number of columns (tags and fields).