From ca24b3be54df8db7ea27797882c92ad3b25b58e0 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Mon, 20 Oct 2025 10:17:56 -0500 Subject: [PATCH] Influxdb3 config options (#6419) * docs(cli): fix global vs serve-specific flag documentation - Remove --verbose from global flags (it's serve-specific) - Document --num-io-threads as global-only flag - Add clear examples showing correct flag positioning - Update serve.md files with global flag usage notes - Fix config-options.md to separate Core/Enterprise examples Resolves incorrect CLI usage patterns that would cause errors. Global flags must go before 'serve', serve-specific flags go after. * docs(cli): remove Tokio runtime options from CLI index pages - Remove detailed Tokio runtime options tables from CLI index pages - Replace with simplified global options and link to config-options - Add examples showing correct global flag positioning - Fix --verbose usage to be serve-specific (after serve command) - Add --num-io-threads example as global flag (before serve command) These detailed options are now documented in config-options.md with proper global vs serve-specific categorization. --- .../core/reference/cli/influxdb3/_index.md | 51 ++-- .../core/reference/cli/influxdb3/serve.md | 13 +- .../reference/cli/influxdb3/_index.md | 52 ++-- .../reference/cli/influxdb3/serve.md | 14 +- .../shared/influxdb3-cli/config-options.md | 259 ++++++++++-------- 5 files changed, 212 insertions(+), 177 deletions(-) diff --git a/content/influxdb3/core/reference/cli/influxdb3/_index.md b/content/influxdb3/core/reference/cli/influxdb3/_index.md index b0788e001..495a03f3d 100644 --- a/content/influxdb3/core/reference/cli/influxdb3/_index.md +++ b/content/influxdb3/core/reference/cli/influxdb3/_index.md @@ -37,36 +37,13 @@ influxdb3 [GLOBAL-OPTIONS] [COMMAND] ## Global options -| Option | | Description | -| :----- | :------------------------------------ | :------------------------------------------------------------------------------------------------ | -| | `--num-threads` | Maximum number of IO runtime threads to use | -| | `--io-runtime-type` | IO tokio runtime type (`current-thread`, `multi-thread` _(default)_, or `multi-thread-alt`) | -| | `--io-runtime-disable-lifo-slot` | Disable LIFO slot of IO runtime | -| | `--io-runtime-event-interval` | Number of scheduler ticks after which the IOtokio runtime scheduler will poll for external events | -| | `--io-runtime-global-queue-interval` | Number of scheduler ticks after which the IO runtime scheduler will poll the global task queue | -| | `--io-runtime-max-blocking-threads` | Limit for additional threads spawned by the IO runtime | -| | `--io-runtime-max-io-events-per-tick` | Maximum number of events to be processed per tick by the tokio IO runtime | -| | `--io-runtime-thread-keep-alive` | Custom timeout for a thread in the blocking pool of the tokio IO runtime | -| | `--io-runtime-thread-priority` | Set thread priority tokio IO runtime workers | -| `-h` | `--help` | Print help information | -| | `--help-all` | Print detailed help information including runtime configuration options | -| `-V` | `--version` | Print version | +| Option | | Description | +| :----- | :---------------- | :-------------------------------------------------------------------- | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information including runtime configuration options | +| `-V` | `--version` | Print version | -### Option environment variables - -You can use the following environment variables to set `influxdb3` global options: - -| Environment Variable | Option | -| :-------------------------------------------- | :------------------------------------ | -| `INFLUXDB3_NUM_THREADS` | `--num-threads` | -| `INFLUXDB3_IO_RUNTIME_TYPE` | `--io-runtime-type` | -| `INFLUXDB3_IO_RUNTIME_DISABLE_LIFO_SLOT` | `--io-runtime-disable-lifo-slot` | -| `INFLUXDB3_IO_RUNTIME_EVENT_INTERVAL` | `--io-runtime-event-interval` | -| `INFLUXDB3_IO_RUNTIME_GLOBAL_QUEUE_INTERVAL` | `--io-runtime-global-queue-interval` | -| `INFLUXDB3_IO_RUNTIME_MAX_BLOCKING_THREADS` | `--io-runtime-max-blocking-threads` | -| `INFLUXDB3_IO_RUNTIME_MAX_IO_EVENTS_PER_TICK` | `--io-runtime-max-io-events-per-tick` | -| `INFLUXDB3_IO_RUNTIME_THREAD_KEEP_ALIVE` | `--io-runtime-thread-keep-alive` | -| `INFLUXDB3_IO_RUNTIME_THREAD_PRIORITY` | `--io-runtime-thread-priority` | +For advanced global configuration options (including `--num-io-threads` and other runtime settings), see [Configuration options](/influxdb3/core/reference/config-options/#global-configuration-options). ## Examples @@ -88,6 +65,17 @@ influxdb3 serve \ --node-id my-host-01 ``` +### Run the InfluxDB 3 server with custom IO threads + + + +```bash +influxdb3 --num-io-threads=8 serve \ + --object-store file \ + --data-dir ~/.influxdb3 \ + --node-id my-host-01 +``` + ### Display short-form help for all commands @@ -109,10 +97,11 @@ influxdb3 --help ```bash -influxdb3 serve -v \ +influxdb3 serve \ --object-store file \ --data-dir ~/.influxdb3 \ - --node-id my-host-01 + --node-id my-host-01 \ + --verbose ``` ### Run {{% product-name %}} with debug logging using LOG_FILTER diff --git a/content/influxdb3/core/reference/cli/influxdb3/serve.md b/content/influxdb3/core/reference/cli/influxdb3/serve.md index a66a59209..ef5b9e019 100644 --- a/content/influxdb3/core/reference/cli/influxdb3/serve.md +++ b/content/influxdb3/core/reference/cli/influxdb3/serve.md @@ -30,6 +30,16 @@ influxdb3 serve [OPTIONS] --node-id > [!NOTE] > `--node-id` supports alphanumeric strings with optional hyphens. +> [!Important] +> #### Global configuration options +> Some configuration options (like [`--num-io-threads`](/influxdb3/core/reference/config-options/#num-io-threads)) are **global** and must be specified **before** the `serve` command: +> +> ```bash +> influxdb3 --num-io-threads=8 serve --node-id=node0 --object-store=file --verbose +> ``` +> +> See [Global configuration options](/influxdb3/core/reference/config-options/#global-configuration-options) for the complete list. + ## Options | Option | | Description | @@ -116,7 +126,6 @@ influxdb3 serve [OPTIONS] --node-id | | `--traces-jaeger-debug-name` | _See [configuration options](/influxdb3/core/reference/config-options/#traces-jaeger-debug-name)_ | | | `--traces-jaeger-max-msgs-per-second` | _See [configuration options](/influxdb3/core/reference/config-options/#traces-jaeger-max-msgs-per-second)_ | | | `--traces-jaeger-tags` | _See [configuration options](/influxdb3/core/reference/config-options/#traces-jaeger-tags)_ | -| `-v` | `--verbose` | Enable verbose output | | | `--virtual-env-location` | _See [configuration options](/influxdb3/core/reference/config-options/#virtual-env-location)_ | | | `--wal-flush-interval` | _See [configuration options](/influxdb3/core/reference/config-options/#wal-flush-interval)_ | | | `--wal-max-write-buffer-size` | _See [configuration options](/influxdb3/core/reference/config-options/#wal-max-write-buffer-size)_ | @@ -164,10 +173,10 @@ influxdb3 serve \ ```bash influxdb3 serve \ - --verbose \ --object-store file \ --data-dir ~/.influxdb3 \ --node-id my-host-01 + --verbose ``` ### Run InfluxDB 3 with debug logging using LOG_FILTER diff --git a/content/influxdb3/enterprise/reference/cli/influxdb3/_index.md b/content/influxdb3/enterprise/reference/cli/influxdb3/_index.md index 863318111..1a52c7f40 100644 --- a/content/influxdb3/enterprise/reference/cli/influxdb3/_index.md +++ b/content/influxdb3/enterprise/reference/cli/influxdb3/_index.md @@ -37,36 +37,13 @@ influxdb3 [GLOBAL-OPTIONS] [COMMAND] ## Global options -| Option | | Description | -| :----- | :------------------------------------ | :------------------------------------------------------------------------------------------------ | -| | `--num-threads` | Maximum number of IO runtime threads to use | -| | `--io-runtime-type` | IO tokio runtime type (`current-thread`, `multi-thread` _(default)_, or `multi-thread-alt`) | -| | `--io-runtime-disable-lifo-slot` | Disable LIFO slot of IO runtime | -| | `--io-runtime-event-interval` | Number of scheduler ticks after which the IOtokio runtime scheduler will poll for external events | -| | `--io-runtime-global-queue-interval` | Number of scheduler ticks after which the IO runtime scheduler will poll the global task queue | -| | `--io-runtime-max-blocking-threads` | Limit for additional threads spawned by the IO runtime | -| | `--io-runtime-max-io-events-per-tick` | Maximum number of events to be processed per tick by the tokio IO runtime | -| | `--io-runtime-thread-keep-alive` | Custom timeout for a thread in the blocking pool of the tokio IO runtime | -| | `--io-runtime-thread-priority` | Set thread priority tokio IO runtime workers | -| `-h` | `--help` | Print help information | -| | `--help-all` | Print detailed help information | -| `-V` | `--version` | Print version | +| Option | | Description | +| :----- | :---------------- | :-------------------------------------------------------------------- | +| `-h` | `--help` | Print help information | +| | `--help-all` | Print detailed help information including runtime configuration options | +| `-V` | `--version` | Print version | -### Option environment variables - -You can use the following environment variables to set `influxdb3` global options: - -| Environment Variable | Option | -| :-------------------------------------------- | :------------------------------------ | -| `INFLUXDB3_NUM_THREADS` | `--num-threads` | -| `INFLUXDB3_IO_RUNTIME_TYPE` | `--io-runtime-type` | -| `INFLUXDB3_IO_RUNTIME_DISABLE_LIFO_SLOT` | `--io-runtime-disable-lifo-slot` | -| `INFLUXDB3_IO_RUNTIME_EVENT_INTERVAL` | `--io-runtime-event-interval` | -| `INFLUXDB3_IO_RUNTIME_GLOBAL_QUEUE_INTERVAL` | `--io-runtime-global-queue-interval` | -| `INFLUXDB3_IO_RUNTIME_MAX_BLOCKING_THREADS` | `--io-runtime-max-blocking-threads` | -| `INFLUXDB3_IO_RUNTIME_MAX_IO_EVENTS_PER_TICK` | `--io-runtime-max-io-events-per-tick` | -| `INFLUXDB3_IO_RUNTIME_THREAD_KEEP_ALIVE` | `--io-runtime-thread-keep-alive` | -| `INFLUXDB3_IO_RUNTIME_THREAD_PRIORITY` | `--io-runtime-thread-priority` | +For advanced global configuration options (including `--num-io-threads` and other runtime settings), see [Configuration options](/influxdb3/enterprise/reference/config-options/#global-configuration-options). ## Examples @@ -93,6 +70,18 @@ influxdb3 serve \ --cluster-id my-cluster-01 ``` +### Run the InfluxDB 3 server with custom IO threads + + + +```bash +influxdb3 --num-io-threads=8 serve \ + --object-store file \ + --data-dir ~/.influxdb3 \ + --node-id my-host-01 \ + --cluster-id my-cluster-01 +``` + ### Display short-form help for all commands @@ -114,11 +103,12 @@ influxdb3 --help ```bash -influxdb3 serve -v \ +influxdb3 serve \ --object-store file \ --data-dir ~/.influxdb3 \ --node-id my-host-01 \ - --cluster-id my-cluster-01 + --cluster-id my-cluster-01 \ + --verbose ``` ### Run {{% product-name %}} with debug logging using LOG_FILTER diff --git a/content/influxdb3/enterprise/reference/cli/influxdb3/serve.md b/content/influxdb3/enterprise/reference/cli/influxdb3/serve.md index c44102dd3..bded5f63b 100644 --- a/content/influxdb3/enterprise/reference/cli/influxdb3/serve.md +++ b/content/influxdb3/enterprise/reference/cli/influxdb3/serve.md @@ -33,6 +33,16 @@ influxdb3 serve [OPTIONS] \ > [!NOTE] > `--node-id` and `--cluster-id` support alphanumeric strings with optional hyphens. +> [!Important] +> #### Global configuration options +> Some configuration options (like [`--num-io-threads`](/influxdb3/enterprise/reference/config-options/#num-io-threads)) are **global** and must be specified **before** the `serve` command: +> +> ```bash +> influxdb3 --num-io-threads=8 serve --node-id=node0 --cluster-id=cluster0 --verbose +> ``` +> +> See [Global configuration options](/influxdb3/enterprise/reference/config-options/#global-configuration-options) for the complete list. + ## Options | Option | | Description | @@ -100,6 +110,7 @@ influxdb3 serve [OPTIONS] \ | {{< req "\*" >}} | `--node-id` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#node-id)_ | | | `--node-id-from-env` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#node-id-from-env)_ | | | `--num-cores` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#num-cores)_ | +| | `--num-datafusion-threads` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#num-datafusion-threads)_ | | | `--num-database-limit` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#num-database-limit)_ | | | `--num-table-limit` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#num-table-limit)_ | | | `--num-total-columns-per-table-limit` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#num-total-columns-per-table-limit)_ | @@ -140,7 +151,6 @@ influxdb3 serve [OPTIONS] \ | | `--traces-jaeger-max-msgs-per-second` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#traces-jaeger-max-msgs-per-second)_ | | | `--traces-jaeger-tags` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#traces-jaeger-tags)_ | | | `--use-pacha-tree` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#use-pacha-tree)_ | -| `-v` | `--verbose` | Enable verbose output | | | `--virtual-env-location` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#virtual-env-location)_ | | | `--wait-for-running-ingestor` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#wait-for-running-ingestor)_ | | | `--wal-flush-interval` | _See [configuration options](/influxdb3/enterprise/reference/config-options/#wal-flush-interval)_ | @@ -220,11 +230,11 @@ influxdb3 serve \ ```bash influxdb3 serve \ - --verbose \ --object-store file \ --data-dir ~/.influxdb3 \ --node-id my-host-01 \ --cluster-id my-cluster-01 + --verbose ``` ### Run InfluxDB 3 with debug logging using LOG_FILTER diff --git a/content/shared/influxdb3-cli/config-options.md b/content/shared/influxdb3-cli/config-options.md index 8d910f1fc..2eca2caa8 100644 --- a/content/shared/influxdb3-cli/config-options.md +++ b/content/shared/influxdb3-cli/config-options.md @@ -1,26 +1,48 @@ -{{< product-name >}} lets you customize your server configuration by using -`influxdb3 serve` command options or by setting environment variables. +{{< product-name >}} lets you customize your configuration by using +`influxdb3` command options or by setting environment variables. ## Configure your server -Pass configuration options to the `influxdb serve` server using either command -options or environment variables. Command options take precedence over -environment variables. +Pass configuration options using either command options or environment variables. +Command options take precedence over environment variables. -##### Example `influxdb3 serve` command options +### Global vs serve-specific options +Some options are **global** (specified before the command) while others are **serve-specific** (specified after `serve`): + +- **Global options**: Apply to the `influxdb3` CLI itself (for example, `--num-io-threads`) +- **Serve options**: Apply only to the `serve` command (for example, `--node-id`, `--object-store`, `--verbose`) + +#### Example command with global and serve-specific options + +{{% show-in "core" %}} ```sh -influxdb3 serve \ +influxdb3 --num-io-threads=4 serve \ --node-id node0 \ -{{% show-in "enterprise" %}} --cluster-id cluster0 \ - --license-email example@email.com \{{% /show-in %}} --object-store file \ --data-dir ~/.influxdb3 \ + --verbose \ --log-filter info ``` +{{% /show-in %}} + +{{% show-in "enterprise" %}} + + +```sh +influxdb3 --num-io-threads=4 serve \ + --node-id node0 \ + --cluster-id cluster0 \ + --license-email example@email.com \ + --object-store file \ + --data-dir ~/.influxdb3 \ + --verbose \ + --log-filter info +``` +{{% /show-in %}} ##### Example environment variables @@ -37,6 +59,30 @@ export LOG_FILTER=info influxdb3 serve ``` +## Global configuration options + +The following options apply to the `influxdb3` CLI globally and must be specified **before** any subcommand (for example, `serve`): + +### num-io-threads + +Sets the number of threads allocated to the IO runtime thread pool. IO threads handle HTTP request serving, line protocol parsing, and file operations. + +> [!Important] +> This is a **global option** that must be specified before the `serve` command. + +```bash +# Set IO threads (global option before serve) +influxdb3 --num-io-threads=8 serve --node-id=node0 --object-store=file +``` + +For detailed information about thread allocation, see the [Resource Limits](#resource-limits) section. + +| influxdb3 option | Environment variable | +| :--------------- | :------------------------- | +| `--num-io-threads` | `INFLUXDB3_NUM_IO_THREADS` | + +--- + ## Server configuration options - [General](#general) @@ -104,14 +150,16 @@ influxdb3 serve - [traces-jaeger-max-msgs-per-second](#traces-jaeger-max-msgs-per-second) - [DataFusion](#datafusion) - [datafusion-num-threads](#datafusion-num-threads) - - [datafusion-runtime-type](#datafusion-runtime-type) - - [datafusion-runtime-disable-lifo-slot](#datafusion-runtime-disable-lifo-slot) - - [datafusion-runtime-event-interval](#datafusion-runtime-event-interval) - - [datafusion-runtime-global-queue-interval](#datafusion-runtime-global-queue-interval) - - [datafusion-runtime-max-blocking-threads](#datafusion-runtime-max-blocking-threads) - - [datafusion-runtime-max-io-events-per-tick](#datafusion-runtime-max-io-events-per-tick) - - [datafusion-runtime-thread-keep-alive](#datafusion-runtime-thread-keep-alive) - - [datafusion-runtime-thread-priority](#datafusion-runtime-thread-priority) + - [datafusion-max-parquet-fanout](#datafusion-max-parquet-fanout) - [datafusion-use-cached-parquet-loader](#datafusion-use-cached-parquet-loader) - [datafusion-config](#datafusion-config) @@ -223,7 +271,7 @@ Required when using the `file` [object store](#object-store). {{% show-in "enterprise" %}} #### mode -Sets the mode to start the server in. +Sets the mode to start the server in, allowing you to create specialized nodes in a distributed cluster. This option supports the following values: @@ -237,6 +285,32 @@ You can specify multiple modes using a comma-delimited list (for example, `inges **Default:** `all` +> [!Important] +> **Thread allocation for different modes:** +> +> - **Ingest mode**: Benefits from additional IO threads for line protocol parsing. For high-throughput +> scenarios with multiple concurrent writers, consider increasing [`--num-io-threads`](#num-io-threads) (global option) +> to 8-16+ to optimize performance. DataFusion threads are still needed for snapshot operations. +> +> - **Query mode**: Benefits from maximizing DataFusion threads. Use most available cores for DataFusion +> with minimal IO threads (2-4). +> +> - **Compact mode**: Primarily uses DataFusion threads for sort/dedupe operations. +> +> - **All mode**: Requires balanced thread allocation based on your workload mix. + +**Example configurations:** +```bash +# High-throughput ingest node (32 cores) +influxdb3 --num-io-threads=12 serve --mode=ingest --num-datafusion-threads=20 + +# Query-optimized node (32 cores) +influxdb3 --num-io-threads=4 serve --mode=query --num-datafusion-threads=28 + +# Balanced all-in-one (32 cores) +influxdb3 --num-io-threads=6 serve --mode=all --num-datafusion-threads=26 +``` + | influxdb3 serve option | Environment variable | | :--------------------- | :-------------------------- | | `--mode` | `INFLUXDB3_ENTERPRISE_MODE` | @@ -1126,102 +1200,25 @@ Sets the maximum number of DataFusion runtime threads to use. --- -#### datafusion-runtime-type + --- @@ -1871,28 +1868,68 @@ Specifies how long to wait for a running ingestor during startup. - [num-cores](#num-cores) +{{% show-in "enterprise" %}}- [num-datafusion-threads](#num-datafusion-threads){{% /show-in %}} +- _[num-io-threads](#num-io-threads) - See [Global configuration options](#global-configuration-options)_ - [num-database-limit](#num-database-limit) - [num-table-limit](#num-table-limit) - [num-total-columns-per-table-limit](#num-total-columns-per-table-limit) #### num-cores +{{% show-in "enterprise" %}} Limits the number of CPU cores that the InfluxDB 3 Enterprise process can use when running on systems where resources are shared. When specified, InfluxDB automatically assigns the number of DataFusion threads and IO threads based on the core count. -**Thread assignment logic:** +**Default thread assignment logic when `num-cores` is set:** - **1-2 cores**: 1 IO thread, 1 DataFusion thread -- **3 cores**: 1 IO thread, 2 DataFusion threads +- **3 cores**: 1 IO thread, 2 DataFusion threads - **4+ cores**: 2 IO threads, (n-2) DataFusion threads +> [!Note] +> You can override the automatic thread assignment by explicitly setting [`--num-io-threads`](#num-io-threads) (global option) +> and [`--num-datafusion-threads`](#num-datafusion-threads). +> This is particularly important for specialized +> workloads like [ingest mode](#mode) where you may need more IO threads than the default allocation. + **Constraints:** - Must be at least 2 - Cannot exceed the number of cores available on the system -- Total thread count from other thread options cannot exceed the `num-cores` value +- Total thread count from `--num-io-threads` (global option) and `--num-datafusion-threads` cannot exceed the `num-cores` value | influxdb3 serve option | Environment variable | | :--------------------- | :-------------------------------- | | `--num-cores` | `INFLUXDB3_ENTERPRISE_NUM_CORES` | +{{% /show-in %}} + +{{% show-in "enterprise" %}} + +> [!Note] +> The [`--num-io-threads`](#num-io-threads) option is a global flag. +--- + +#### num-datafusion-threads + +Sets the number of threads allocated to the DataFusion runtime thread pool. DataFusion threads handle: +- Query execution and processing +- Data aggregation and transformation +- Snapshot creation (sort/dedupe operations) +- Parquet file generation + +**Default behavior:** +- If not specified and `--num-cores` is not set: All available cores minus IO threads +- If not specified and `--num-cores` is set: Automatically determined based on core count (see [`--num-cores`](#num-cores)) + +> [!Note] +> DataFusion threads are used for both query processing and snapshot operations. +> Even ingest-only nodes use DataFusion threads during WAL snapshot creation. + +**Constraints:** +- When used with `--num-cores`, the sum of `--num-io-threads` and `--num-datafusion-threads` cannot exceed the `num-cores` value + +| influxdb3 serve option | Environment variable | +| :----------------------------- | :-------------------------------------- | +| `--num-datafusion-threads` | `INFLUXDB3_NUM_DATAFUSION_THREADS` | +{{% /show-in %}} ---