From 089190dc60f2a0e8cece092607e3b94af7228b65 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Wed, 5 Nov 2025 00:04:44 -0500 Subject: [PATCH] fix(influxdb3): influxdb3 install package command syntax and details (#6512) * fix(influxdb3): override PLUGIN_DIR env var in compose services to match --plugin-dir - Set INFLUXDB3_PLUGIN_DIR=/var/lib/influxdb3/plugins in both influxdb3-core and influxdb3-enterprise services - Overrides Docker image default (/plugins) to match --plugin-dir command argument - Ensures package installation works correctly with bind-mounted plugin directory - Update influxdb3 install package CLI documentation with complete options reference * docs(influxdb3): add Docker disk space troubleshooting for package installation Add troubleshooting section for 'No space left on device' errors when installing packages in Docker environments. Includes commands to check Docker disk usage and clean up unused images and build cache. Also fix Vale linting errors by changing 'URL' to 'url'. * Apply suggestion from @jstirnaman * fix(ci): Vale rules for URL/-url-/WAL/-wal-/Parquet/-parquet-: 1. Simplified the Parquet vocabulary entry from the regex to just Parquet 2. Added separate regex patterns for hyphenated identifiers * feat(influxdb3): describe use cases and effects of --package-manager disabled --- .../vocabularies/InfluxDataDocs/accept.txt | 6 +- compose.yaml | 8 +- .../shared/influxdb3-cli/config-options.md | 548 ++++++++++-------- .../shared/influxdb3-cli/install/package.md | 122 +++- 4 files changed, 412 insertions(+), 272 deletions(-) diff --git a/.ci/vale/styles/config/vocabularies/InfluxDataDocs/accept.txt b/.ci/vale/styles/config/vocabularies/InfluxDataDocs/accept.txt index 8a451ac33..e9f7aa19a 100644 --- a/.ci/vale/styles/config/vocabularies/InfluxDataDocs/accept.txt +++ b/.ci/vale/styles/config/vocabularies/InfluxDataDocs/accept.txt @@ -31,7 +31,8 @@ LogicalPlan [Mm]onitor MBs? PBs? -Parquet|\b\w*-*parquet-\w*\b|\b--\w*parquet\w*\b|`[^`]*parquet[^`]*` +Parquet +\w*-?\w*parquet\w*-\w* Redoc SQLAlchemy SQLAlchemy @@ -41,9 +42,11 @@ System.Data.Odbc TBs? \bUI\b URL +\w*-?\w*url\w*-\w* US (East|West|Central|North|South|Northeast|Northwest|Southeast|Southwest) Unix WALs? +\w*-?wal-\w* Webpack [pP]y.*\b \b\w+_\w+\b @@ -82,7 +85,6 @@ quoteChar retentionRules sourceBucket tagKey -url[s]? v2 v3 venv diff --git a/compose.yaml b/compose.yaml index 5d3887a13..e766e867f 100644 --- a/compose.yaml +++ b/compose.yaml @@ -334,6 +334,7 @@ services: target: /var/lib/influxdb3/plugins/custom environment: - INFLUXDB3_AUTH_TOKEN=/run/secrets/influxdb3-core-admin-token + - INFLUXDB3_PLUGIN_DIR=/var/lib/influxdb3/plugins secrets: - influxdb3-core-admin-token influxdb3-enterprise: @@ -357,14 +358,15 @@ services: - --verbose environment: - INFLUXDB3_AUTH_TOKEN=/run/secrets/influxdb3-enterprise-admin-token + - INFLUXDB3_PLUGIN_DIR=/var/lib/influxdb3/plugins volumes: - - type: bind + - type: bind source: test/.influxdb3/enterprise/data target: /var/lib/influxdb3/data - - type: bind + - type: bind source: test/.influxdb3/plugins/influxdata target: /var/lib/influxdb3/plugins - - type: bind + - type: bind source: test/.influxdb3/enterprise/plugins target: /var/lib/influxdb3/plugins/custom secrets: diff --git a/content/shared/influxdb3-cli/config-options.md b/content/shared/influxdb3-cli/config-options.md index ff3d2d06d..a8e2957f5 100644 --- a/content/shared/influxdb3-cli/config-options.md +++ b/content/shared/influxdb3-cli/config-options.md @@ -1,4 +1,5 @@ + {{< product-name >}} lets you customize your configuration by using `influxdb3` command options or by setting environment variables. @@ -17,6 +18,7 @@ Some options are **global** (specified before the command) while others are **se #### Example command with global and serve-specific options {{% show-in "core" %}} + ```sh @@ -27,9 +29,11 @@ influxdb3 --num-io-threads=4 serve \ --verbose \ --log-filter info ``` + {{% /show-in %}} {{% show-in "enterprise" %}} + ```sh @@ -42,6 +46,7 @@ influxdb3 --num-io-threads=4 serve \ --verbose \ --log-filter info ``` + {{% /show-in %}} ##### Example environment variables @@ -67,7 +72,7 @@ The following options apply to the `influxdb3` CLI globally and must be specifie Sets the number of threads allocated to the IO runtime thread pool. IO threads handle HTTP request serving, line protocol parsing, and file operations. -> [!Important] +> \[!Important] > `--num-io-threads` is a **global option** that must be specified before the `serve` command. {{% show-in "enterprise" %}} @@ -83,16 +88,16 @@ influxdb3 --num-io-threads=8 serve --node-id=node0 --object-store=file For detailed information about thread allocation, see the [Resource Limits](#resource-limits) section. {{% /show-in %}} -| influxdb3 option | Environment variable | -| :--------------- | :------------------------- | +| influxdb3 option | Environment variable | +| :----------------- | :------------------------- | | `--num-io-threads` | `INFLUXDB3_NUM_IO_THREADS` | ---- +*** ## Server configuration options - [General](#general) -{{% show-in "enterprise" %}}- [Licensing](#licensing){{% /show-in %}} + {{% show-in "enterprise" %}}- [Licensing](#licensing){{% /show-in %}} - [Security](#security) - [AWS](#aws) - [Google Cloud Service](#google-cloud-service) @@ -107,43 +112,46 @@ For detailed information about thread allocation, see the [Resource Limits](#res - [Compaction](#compaction) - [Caching](#caching) - [Processing Engine](#processing-engine) -{{% show-in "enterprise" %}} + {{% show-in "enterprise" %}} - [Cluster Management](#cluster-management) -{{% /show-in %}} + {{% /show-in %}} - [Resource Limits](#resource-limits) - [Data Lifecycle Management](#data-lifecycle-management) - [Telemetry](#telemetry) - [TCP Listeners](#tcp-listeners) ---- +*** ### General {{% show-in "enterprise" %}} + - [cluster-id](#cluster-id) -{{% /show-in %}} + {{% /show-in %}} - [data-dir](#data-dir) -{{% show-in "enterprise" %}} + {{% show-in "enterprise" %}} - [mode](#mode) -{{% /show-in %}} + {{% /show-in %}} - [node-id](#node-id) -{{% show-in "enterprise" %}} + {{% show-in "enterprise" %}} - [node-id-from-env](#node-id-from-env) -{{% /show-in %}} + {{% /show-in %}} - [object-store](#object-store) - [query-file-limit](#query-file-limit) {{% show-in "enterprise" %}} + #### cluster-id -Specifies the cluster identifier that prefixes the object store path for the Enterprise Catalog. +Specifies the cluster identifier that prefixes the object store path for the Enterprise Catalog. This value must be different than the [`--node-id`](#node-id) value. -| influxdb3 serve option | Environment variable | -| :--------------------- | :--------------------------------- | -| `--cluster-id` | `INFLUXDB3_ENTERPRISE_CLUSTER_ID` | +| influxdb3 serve option | Environment variable | +| :--------------------- | :-------------------------------- | +| `--cluster-id` | `INFLUXDB3_ENTERPRISE_CLUSTER_ID` | + +*** ---- {{% /show-in %}} #### data-dir @@ -155,16 +163,17 @@ Required when using the `file` [object store](#object-store). | :--------------------- | :------------------- | | `--data-dir` | `INFLUXDB3_DB_DIR` | ---- +*** {{% show-in "enterprise" %}} + #### mode Sets the mode to start the server in, allowing you to create specialized nodes in a distributed cluster. This option supports the following values: -- `all` _(default)_: Enables all server modes +- `all` *(default)*: Enables all server modes - `ingest`: Enables only data ingest capabilities - `query`: Enables only query capabilities - `compact`: Enables only compaction processes @@ -174,7 +183,7 @@ You can specify multiple modes using a comma-delimited list (for example, `inges **Default:** `all` -> [!Important] +> \[!Important] > **Thread allocation for different modes:** > > - **Ingest mode**: Benefits from additional IO threads for line protocol parsing. For high-throughput @@ -189,6 +198,7 @@ You can specify multiple modes using a comma-delimited list (for example, `inges > - **All mode**: Requires balanced thread allocation based on your workload mix. **Example configurations:** + ```bash # High-throughput ingest node (32 cores) influxdb3 --num-io-threads=12 serve --mode=ingest --datafusion-num-threads=20 @@ -204,7 +214,8 @@ influxdb3 --num-io-threads=6 serve --mode=all --datafusion-num-threads=26 | :--------------------- | :-------------------------- | | `--mode` | `INFLUXDB3_ENTERPRISE_MODE` | ---- +*** + {{% /show-in %}} #### node-id @@ -217,9 +228,10 @@ configuration--for example, the same bucket. | :--------------------- | :--------------------------------- | | `--node-id` | `INFLUXDB3_NODE_IDENTIFIER_PREFIX` | ---- +*** {{% show-in "enterprise" %}} + #### node-id-from-env Specifies the node identifier used as a prefix in all object store file paths. @@ -240,7 +252,8 @@ export DATABASE_NODE=node0 && influxdb3 serve \ --data-dir ~/.influxdb3/data ``` ---- +*** + {{% /show-in %}} #### object-store @@ -259,9 +272,10 @@ This option supports the following values: | :--------------------- | :----------------------- | | `--object-store` | `INFLUXDB3_OBJECT_STORE` | ---- +*** {{% show-in "enterprise" %}} + ### Licensing #### license-email @@ -274,7 +288,7 @@ This option is mutually exclusive with [license-file](#license-file). | :--------------------- | :----------------------------------- | | `--license-email` | `INFLUXDB3_ENTERPRISE_LICENSE_EMAIL` | ---- +*** #### license-file @@ -282,11 +296,11 @@ Specifies the path to a license file for {{< product-name >}}. When provided, th file's contents are used instead of requesting a new license. This option is mutually exclusive with [license-email](#license-email). -| influxdb3 serve option | Environment variable | -| :--------------------- | :----------------------------------- | -| `--license-file` | `INFLUXDB3_ENTERPRISE_LICENSE_FILE` | +| influxdb3 serve option | Environment variable | +| :--------------------- | :---------------------------------- | +| `--license-file` | `INFLUXDB3_ENTERPRISE_LICENSE_FILE` | ---- +*** #### license-type @@ -297,11 +311,12 @@ interactive license prompt. Provide one of the following license types: - `trial` - `commercial` -| influxdb3 serve option | Environment variable | -| :--------------------- | :----------------------------------- | -| `--license-type` | `INFLUXDB3_ENTERPRISE_LICENSE_TYPE` | +| influxdb3 serve option | Environment variable | +| :--------------------- | :---------------------------------- | +| `--license-type` | `INFLUXDB3_ENTERPRISE_LICENSE_TYPE` | + +*** ---- {{% /show-in %}} ### Security @@ -313,61 +328,61 @@ interactive license prompt. Provide one of the following license types: - [disable-authz](#disable-authz) - [admin-token-recovery-http-bind](#admin-token-recovery-http-bind) - [admin-token-file](#admin-token-file) -{{% show-in "enterprise" %}}- [permission-tokens-file](#permission-tokens-file){{% /show-in %}} + {{% show-in "enterprise" %}}- [permission-tokens-file](#permission-tokens-file){{% /show-in %}} #### tls-key The path to a key file for TLS to be enabled. -| influxdb3 serve option | Environment variable | -| :--------------------- | :--------------------- | -| `--tls-key` | `INFLUXDB3_TLS_KEY` | +| influxdb3 serve option | Environment variable | +| :--------------------- | :------------------- | +| `--tls-key` | `INFLUXDB3_TLS_KEY` | ---- +*** #### tls-cert The path to a cert file for TLS to be enabled. -| influxdb3 serve option | Environment variable | -| :--------------------- | :--------------------- | -| `--tls-cert` | `INFLUXDB3_TLS_CERT` | +| influxdb3 serve option | Environment variable | +| :--------------------- | :------------------- | +| `--tls-cert` | `INFLUXDB3_TLS_CERT` | ---- +*** #### tls-minimum-version -The minimum version for TLS. +The minimum version for TLS. Valid values are `tls-1.2` or `tls-1.3`. Default is `tls-1.2`. -| influxdb3 serve option | Environment variable | -| :---------------------- | :----------------------- | +| influxdb3 serve option | Environment variable | +| :---------------------- | :------------------------------ | | `--tls-minimum-version` | `INFLUXDB3_TLS_MINIMUM_VERSION` | ---- +*** #### without-auth Disables authentication for all server actions (CLI commands and API requests). The server processes all requests without requiring tokens or authentication. -| influxdb3 serve option | Environment variable | -| :--------------------- | :---------------------------- | -| `--without-auth` | `INFLUXDB3_START_WITHOUT_AUTH`| +| influxdb3 serve option | Environment variable | +| :--------------------- | :----------------------------- | +| `--without-auth` | `INFLUXDB3_START_WITHOUT_AUTH` | ---- +*** #### disable-authz -Optionally disable authz by passing in a comma separated list of resources. +Optionally disable authz by passing in a comma separated list of resources. Valid values are `health`, `ping`, and `metrics`. | influxdb3 serve option | Environment variable | | :--------------------- | :------------------------ | | `--disable-authz` | `INFLUXDB3_DISABLE_AUTHZ` | ---- +*** #### admin-token-recovery-http-bind @@ -375,7 +390,7 @@ Enables an admin token recovery HTTP server on a separate port. This server allows regenerating lost admin tokens without existing authentication. The server automatically shuts down after a successful token regeneration. -> [!Warning] +> \[!Warning] > This option creates an unauthenticated endpoint that can regenerate admin tokens. > Only use this when you have lost access to your admin token and ensure the > server is only accessible from trusted networks. @@ -396,7 +411,7 @@ influxdb3 serve --admin-token-recovery-http-bind influxdb3 create token --admin --regenerate --host http://127.0.0.1:8182 ``` ---- +*** #### admin-token-file @@ -412,6 +427,7 @@ Offline admin tokens are defined in a JSON-formatted file. Use the following command to generate an offline admin token file: + ```bash { placeholders="./path/to/admin-token.json" } influxdb3 create token --admin \ --name "example-admin-token" \ @@ -422,6 +438,7 @@ influxdb3 create token --admin \ {{< expand-wrapper >}} {{% expand "View example offline admin token file" %}} + ```json { "token": "apiv3_0XXXX-xxxXxXxxxXX_OxxxX...", @@ -429,6 +446,7 @@ influxdb3 create token --admin \ "expiry_millis": 1756400061529 } ``` + {{% /expand %}} {{< /expand-wrapper >}} @@ -449,9 +467,10 @@ influxdb3 create token \ influxdb3 serve --admin-token-file ./path/to/admin-token.json ``` ---- +*** {{% show-in "enterprise" %}} + #### permission-tokens-file Specifies an offline permission (resource) tokens file to use if no resource @@ -481,6 +500,7 @@ influxdb3 create token \ {{< expand-wrapper >}} {{% expand "View example offline permission tokens file" %}} + ```json { "create_databases": [ @@ -501,10 +521,11 @@ influxdb3 create token \ ] } ``` + {{% /expand %}} {{< /expand-wrapper >}} -> [!Note] +> \[!Note] > If you write a new offline permission token to an existing permission token file, > the command appends the new token to the existing output file. @@ -527,7 +548,8 @@ influxdb3 create token \ influxdb3 serve --permission-tokens-file ./path/to/tokens.json ``` ---- +*** + {{% /show-in %}} ### AWS @@ -550,7 +572,7 @@ permission to read from and write to the specified S3 bucket. | :--------------------- | :------------------- | | `--aws-access-key-id` | `AWS_ACCESS_KEY_ID` | ---- +*** #### aws-secret-access-key @@ -561,7 +583,7 @@ goes with the specified access key ID. | :------------------------ | :---------------------- | | `--aws-secret-access-key` | `AWS_SECRET_ACCESS_KEY` | ---- +*** #### aws-default-region @@ -574,7 +596,7 @@ the specified bucket if different from the fallback value. | :--------------------- | :------------------- | | `--aws-default-region` | `AWS_DEFAULT_REGION` | ---- +*** #### aws-endpoint @@ -584,7 +606,7 @@ When using an Amazon S3 compatibility storage service, set this to the endpoint. | :--------------------- | :------------------- | | `--aws-endpoint` | `AWS_ENDPOINT` | ---- +*** #### aws-session-token @@ -595,7 +617,7 @@ handy when using a federated login or SSO and fetching credentials via the UI. | :--------------------- | :------------------- | | `--aws-session-token` | `AWS_SESSION_TOKEN` | ---- +*** #### aws-allow-http @@ -605,7 +627,7 @@ Allows unencrypted HTTP connections to AWS. | :--------------------- | :------------------- | | `--aws-allow-http` | `AWS_ALLOW_HTTP` | ---- +*** #### aws-skip-signature @@ -615,7 +637,7 @@ If enabled, S3 object stores do not fetch credentials and do not sign requests. | :--------------------- | :------------------- | | `--aws-skip-signature` | `AWS_SKIP_SIGNATURE` | ---- +*** #### aws-credentials-file @@ -646,7 +668,7 @@ request. | :----------------------- | :--------------------- | | `--aws-credentials-file` | `AWS_CREDENTIALS_FILE` | ---- +*** ### Google Cloud Service @@ -661,7 +683,7 @@ JSON file that contains the Google credentials. | :------------------------- | :----------------------- | | `--google-service-account` | `GOOGLE_SERVICE_ACCOUNT` | ---- +*** ### Microsoft Azure @@ -679,7 +701,7 @@ when navigating to **All Services > Storage accounts > `[name]`**. | :------------------------ | :---------------------- | | `--azure-storage-account` | `AZURE_STORAGE_ACCOUNT` | ---- +*** #### azure-storage-access-key @@ -690,7 +712,7 @@ values in the Storage account's **Settings > Access keys**. | :--------------------------- | :------------------------- | | `--azure-storage-access-key` | `AZURE_STORAGE_ACCESS_KEY` | ---- +*** #### azure-endpoint @@ -701,7 +723,7 @@ Storage endpoint. | :--------------------- | :------------------- | | `--azure-endpoint` | `AZURE_ENDPOINT` | ---- +*** #### azure-allow-http @@ -714,7 +736,7 @@ to Azure Blob Storage. | :--------------------- | :------------------- | | `--azure-allow-http` | `AZURE_ALLOW_HTTP` | ---- +*** ### Object Storage @@ -735,7 +757,7 @@ Sets the name of the object storage bucket to use. Must also set | :--------------------- | :------------------- | | `--bucket` | `INFLUXDB3_BUCKET` | ---- +*** #### object-store-connection-limit @@ -748,7 +770,7 @@ this value. | :-------------------------------- | :------------------------------ | | `--object-store-connection-limit` | `OBJECT_STORE_CONNECTION_LIMIT` | ---- +*** #### object-store-http2-only @@ -758,7 +780,7 @@ Forces HTTP/2 connections to network-based object stores. | :-------------------------- | :------------------------ | | `--object-store-http2-only` | `OBJECT_STORE_HTTP2_ONLY` | ---- +*** #### object-store-http2-max-frame-size @@ -768,7 +790,7 @@ Sets the maximum frame size (in bytes/octets) for HTTP/2 connections. | :------------------------------------ | :---------------------------------- | | `--object-store-http2-max-frame-size` | `OBJECT_STORE_HTTP2_MAX_FRAME_SIZE` | ---- +*** #### object-store-max-retries @@ -778,7 +800,7 @@ Defines the maximum number of times to retry a request. | :--------------------------- | :------------------------- | | `--object-store-max-retries` | `OBJECT_STORE_MAX_RETRIES` | ---- +*** #### object-store-retry-timeout @@ -789,7 +811,7 @@ further retries are be attempted. | :----------------------------- | :--------------------------- | | `--object-store-retry-timeout` | `OBJECT_STORE_RETRY_TIMEOUT` | ---- +*** #### object-store-cache-endpoint @@ -799,7 +821,7 @@ Sets the endpoint of an S3-compatible, HTTP/2-enabled object store cache. | :------------------------------ | :---------------------------- | | `--object-store-cache-endpoint` | `OBJECT_STORE_CACHE_ENDPOINT` | ---- +*** ### Logs @@ -816,7 +838,7 @@ Sets the filter directive for logs. | :--------------------- | :------------------- | | `--log-filter` | `LOG_FILTER` | ---- +*** #### log-destination @@ -824,7 +846,7 @@ Specifies the destination for logs. This option supports the following values: -- `stdout` _(default)_ +- `stdout` *(default)* - `stderr` **Default:** `stdout` @@ -833,7 +855,7 @@ This option supports the following values: | :--------------------- | :------------------- | | `--log-destination` | `LOG_DESTINATION` | ---- +*** #### log-format @@ -841,7 +863,7 @@ Defines the message format for logs. This option supports the following values: -- `full` _(default)_ +- `full` *(default)* **Default:** `full` @@ -849,7 +871,7 @@ This option supports the following values: | :--------------------- | :------------------- | | `--log-format` | `LOG_FORMAT` | ---- +*** #### query-log-size @@ -862,7 +884,7 @@ log before older queries are evicted to make room for new ones. | :--------------------- | :------------------------- | | `--query-log-size` | `INFLUXDB3_QUERY_LOG_SIZE` | ---- +*** ### Traces @@ -885,7 +907,7 @@ Sets the type of tracing exporter. | :--------------------- | :------------------- | | `--traces-exporter` | `TRACES_EXPORTER` | ---- +*** #### traces-exporter-jaeger-agent-host @@ -897,7 +919,7 @@ Specifies the Jaeger agent network hostname for tracing. | :------------------------------------ | :---------------------------------- | | `--traces-exporter-jaeger-agent-host` | `TRACES_EXPORTER_JAEGER_AGENT_HOST` | ---- +*** #### traces-exporter-jaeger-agent-port @@ -909,7 +931,7 @@ Defines the Jaeger agent network port for tracing. | :------------------------------------ | :---------------------------------- | | `--traces-exporter-jaeger-agent-port` | `TRACES_EXPORTER_JAEGER_AGENT_PORT` | ---- +*** #### traces-exporter-jaeger-service-name @@ -921,7 +943,7 @@ Sets the Jaeger service name for tracing. | :-------------------------------------- | :------------------------------------ | | `--traces-exporter-jaeger-service-name` | `TRACES_EXPORTER_JAEGER_SERVICE_NAME` | ---- +*** #### traces-exporter-jaeger-trace-context-header-name @@ -933,7 +955,7 @@ Specifies the header name used for passing trace context. | :--------------------------------------------------- | :------------------------------------------------- | | `--traces-exporter-jaeger-trace-context-header-name` | `TRACES_EXPORTER_JAEGER_TRACE_CONTEXT_HEADER_NAME` | ---- +*** #### traces-jaeger-debug-name @@ -945,7 +967,7 @@ Specifies the header name used for force sampling in tracing. | :--------------------------- | :---------------------------------- | | `--traces-jaeger-debug-name` | `TRACES_EXPORTER_JAEGER_DEBUG_NAME` | ---- +*** #### traces-jaeger-tags @@ -955,7 +977,7 @@ Defines a set of `key=value` pairs to annotate tracing spans with. | :--------------------- | :---------------------------- | | `--traces-jaeger-tags` | `TRACES_EXPORTER_JAEGER_TAGS` | ---- +*** #### traces-jaeger-max-msgs-per-second @@ -967,7 +989,7 @@ Specifies the maximum number of messages sent to a Jaeger service per second. | :------------------------------------ | :---------------------------------- | | `--traces-jaeger-max-msgs-per-second` | `TRACES_JAEGER_MAX_MSGS_PER_SECOND` | ---- +*** ### DataFusion @@ -994,11 +1016,11 @@ Sets the maximum number of DataFusion runtime threads to use. | :------------------------- | :--------------------------------- | | `--datafusion-num-threads` | `INFLUXDB3_DATAFUSION_NUM_THREADS` | ---- +*** #### datafusion-max-parquet-fanout -When multiple parquet files are required in a sorted way +When multiple Parquet files are required in a sorted way (deduplication for example), specifies the maximum fanout. **Default:** `1000` @@ -1007,17 +1029,17 @@ When multiple parquet files are required in a sorted way | :-------------------------------- | :---------------------------------------- | | `--datafusion-max-parquet-fanout` | `INFLUXDB3_DATAFUSION_MAX_PARQUET_FANOUT` | ---- +*** #### datafusion-use-cached-parquet-loader -Uses a cached parquet loader when reading parquet files from the object store. +Uses a cached Parquet loader when reading Parquet files from the object store. | influxdb3 serve option | Environment variable | | :--------------------------------------- | :----------------------------------------------- | | `--datafusion-use-cached-parquet-loader` | `INFLUXDB3_DATAFUSION_USE_CACHED_PARQUET_LOADER` | ---- +*** #### datafusion-config @@ -1028,7 +1050,7 @@ Provides custom configuration to DataFusion as a comma-separated list of | :--------------------- | :---------------------------- | | `--datafusion-config` | `INFLUXDB3_DATAFUSION_CONFIG` | ---- +*** ### HTTP @@ -1045,7 +1067,7 @@ Specifies the maximum size of HTTP requests. | :------------------------ | :-------------------------------- | | `--max-http-request-size` | `INFLUXDB3_MAX_HTTP_REQUEST_SIZE` | ---- +*** #### http-bind @@ -1057,7 +1079,7 @@ Defines the address on which InfluxDB serves HTTP API requests. | :--------------------- | :------------------------- | | `--http-bind` | `INFLUXDB3_HTTP_BIND_ADDR` | ---- +*** ### Memory @@ -1077,7 +1099,7 @@ example: `8000000000` or `10%`. | :---------------------- | :------------------------------ | | `--exec-mem-pool-bytes` | `INFLUXDB3_EXEC_MEM_POOL_BYTES` | ---- +*** #### force-snapshot-mem-threshold @@ -1091,7 +1113,7 @@ percentage (portion of available memory) or absolute value in MB--for example: ` | :------------------------------- | :--------------------------------------- | | `--force-snapshot-mem-threshold` | `INFLUXDB3_FORCE_SNAPSHOT_MEM_THRESHOLD` | ---- +*** ### Write-Ahead Log (WAL) @@ -1114,7 +1136,7 @@ Use `s` for seconds or `ms` for milliseconds. For local disks, `100 ms` is recom | :--------------------- | :----------------------------- | | `--wal-flush-interval` | `INFLUXDB3_WAL_FLUSH_INTERVAL` | ---- +*** #### wal-snapshot-size @@ -1127,7 +1149,7 @@ multiplied by the interval, determines how often snapshots are taken. | :--------------------- | :---------------------------- | | `--wal-snapshot-size` | `INFLUXDB3_WAL_SNAPSHOT_SIZE` | ---- +*** #### wal-max-write-buffer-size @@ -1140,12 +1162,12 @@ flush must be executed and succeed. | :---------------------------- | :------------------------------------ | | `--wal-max-write-buffer-size` | `INFLUXDB3_WAL_MAX_WRITE_BUFFER_SIZE` | ---- +*** #### snapshotted-wal-files-to-keep Specifies the number of snapshotted WAL files to retain in the object store. -Flushing the WAL files does not clear the WAL files immediately; +Flushing the WAL files does not clear the WAL files immediately; they are deleted when the number of snapshotted WAL files exceeds this number. **Default:** `300` @@ -1154,7 +1176,7 @@ they are deleted when the number of snapshotted WAL files exceeds this number. | :-------------------------------- | :-------------------------------- | | `--snapshotted-wal-files-to-keep` | `INFLUXDB3_NUM_WAL_FILES_TO_KEEP` | ---- +*** #### wal-replay-fail-on-error @@ -1162,11 +1184,11 @@ Determines whether WAL replay should fail when encountering errors. **Default:** `false` -| influxdb3 serve option | Environment variable | -| :--------------------------- | :------------------------------------- | -| `--wal-replay-fail-on-error` | `INFLUXDB3_WAL_REPLAY_FAIL_ON_ERROR` | +| influxdb3 serve option | Environment variable | +| :--------------------------- | :----------------------------------- | +| `--wal-replay-fail-on-error` | `INFLUXDB3_WAL_REPLAY_FAIL_ON_ERROR` | ---- +*** #### wal-replay-concurrency-limit @@ -1176,25 +1198,28 @@ The default is dynamically determined. **Default:** `max(num_cpus, 10)` -| influxdb3 serve option | Environment variable | -| :--------------------------------- | :------------------------------------------ | -| `--wal-replay-concurrency-limit` | `INFLUXDB3_WAL_REPLAY_CONCURRENCY_LIMIT` | +| influxdb3 serve option | Environment variable | +| :------------------------------- | :--------------------------------------- | +| `--wal-replay-concurrency-limit` | `INFLUXDB3_WAL_REPLAY_CONCURRENCY_LIMIT` | ---- +*** ### Compaction {{% show-in "enterprise" %}} + + - [compaction-max-num-files-per-plan](#compaction-max-num-files-per-plan) - [compaction-gen2-duration](#compaction-gen2-duration) - [compaction-multipliers](#compaction-multipliers) - [compaction-cleanup-wait](#compaction-cleanup-wait) - [compaction-check-interval](#compaction-check-interval) -{{% /show-in %}} + {{% /show-in %}} - [gen1-duration](#gen1-duration) {{% show-in "enterprise" %}} +