From 7793554c6677b11cd2da0ccac652025e94873d54 Mon Sep 17 00:00:00 2001 From: pierwill <19642016+pierwill@users.noreply.github.com> Date: Thu, 17 Feb 2022 11:14:44 -0600 Subject: [PATCH] InfluxDB Enterprise 1.9.6 (#3650) * Document meta-node HTTP access logging (#3486) * Document `influxd-ctl backup -estimate` flag (#3484) Closes #3480 * add new option for SIGTERM (#3496) * Document `-meta-only-overwrite-force` restore flag for Enterprise (#3487) * Document `max-concurrent-deletes` option (#3697) * Update Enterprise cluster metrics: add `openConnections` (#3703) Closes #3653 * Remove marketplace offerings from Enterprise * Use bytes in certain Enterprise config examples (#3743) * InfluxDB Enterprise 1.9: remove default values from configuration headings (#3759) Co-authored-by: kelseiv <47797004+kelseiv@users.noreply.github.com> Co-authored-by: Scott Anderson Co-authored-by: Sam Arnold --- .../production_installation/_index.md | 2 - .../release-notes-changelog.md | 52 +- .../v1.9/administration/backup-and-restore.md | 31 +- .../configure/anti-entropy/_index.md | 4 +- .../configure/config-data-nodes.md | 597 +++++++++++++----- .../configure/config-meta-nodes.md | 148 +++-- .../v1.9/administration/configure/ports.md | 4 +- .../configure/security/authentication.md | 2 +- .../security/configure-password-hashing.md | 4 +- .../v1.9/concepts/file-system-layout.md | 10 +- .../v1.9/concepts/storage_engine.md | 4 +- .../v1.9/features/clustering-features.md | 7 +- .../v1.9/guides/authenticate.md | 2 +- .../v1.9/guides/downsample_and_retain.md | 2 +- .../introduction/install-and-deploy/_index.md | 33 - .../install-and-deploy/deploying/_index.md | 20 - .../deploying/aws/_index.md | 19 - .../deploying/aws/config-options.md | 32 - .../deploying/aws/setting-up-template.md | 58 -- .../install-and-deploy/deploying/azure.md | 76 --- .../deploying/google-cloud-platform.md | 103 --- .../installation/_index.md | 11 +- .../installation/chrono_install.md | 2 +- .../installation/data_node_installation.md | 4 +- .../installation/meta_node_installation.md | 2 +- .../introduction/installation_requirements.md | 2 +- .../v1.9/query_language/manage-database.md | 4 +- .../v1.9/supported_protocols/prometheus.md | 2 +- content/enterprise_influxdb/v1.9/tools/api.md | 4 +- .../v1.9/tools/influxd-ctl.md | 41 +- .../frequently-asked-questions.md | 2 +- .../tools/measurements-internal.md | 6 + 32 files changed, 711 insertions(+), 579 deletions(-) delete mode 100644 content/enterprise_influxdb/v1.9/introduction/install-and-deploy/_index.md delete mode 100644 content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/_index.md delete mode 100644 content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/_index.md delete mode 100644 content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/config-options.md delete mode 100644 content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/setting-up-template.md delete mode 100644 content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/azure.md delete mode 100644 content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/google-cloud-platform.md rename content/enterprise_influxdb/v1.9/introduction/{install-and-deploy => }/installation/_index.md (55%) rename content/enterprise_influxdb/v1.9/introduction/{install-and-deploy => }/installation/chrono_install.md (93%) rename content/enterprise_influxdb/v1.9/introduction/{install-and-deploy => }/installation/data_node_installation.md (99%) rename content/enterprise_influxdb/v1.9/introduction/{install-and-deploy => }/installation/meta_node_installation.md (99%) diff --git a/content/enterprise_influxdb/v1.8/install-and-deploy/production_installation/_index.md b/content/enterprise_influxdb/v1.8/install-and-deploy/production_installation/_index.md index d10d3461f..bcab1bfb3 100644 --- a/content/enterprise_influxdb/v1.8/install-and-deploy/production_installation/_index.md +++ b/content/enterprise_influxdb/v1.8/install-and-deploy/production_installation/_index.md @@ -15,5 +15,3 @@ Complete the following steps to install an InfluxDB Enterprise cluster in your o 1. [Install InfluxDB Enterprise meta nodes](/enterprise_influxdb/v1.8/install-and-deploy/production_installation/meta_node_installation/) 2. [Install InfluxDB data nodes](/enterprise_influxdb/v1.8/install-and-deploy/production_installation/data_node_installation/) 3. [Install Chronograf](/enterprise_influxdb/v1.8/install-and-deploy/production_installation/chrono_install/) - -> **Note:** If you're looking for cloud infrastructure and services, check out how to deploy InfluxDB Enterprise (production-ready) on a cloud provider of your choice: [Azure](/enterprise_influxdb/v1.8/install-and-deploy/deploying/azure/), [GCP](/enterprise_influxdb/v1.8/install-and-deploy/deploying/google-cloud-platform/), or [AWS](/enterprise_influxdb/v1.8/install-and-deploy/deploying/aws/). diff --git a/content/enterprise_influxdb/v1.9/about-the-project/release-notes-changelog.md b/content/enterprise_influxdb/v1.9/about-the-project/release-notes-changelog.md index 2c062dad3..4c8c16880 100644 --- a/content/enterprise_influxdb/v1.9/about-the-project/release-notes-changelog.md +++ b/content/enterprise_influxdb/v1.9/about-the-project/release-notes-changelog.md @@ -9,6 +9,52 @@ menu: parent: About the project --- +## 1.9.6 [2022-02-16] + +{{% note %}} InfluxDB Enterprise offerings are no longer available on AWS, Azure, and GCP marketplaces. Please [contact Sales](https://www.influxdata.com/contact-sales/) to request an license key to [install InfluxDB Enterprise in your own environment](/enterprise_influxdb/v1.9/introduction/installation/). +{{% /note %}} + +### Features + +#### Backup enhancements + +- **Revert damaged meta nodes to a previous state**: Add the `-meta-only-overwrite-force` option to [`influxd-ctl restore`](/enterprise_influxdb/v1.9/tools/influxd-ctl/#restore) to revert damaged meta nodes in an existing cluster to a previous state when restoring an InfluxDB Enterprise database. + +- **Estimate the size of a backup** (full or incremental) and provide progress messages. Add `-estimate` option to [`influxd-ctl backup`](/enterprise_influxdb/v1.9/tools/influxd-ctl/#backup) to estimate the size of a backup (full or incremental) and provide progress messages. Prints the number of files to back up, the percentage of bytes transferred for each file (organized by shard), and the estimated time remaining to complete the backup. + +#### Logging enhancements + +- **Log active queries when a process is terminated**: Add the [`termination-query-log`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#termination-query-log--false) configuration option. When set to `true` all running queries are printed to the log when a data node process receives a `SIGTERM` (for example, a Kubernetes process exceeds the container memory limit or the process is terminated). + +- **Log details of HTTP calls to meta nodes**. When [`cluster-tracing`](/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes/#cluster-tracing--false) is enabled, all API calls to meta nodes are now logged with details providing an audit trail including IP address of caller, specific API being invoked, action being invoked, and more. + +### Maintenance updates + +- Update to [Flux v0.140](/flux/v0.x/release-notes/#v01400-2021-11-22). +- Upgrade to Go 1.17. +- Upgrade `protobuf` library. + +### Bug fixes + +#### Data + +- Adjust shard start and end times to avoid overlaps in existing shards. This resolves issues with existing shards (truncated or not) that have a different shard duration than the current default. +- `DROP SHARD` now successfully ignores "shard not found errors." + +#### Errors + +- Fix panic when running `influxd config`. +- Ensure `influxd-ctl entropy` commands use the correct TLS settings. + +#### Profiling + +- Resolve issue to enable [mutex profiling](/enterprise_influxdb/v1.9/tools/api/#debugpprof-http-endpoint). + +#### influx-ctl updates + +- Improve [`influxd-ctl join`](/enterprise_influxdb/v1.9/tools/influxd-ctl/#join) robustness and provide better error messages on failure. +- Add user friendly error message when accessing a TLS-enabled server without TLS enabled on client. + ## v1.9.5 [2021-10-11] {{% note %}} @@ -67,7 +113,7 @@ Changes below are included in InfluxDB Enterprise 1.9.5. - Add [configurable password hashing](/enterprise_influxdb/v1.9/administration/configure-password-hashing/) with `bcrypt` and `pbkdf2` support. - Add retry with exponential back-off to anti-entropy repair. - Add logging to compaction. -- Add [`total-buffer-bytes`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#total-buffer-bytes--0) configuration parameter to subscriptions. +- Add [`total-buffer-bytes`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#total-buffer-bytes) configuration parameter to subscriptions. This option is intended to help alleviate out-of-memory errors. - Update to [Flux v0.120.1.](/influxdb/v2.0/reference/release-notes/flux/#v01201-2021-07-06) @@ -99,7 +145,7 @@ in that there is no corresponding InfluxDB OSS release. These queries now return a `cardinality estimation` column header where before they returned `count`. - Improve diagnostics for license problems. Add [license expiration date](/enterprise_influxdb/v1.9/features/clustering-features/#entitlements) to `debug/vars` metrics. -- Add improved [ingress metrics](/enterprise_influxdb/v1.9/administration/config-data-nodes/#ingress-metric-by-measurement-enabled--false) to track points written by measurement and by login. +- Add improved [ingress metrics](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#ingress-metric-by-measurement-enabled) to track points written by measurement and by login. Allow for collection of statistics regarding points, values, and new series written per measurement and by login. This data is collected and exposed at the data node level. With these metrics you can, for example: @@ -107,7 +153,7 @@ in that there is no corresponding InfluxDB OSS release. monitor the growth of series within a measurement, and track what user credentials are being used to write data. - Support authentication for Kapacitor via LDAP. -- Support for [configuring Flux query resource usage](/enterprise_influxdb/v1.9/administration/config-data-nodes/#flux-controller) (concurrency, memory, etc.). +- Support for [configuring Flux query resource usage](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#flux-controller) (concurrency, memory, etc.). - Upgrade to [Flux v0.113.0](/influxdb/v2.0/reference/release-notes/flux/#v01130-2021-04-21). - Update Prometheus remote protocol to allow streamed reading. - Improve performance of sorted merge iterator. diff --git a/content/enterprise_influxdb/v1.9/administration/backup-and-restore.md b/content/enterprise_influxdb/v1.9/administration/backup-and-restore.md index 8e0441e1b..070539d7f 100644 --- a/content/enterprise_influxdb/v1.9/administration/backup-and-restore.md +++ b/content/enterprise_influxdb/v1.9/administration/backup-and-restore.md @@ -176,9 +176,9 @@ $ ls ./telegrafbackup 20160803T222811Z.manifest 20160803T222811Z.meta 20160803T222811Z.s4.tar.gz ``` -#### Perform a metastore only backup +#### Perform a metadata only backup -Perform a meta store only backup into a specific directory with the command below. +Perform a metadata only backup into a specific directory with the command below. The directory must already exist. ```bash @@ -316,8 +316,8 @@ Restored from my-incremental-backup/ in 83.892591ms, transferred 588800 bytes ##### Restore from a metadata backup -In this example, the `restore` command restores an metadata backup stored -in the `metadata-backup/` directory. +In this example, the `restore` command restores a [metadata backup](#perform-a-metadata-only-backup) +stored in the `metadata-backup/` directory. ```bash # Syntax @@ -402,6 +402,29 @@ time written 1970-01-01T00:00:00Z 471 ``` +##### Restore (overwrite) metadata from a full or incremental backup to fix damaged metadata + +1. Identify a backup with uncorrupted metadata from which to restore. +2. Restore from backup with `-meta-only-overwrite-force`. + + {{% warn %}} + Only use the `-meta-only-overwrite-force` flag to restore from backups of the target cluster. + If you use this flag with metadata from a different cluster, you will lose data. + (since metadata includes shard assignments to data nodes). + {{% /warn %}} + + ```bash + # Syntax + influxd-ctl restore -meta-only-overwrite-force + + # Example + $ influxd-ctl restore -meta-only-overwrite-force my-incremental-backup/ + Using backup directory: my-incremental-backup/ + Using meta backup: 20200101T000000Z.meta + Restoring meta data... Done. Restored in 21.373019ms, 1 shards mapped + Restored from my-incremental-backup/ in 19.2311ms, transferred 588 bytes + ``` + #### Common issues with restore ##### Restore writes information not part of the original backup diff --git a/content/enterprise_influxdb/v1.9/administration/configure/anti-entropy/_index.md b/content/enterprise_influxdb/v1.9/administration/configure/anti-entropy/_index.md index eab4bb7ef..3f8bb3b87 100644 --- a/content/enterprise_influxdb/v1.9/administration/configure/anti-entropy/_index.md +++ b/content/enterprise_influxdb/v1.9/administration/configure/anti-entropy/_index.md @@ -34,7 +34,7 @@ If data inconsistencies are detected among shards in a shard group, [invoke the In the repair process, the Anti-Entropy service will sync the necessary updates from other shards within a shard group. -By default, the service performs consistency checks every 5 minutes. This interval can be modified in the [`anti-entropy.check-interval`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#check-interval-5m) configuration setting. +By default, the service performs consistency checks every 5 minutes. This interval can be modified in the [`anti-entropy.check-interval`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#check-interval) configuration setting. The Anti-Entropy service can only address missing or inconsistent shards when there is at least one copy of the shard available. @@ -178,7 +178,7 @@ until it either shows as being in the queue, being repaired, or no longer in the ## Configuration -The configuration settings for the Anti-Entropy service are described in [Anti-Entropy settings](/enterprise_influxdb/v1.9/administration/config-data-nodes#anti-entropy) section of the data node configuration. +The configuration settings for the Anti-Entropy service are described in [Anti-Entropy settings](/enterprise_influxdb/v1.9/administration/config-data-nodes/#anti-entropy-ae-settings) section of the data node configuration. To enable the Anti-Entropy service, change the default value of the `[anti-entropy].enabled = false` setting to `true` in the `influxdb.conf` file of each of your data nodes. diff --git a/content/enterprise_influxdb/v1.9/administration/configure/config-data-nodes.md b/content/enterprise_influxdb/v1.9/administration/configure/config-data-nodes.md index 1eb0dc409..a9249f6e4 100644 --- a/content/enterprise_influxdb/v1.9/administration/configure/config-data-nodes.md +++ b/content/enterprise_influxdb/v1.9/administration/configure/config-data-nodes.md @@ -49,25 +49,33 @@ All commented-out settings will be determined by the internal defaults. ## Global settings -#### `reporting-disabled = false` +#### `reporting-disabled` + +Default is `false`. Once every 24 hours InfluxDB Enterprise will report usage data to usage.influxdata.com. The data includes a random ID, os, arch, version, the number of series and other usage data. No data from user databases is ever transmitted. Change this option to true to disable reporting. -#### `bind-address = ":8088"` +#### `bind-address` + +Default is `":8088"`. The TCP bind address used by the RPC service for inter-node communication and [backup and restore](/enterprise_influxdb/v1.9/administration/backup-and-restore/). Environment variable: `INFLUXDB_BIND_ADDRESS` -#### `hostname = "localhost"` +#### `hostname` + +Default is `"localhost"`. The hostname of the [data node](/enterprise_influxdb/v1.9/concepts/glossary/#data-node). This must be resolvable by all other nodes in the cluster. Environment variable: `INFLUXDB_HOSTNAME` -#### `gossip-frequency = "3s"` +#### `gossip-frequency` + +Default is `"3s"`. How often to update the cluster with this node's internal status. @@ -81,7 +89,9 @@ Environment variable: `INFLUXDB_GOSSIP_FREQUENCY` The `[enterprise]` section contains the parameters for the meta node's registration with the [InfluxDB Enterprise License Portal](https://portal.influxdata.com/). -#### `license-key = ""` +#### `license-key` + +Default is `""`. The license key created for you on [InfluxPortal](https://portal.influxdata.com). The meta node transmits the license key to [portal.influxdata.com](https://portal.influxdata.com) over port 80 or port 443 and receives a temporary JSON license file in return. The server caches the license file locally. @@ -98,7 +108,9 @@ mutually exclusive and one must remain set to the empty string. Environment variable: `INFLUXDB_ENTERPRISE_LICENSE_KEY` -#### `license-path = ""` +#### `license-path` + +Default is `""`. The local path to the permanent JSON license file that you received from InfluxData for instances that do not have access to the internet. The data process will only function for a limited time without a valid license file. @@ -124,7 +136,9 @@ Environment variable: `INFLUXDB_ENTERPRISE_LICENSE_PATH` Settings related to how the data nodes interact with the meta nodes. -#### `dir = "/var/lib/influxdb/meta"` +#### `dir` + +Default is `"/var/lib/influxdb/meta"`. The directory where the cluster metadata is stored. @@ -132,39 +146,50 @@ The directory where the cluster metadata is stored. Environment variable: `INFLUXDB_META_DIR` -#### `meta-tls-enabled = false` +#### `meta-tls-enabled` + +Default is `false`. Whether to use TLS when connecting to meta nodes. -Set to `true` to if [`https-enabled`](#https-enabled-false) is set to `true`. +Set to `true` to if [`https-enabled`](#https-enabled) is set to `true`. Environment variable: `INFLUXDB_META_META_TLS_ENABLED` -#### `meta-insecure-tls = false` +#### `meta-insecure-tls` + +Default is `false`. Allows insecure TLS connections to meta nodes. This is useful when testing with self-signed certificates. -Set to `true` to allow the data node to accept self-signed certificates if [`https-enabled`](#https-enabled-false) is set to `true`. +Set to `true` to allow the data node to accept self-signed certificates if [`https-enabled`](#https-enabled) is set to `true`. Environment variable: `INFLUXDB_META_META_INSECURE_TLS` -#### `meta-auth-enabled = false` +#### `meta-auth-enabled` + +Default is `false`. This setting must have the same value as the meta nodes' `[meta] auth-enabled` configuration. -Set to `true` if [`auth-enabled`](#auth-enabled-false) is set to `true` in the meta node configuration files. +Set to `true` if [`auth-enabled`](#auth-enabled) is set to `true` in the meta node configuration files. For JWT authentication, also see the [`meta-internal-shared-secret`](#meta-internal-shared-secret) configuration option. Environment variable: `INFLUXDB_META_META_AUTH_ENABLED` -#### `meta-internal-shared-secret = ""` +#### `meta-internal-shared-secret` + +Default is `""`. The shared secret used by the internal API for JWT authentication between InfluxDB nodes. -This value must be the same as the [`internal-shared-secret`](/enterprise_influxdb/v1.9/administration/config-meta-nodes/#internal-shared-secret) specified in the meta node configuration file. +This value must be the same as the [`internal-shared-secret`](/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes/#internal-shared-secret) +specified in the meta node configuration file. Environment variable: `INFLUXDB_META_META_INTERNAL_SHARED_SECRET` -#### `retention-autocreate = true` +#### `retention-autocreate` + +Default is `true`. Automatically creates a default [retention policy](/enterprise_influxdb/v1.9/concepts/glossary/#retention-policy-rp) (RP) when the system creates a database. The default RP (`autogen`) has an infinite duration, a shard group duration of seven days, and a replication factor set to the number of data nodes in the cluster. @@ -173,28 +198,34 @@ Set this option to `false` to prevent the system from creating the `autogen` RP Environment variable: `INFLUXDB_META_RETENTION_AUTOCREATE` -#### `logging-enabled = true` +#### `logging-enabled` + +Default is `true`. Whether log messages are printed for the meta service. Environment variable: `INFLUXDB_META_LOGGING_ENABLED` -#### `password-hash = bcrypt` +#### `password-hash` + +Default is `bcrypt`. Configures password hashing algorithm. Supported options are: `bcrypt` (the default), `pbkdf2-sha256`, and `pbkdf2-sha512` -This setting must have the same value as the meta node option [`meta.password-hash`](/enterprise_influxdb/v1.9/administration/config-meta-nodes/#password-hash--bcrypt). +This setting must have the same value as the meta node option [`meta.password-hash`](/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes/#password-hash). -For detailed configuration information, see [`meta.password-hash`](/enterprise_influxdb/v1.9/administration/config-meta-nodes/#password-hash--bcrypt). +For detailed configuration information, see [`meta.password-hash`](/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes/#password-hash). Environment variable: `INFLUXDB_META_PASSWORD_HASH` -#### `ensure-fips = false` +#### `ensure-fips` + +Default is `false`. When `true`, enables a FIPS-readiness check on startup. Default is `false`. -For detailed configuration information, see [`meta.ensure-fips`](/enterprise_influxdb/v1.9/administration/config-meta-nodes/#ensure-fips--false). +For detailed configuration information, see [`meta.ensure-fips`](/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes/#ensure-fips). Environment variable: `INFLUXDB_META_ENSURE_FIPS` @@ -208,33 +239,43 @@ Controls where the actual shard data for InfluxDB lives and how it is compacted "dir" may need to be changed to a suitable place for your system. The defaults should work for most systems. -#### `dir = "/var/lib/influxdb/data"` +#### `dir` + +Default is `"/var/lib/influxdb/data"`. The directory where the TSM storage engine stores TSM (read-optimized) files. Environment variable: `INFLUXDB_DATA_DIR` -#### `wal-dir = "/var/lib/influxdb/wal"` +#### `wal-dir` + +Default is `"/var/lib/influxdb/wal"`. The directory where the TSM storage engine stores WAL (write-optimized) files. Environment variable: `INFLUXDB_DATA_WAL_DIR` -#### `trace-logging-enabled = false` +#### `trace-logging-enabled` + +Default is `false`. Trace logging provides more verbose output around the TSM engine. Turning this on can provide more useful output for debugging TSM engine issues. Environmental variable: `INFLUXDB_DATA_TRACE_LOGGING_ENABLED` -#### `query-log-enabled = true` +#### `query-log-enabled` + +Default is `true`. Whether queries should be logged before execution. Very useful for troubleshooting, but will log any sensitive data contained within a query. Environment variable: `INFLUXDB_DATA_QUERY_LOG_ENABLED` -#### `wal-fsync-delay = "0s"` +#### `wal-fsync-delay` + +Default is `"0s"`. The amount of time that a write waits before fsyncing. Use a duration greater than 0 to batch up multiple fsync calls. @@ -244,14 +285,18 @@ InfluxData recommends values ranging from `0ms` to `100ms` for non-SSD disks. Environment variable: `INFLUXDB_DATA_WAL_FSYNC_DELAY` -#### `ingress-metric-by-measurement-enabled = false` +#### `ingress-metric-by-measurement-enabled` + +Default is `false`. When `true`, collect statistics of points, values and new series written per-measurement. Metrics are gathered per data node. These can be accessed via the `/debug/vars` endpoint and in the `_internal` database if enabled. Environment variable: `INFLUXDB_DATA_INGRESS_METRIC_BY_MEASUREMENT_ENABLED` -#### `ingress-metric-by-login-enabled = false` +#### `ingress-metric-by-login-enabled` + +Default is `false`. When `true`, collect statistics of points, values and new series written per-login. Metrics are gathered per data node. These can be accessed via the `/debug/vars` endpoint and in the `_internal` database if enabled. @@ -260,25 +305,33 @@ Environment variable: `INFLUXDB_DATA_INGRESS_METRIC_BY_LOGIN_ENABLED` ### Data settings for the TSM engine -#### `cache-max-memory-size = "1g"` +#### `cache-max-memory-size` -The maximum size a shard cache can reach before it starts rejecting writes. +Default is `1000000000`. + +The maximum size in bytes that a shard cache can reach before it starts rejecting writes. Environment variable: `INFLUXDB_DATA_CACHE_MAX_MEMORY_SIZE` -#### `cache-snapshot-memory-size = "25m"` +#### `cache-snapshot-memory-size` -The size at which the TSM engine will snapshot the cache and write it to a TSM file, freeing up memory. +Default is `26214400`. + +The size in bytes at which the TSM engine will snapshot the cache and write it to a TSM file, freeing up memory. Environment variable: `INFLUXDB_DATA_CACHE_SNAPSHOT_MEMORY_SIZE` -#### `cache-snapshot-write-cold-duration = "10m"` +#### `cache-snapshot-write-cold-duration` + +Default is `"10m"`. The length of time at which the TSM engine will snapshot the cache and write it to a new TSM file if the shard hasn't received writes or deletes. Environment variable: `INFLUXDB_DATA_CACHE_SNAPSHOT_WRITE_COLD_DURATION` -#### `max-concurrent-compactions = 0` +#### `max-concurrent-compactions` + +Default is `0`. The maximum number of concurrent full and level compactions that can run at one time. A value of `0` (unlimited compactions) results in 50% of `runtime.GOMAXPROCS(0)` used at runtime, @@ -288,26 +341,35 @@ This setting does not apply to cache snapshotting. Environmental variable: `INFLUXDB_DATA_CACHE_MAX_CONCURRENT_COMPACTIONS` -#### `compact-throughput = "48m"` +#### `compact-throughput` + +Default is `50331648`. The maximum number of bytes per seconds TSM compactions write to disk. Default is `"48m"` (48 million). Note that short bursts are allowed to happen at a possibly larger value, set by `compact-throughput-burst`. Environment variable: `INFLUXDB_DATA_COMPACT_THROUGHPUT` -#### `compact-throughput-burst = "48m"` + +#### `compact-throughput-burst` + +Default is `50331648`. The maximum number of bytes per seconds TSM compactions write to disk during brief bursts. Default is `"48m"` (48 million). Environment variable: `INFLUXDB_DATA_COMPACT_THROUGHPUT_BURST` -#### `compact-full-write-cold-duration = "4h"` +#### `compact-full-write-cold-duration` + +Default is `"4h"`. The duration at which to compact all TSM and TSI files in a shard if it has not received a write or delete. Environment variable: `INFLUXDB_DATA_COMPACT_FULL_WRITE_COLD_DURATION` -#### `index-version = "inmem"` +#### `index-version` + +Default is `"inmem"`. The type of shard index to use for new shards. The default (`inmem`) is to use an in-memory index that is recreated at startup. @@ -318,7 +380,9 @@ Environment variable: `INFLUXDB_DATA_INDEX_VERSION` ### In-memory (`inmem`) index settings -#### `max-series-per-database = 1000000` +#### `max-series-per-database` + +Default is `1000000`. The maximum number of [series](/enterprise_influxdb/v1.9/concepts/glossary/#series) allowed per database before writes are dropped. The default setting is `1000000` (one million). @@ -338,7 +402,9 @@ If a point causes the number of series in a database to exceed Environment variable: `INFLUXDB_DATA_MAX_SERIES_PER_DATABASE` -#### `max-values-per-tag = 100000` +#### `max-values-per-tag` + +Default is `100000`. The maximum number of [tag values](/enterprise_influxdb/v1.9/concepts/glossary/#tag-value) allowed per [tag key](/enterprise_influxdb/v1.9/concepts/glossary/#tag-key). The default value is `100000` (one hundred thousand). @@ -356,7 +422,9 @@ Environment variable: `INFLUXDB_DATA_MAX_VALUES_PER_TAG` ### TSI (`tsi1`) index settings -#### `max-index-log-file-size = "1m"` +#### `max-index-log-file-size` + +Default is `1048576`. The threshold, in bytes, when an index write-ahead log (WAL) file will compact into an index file. Lower sizes will cause log files to be compacted more @@ -368,7 +436,9 @@ Values without a size suffix are in bytes. Environment variable: `INFLUXDB_DATA_MAX_INDEX_LOG_FILE_SIZE` -#### `series-id-set-cache-size = 100` +#### `series-id-set-cache-size` + +Default is `100`. The size of the internal cache used in the TSI index to store previously calculated series results. Cached results will be returned quickly from the cache rather @@ -400,14 +470,18 @@ a single-use TCP connection may be used. For information on InfluxDB `_internal` measurement statistics related to clusters, RPCs, and shards, see [Measurements for monitoring InfluxDB Enterprise (`_internal`)](/platform/monitoring/influxdata-platform/tools/measurements-internal/#cluster-enterprise-only). -#### `dial-timeout = "1s"` +#### `dial-timeout` + +Default is `"1s"`. The duration for which the meta node waits for a connection to a remote data node before the meta node attempts to connect to a different remote data node. This setting applies to queries only. Environment variable: `INFLUXDB_CLUSTER_DIAL_TIMEOUT` -#### `pool-max-idle-time = "60s"` +#### `pool-max-idle-time` + +Default is `"60s"`. The maximum time that a TCP connection to another data node remains idle in the connection pool. When the connection is idle longer than the specified duration, the inactive connection is reaped — @@ -416,7 +490,9 @@ idle connections minimizes inactive connections, decreases system load, and prev Environment variable: `INFLUXDB_CLUSTER_POOL_MAX_IDLE_TIME` -#### `pool-max-idle-streams = 100` +#### `pool-max-idle-streams` + +Default is `100`. The maximum number of idle RPC stream connections to retain in an idle pool between two nodes. When a new RPC request is issued, a connection is temporarily pulled from the idle pool, used, and then returned. @@ -428,7 +504,9 @@ so it is unlikely that changing this value will measurably improve performance b Environment variable: `INFLUXDB_CLUSTER_POOL_MAX_IDLE_STREAMS` -#### `allow-out-of-order-writes = false` +#### `allow-out-of-order-writes` + +Default is `false`. By default, this option is set to false and writes are processed in the order that they are received. This means if any points are in the hinted handoff (HH) queue for a shard, all incoming points must go into the HH queue. @@ -442,33 +520,45 @@ Point 1 (`cpu v=1.0 1234`) arrives at `node1`, attempts to replicate on `node2`, Environment variable: `INFLUXDB_CLUSTER_ALLOW_OUT_OF_ORDER` -#### `shard-reader-timeout = "0"` +#### `shard-reader-timeout` + +Default is `"0"`. The default timeout set on shard readers. The time in which a query connection must return its response after which the system returns an error. Environment variable: `INFLUXDB_CLUSTER_SHARD_READER_TIMEOUT` -#### `https-enabled = false` +#### `https-enabled` + +Default is `false`. Determines whether data nodes use HTTPS to communicate with each other. -#### `https-certificate = ""` +#### `https-certificate` + +Default is `""`. The SSL certificate to use when HTTPS is enabled. The certificate should be a PEM-encoded bundle of the certificate and key. If it is just the certificate, a key must be specified in `https-private-key`. -#### `https-private-key = ""` +#### `https-private-key` + +Default is `""`. Use a separate private key location. -#### `https-insecure-tls = false` +#### `https-insecure-tls` + +Default is `false`. Whether data nodes will skip certificate validation communicating with each other over HTTPS. This is useful when testing with self-signed certificates. -#### `cluster-tracing = false` +#### `cluster-tracing` + +Default is `false`. Enables cluster trace logging. Set to `true` to enable logging of cluster communications. @@ -476,13 +566,17 @@ Enable this setting to verify connectivity issues between data nodes. Environment variable: `INFLUXDB_CLUSTER_CLUSTER_TRACING` -#### `write-timeout = "10s"` +#### `write-timeout` + +Default is `"10s"`. The duration a write request waits until a "timeout" error is returned to the caller. The default value is 10 seconds. Environment variable: `INFLUXDB_CLUSTER_WRITE_TIMEOUT` -#### `max-concurrent-queries = 0` +#### `max-concurrent-queries` + +Default is `0`. The maximum number of concurrent queries allowed to be executing at one time. If a query is executed and exceeds this limit, an error is returned to the caller. @@ -490,14 +584,26 @@ This limit can be disabled by setting it to `0`. Environment variable: `INFLUXDB_CLUSTER_MAX_CONCURRENT_QUERIES` -#### `query-timeout = "0s"` +#### `max-concurrent-deletes` + +The default is `1`. + +The maximum number of allowed simultaneous `DELETE` calls on a shard. + +Environment variable: `INFLUXDB_CLUSTER_MAX_CONCURRENT_DELETES` + +#### `query-timeout` + +Default is `"0s"`. The maximum time a query is allowed to execute before being killed by the system. This limit can help prevent run away queries. Setting the value to `0` disables the limit. Environment variable: `INFLUXDB_CLUSTER_QUERY_TIMEOUT` -#### `log-queries-after = "0s"` +#### `log-queries-after` + +Default is `"0s"`. The time threshold when a query will be logged as a slow query. This limit can be set to help discover slow or resource intensive queries. @@ -505,27 +611,39 @@ Setting the value to `0` disables the slow query logging. Environment variable: `INFLUXDB_CLUSTER_LOG_QUERIES_AFTER` -#### `max-select-point = 0` +#### `max-select-point` + +Default is `0`. The maximum number of points a SELECT statement can process. A value of `0` will make the maximum point count unlimited. Environment variable: `INFLUXDB_CLUSTER_MAX_SELECT_POINT` -#### `max-select-series = 0` +#### `max-select-series` + +Default is `0`. The maximum number of series a SELECT can run. A value of `0` will make the maximum series count unlimited. Environment variable: `INFLUXDB_CLUSTER_MAX_SELECT_SERIES` -#### `max-select-buckets = 0` +#### `max-select-buckets` + +Default is `0`. The maximum number of group by time buckets a SELECT can create. A value of `0` will make the maximum number of buckets unlimited. Environment variable: `INFLUXDB_CLUSTER_MAX_SELECT_BUCKETS` +#### `termination-query-log = false` + +Set to `true` to print all running queries to the log when a data node process receives a `SIGTERM` (for example, a k8s process exceeds the container memory limit or the process is terminated). + +Environment variable: `INFLUXDB_CLUSTER_TERMINATION_QUERY_LOG` + ----- ## Hinted Handoff settings @@ -534,32 +652,42 @@ Environment variable: `INFLUXDB_CLUSTER_MAX_SELECT_BUCKETS` Controls the hinted handoff (HH) queue, which allows data nodes to temporarily cache writes destined for another data node when that data node is unreachable. -#### `batch-size = 512000` +#### `batch-size` + +Default is `512000`. The maximum number of bytes to write to a shard in a single request. Environment variable: `INFLUXDB_HINTED_HANDOFF_BATCH_SIZE` -#### `max-pending-writes = 1024` +#### `max-pending-writes` + +Default is `1024`. The maximum number of incoming pending writes allowed in the hinted handoff queue. Environment variable: `INFLUXDB_HINTED_HANDOFF_MAX_PENDING_WRITES` -#### `dir = "/var/lib/influxdb/hh"` +#### `dir` + +Default is `"/var/lib/influxdb/hh"`. The hinted handoff directory where the durable queue will be stored on disk. Environment variable: `INFLUXDB_HINTED_HANDOFF_DIR` -#### `enabled = true` +#### `enabled` + +Default is `true`. Set to `false` to disable hinted handoff. Disabling hinted handoff is not recommended and can lead to data loss if another data node is unreachable for any length of time. Environment variable: `INFLUXDB_HINTED_HANDOFF_ENABLED` -#### `max-size = 10737418240` +#### `max-size` + +Default is `10737418240`. The maximum size of the hinted handoff queue in bytes. Each queue is for one and only one other data node in the cluster. @@ -567,7 +695,9 @@ If there are N data nodes in the cluster, each data node may have up to N-1 hint Environment variable: `INFLUXDB_HINTED_HANDOFF_MAX_SIZE` -#### `max-age = "168h0m0s"` +#### `max-age` + +Default is `"168h0m0s"`. The time interval that writes sit in the queue before they are purged. The time is determined by how long the batch has been in the queue, not by the timestamps in the data. @@ -575,7 +705,9 @@ If another data node is unreachable for more than the `max-age` it can lead to d Environment variable: `INFLUXDB_HINTED_HANDOFF_MAX_AGE` -#### `retry-concurrency = 20` +#### `retry-concurrency` + +Default is `20`. The maximum number of hinted handoff blocks that the source data node attempts to write to each destination data node. Hinted handoff blocks are sets of data that belong to the same shard and have the same destination data node. @@ -590,19 +722,25 @@ Note that increasing `retry-concurrency` also increases network traffic. Environment variable: `INFLUXDB_HINTED_HANDOFF_RETRY_CONCURRENCY` -#### `retry-rate-limit = 0` +#### `retry-rate-limit` + +Default is `0`. The rate limit (in bytes per second) that hinted handoff retries hints. A value of `0` disables the rate limit. Environment variable: `INFLUXDB_HINTED_HANDOFF_RETRY_RATE_LIMIT` -#### `retry-interval = "1s"` +#### `retry-interval` + +Default is `"1s"`. The time period after which the hinted handoff retries a write after the write fails. Environment variable: `INFLUXDB_HINTED_HANDOFF_RETRY_INTERVAL` -#### `retry-max-interval = "10s"` +#### `retry-max-interval` + +Default is `"10s"`. The maximum interval after which the hinted handoff retries a write after the write fails. The `retry-max-interval` option is no longer in use and will be removed from the configuration file in a future release. @@ -610,7 +748,9 @@ Changing the `retry-max-interval` setting has no effect on your cluster. Environment variable: `INFLUXDB_HINTED_HANDOFF_RETRY_MAX_INTERVAL` -#### `purge-interval = "1m0s"` +#### `purge-interval` + +Default is `"1m0s"`. The interval at which InfluxDB checks to purge data that are above `max-age`. @@ -626,20 +766,26 @@ For information about the Anti-Entropy service, see [Anti-entropy service in Inf Controls the copying and repairing of shards to ensure that data nodes contain the shard data they are supposed to. -#### `enabled = false` +#### `enabled` + +Default is `false`. Enables the anti-entropy service. Default value is `false`. Environment variable: `INFLUXDB_ANTI_ENTROPY_ENABLED` -#### `check-interval = "5m"` +#### `check-interval` + +Default is `"5m"`. The interval of time when anti-entropy checks run on each data node. Environment variable: `INFLUXDB_ANTI_ENTROPY_CHECK_INTERVAL` -#### `max-fetch = 10` +#### `max-fetch` + +Default is `10`. The maximum number of shards that a single data node will copy or repair in parallel. @@ -653,14 +799,18 @@ higher CPU load as new shard digest files are created. The added load drops off after shard digests are completed for existing shards. {{% /note %}} -#### `max-sync = 1` +#### `max-sync` + +Default is `1`. The maximum number of concurrent sync operations that should be performed. Modify this setting only when requested by InfluxData support. Environment variable: `INFLUXDB_ANTI_ENTROPY_MAX_SYNC` -#### `auto-repair-missing = true` +#### `auto-repair-missing` + +Default is `true`. Enables missing shards to automatically be repaired. @@ -674,14 +824,18 @@ Environment variable: `INFLUXDB_ANTI_ENTROPY_AUTO_REPAIR_MISSING` Controls the enforcement of retention policies for evicting old data. -#### `enabled = true` +#### `enabled` + +Default is `true`. Enables retention policy enforcement. Default value is `true`. Environment variable: `INFLUXDB_RETENTION_ENABLED` -#### `check-interval = "30m0s"` +#### `check-interval` + +Default is `"30m0s"`. The interval of time when retention policy enforcement checks run. @@ -696,19 +850,25 @@ Environment variable: `INFLUXDB_RETENTION_CHECK_INTERVAL` Controls the precreation of shards, so they are available before data arrives. Only shards that, after creation, will have both a start- and end-time in the future, will ever be created. Shards are never precreated that would be wholly or partially in the past. -#### `enabled = true` +#### `enabled` + +Default is `true`. Enables the shard precreation service. Environment variable: `INFLUXDB_SHARD_PRECREATION_ENABLED` -#### `check-interval = "10m"` +#### `check-interval` + +Default is `"10m"`. The interval of time when the check to precreate new shards runs. Environment variable: `INFLUXDB_SHARD_PRECREATION_CHECK_INTERVAL` -#### `advance-period = "30m"` +#### `advance-period` + +Default is `"30m"`. The default period ahead of the end time of a shard group that its successor group is created. @@ -730,25 +890,33 @@ For InfluxDB Enterprise production systems, InfluxData recommends including a de * On the dedicated InfluxDB monitoring instance, set `store-enabled = false` to avoid potential performance and storage issues. * On each InfluxDB cluster node, install a Telegraf input plugin and Telegraf output plugin configured to report data to the dedicated InfluxDB monitoring instance. -#### `store-enabled = true` +#### `store-enabled` + +Default is `true`. Enables the internal storage of statistics. Environment variable: `INFLUXDB_MONITOR_STORE_ENABLED` -#### `store-database = "_internal"` +#### `store-database` + +Default is `"_internal"`. The destination database for recorded statistics. Environment variable: `INFLUXDB_MONITOR_STORE_DATABASE` -#### `store-interval = "10s"` +#### `store-interval` + +Default is `"10s"`. The interval at which to record statistics. Environment variable: `INFLUXDB_MONITOR_STORE_INTERVAL` -#### `remote-collect-interval = "10s"` +#### `remote-collect-interval` + +Default is `"10s"`. The time interval to poll other data nodes' stats when aggregating cluster stats. @@ -762,47 +930,63 @@ Environment variable: `INFLUXDB_MONITOR_REMOTE_COLLECT_INTERVAL` Controls how the HTTP endpoints are configured. These are the primary mechanism for getting data into and out of InfluxDB. -#### `enabled = true` +#### `enabled` + +Default is `true`. Enables HTTP endpoints. Environment variable: `INFLUXDB_HTTP_ENABLED` -#### `flux-enabled = false` +#### `flux-enabled` + +Default is `false`. Determines whether the Flux query endpoint is enabled. To enable the use of Flux queries, set the value to `true`. Environment variable: `INFLUXDB_HTTP_FLUX_ENABLED` -#### `bind-address = ":8086"` +#### `bind-address` + +Default is `":8086"`. The bind address used by the HTTP service. Environment variable: `INFLUXDB_HTTP_BIND_ADDRESS` -#### `auth-enabled = false` +#### `auth-enabled` + +Default is `false`. Enables HTTP authentication. Environment variable: `INFLUXDB_HTTP_AUTH_ENABLED` -#### `realm = "InfluxDB"` +#### `realm` + +Default is `"InfluxDB"`. The default realm sent back when issuing a basic authorization challenge. Environment variable: `INFLUXDB_HTTP_REALM` -#### `log-enabled = true` +#### `log-enabled` + +Default is `true`. Enables HTTP request logging. Environment variable: `INFLUXDB_HTTP_LOG_ENABLED` -#### `suppress-write-log = false` +#### `suppress-write-log` + +Default is `false`. Determines whether the HTTP write request logs should be suppressed when the log is enabled. -#### `access-log-path = ""` +#### `access-log-path` + +Default is `""`. The path to the access log, which determines whether detailed write logging is enabled using `log-enabled = true`. Specifies whether HTTP request logging is written to the specified path when enabled. @@ -813,7 +997,9 @@ If `influxd` is unable to access the specified path, it will log an error and fa Environment variable: `INFLUXDB_HTTP_ACCESS_LOG_PATH` -#### `access-log-status-filters = []` +#### `access-log-status-filters` + +Default is `[]`. Filters which requests should be logged. Each filter is of the pattern `nnn`, `nnx`, or `nxx` where `n` is a number and `x` is the wildcard for any number. @@ -843,26 +1029,34 @@ When using environment variables, the values can be supplied as follows. The `_n` at the end of the environment variable represents the array position of the entry. -#### `write-tracing = false` +#### `write-tracing` + +Default is `false`. Enables detailed write logging. Environment variable: `INFLUXDB_HTTP_WRITE_TRACING` -#### `pprof-enabled = true` +#### `pprof-enabled` + +Default is `true`. Determines whether the `/pprof` endpoint is enabled. This endpoint is used for troubleshooting and monitoring. Environment variable: `INFLUXDB_HTTP_PPROF_ENABLED` -#### `https-enabled = false` +#### `https-enabled` + +Default is `false`. Enables HTTPS. Environment variable: `INFLUXDB_HTTP_HTTPS_ENABLED` -#### `https-certificate = "/etc/ssl/influxdb.pem"` +#### `https-certificate` + +Default is `"/etc/ssl/influxdb.pem"`. The SSL certificate to use when HTTPS is enabled. The certificate should be a PEM-encoded bundle of the certificate and key. @@ -870,19 +1064,25 @@ If it is just the certificate, a key must be specified in `https-private-key`. Environment variable: `INFLUXDB_HTTP_HTTPS_CERTIFICATE` -#### `https-private-key = ""` +#### `https-private-key` + +Default is `""`. The location of the separate private key. Environment variable: `INFLUXDB_HTTP_HTTPS_PRIVATE_KEY` -#### `shared-secret = ""` +#### `shared-secret` + +Default is `""`. The JWT authorization shared secret used to validate requests using JSON web tokens (JWTs). Environment variable: `INFLUXDB_HTTP_SHARED_SECRET` -#### `max-body-size = 25000000` +#### `max-body-size` + +Default is `25000000`. The maximum size, in bytes, of a client request body. When a HTTP client sends data that exceeds the configured maximum size, a `413 Request Entity Too Large` HTTP response is returned. @@ -890,7 +1090,9 @@ To disable the limit, set the value to `0`. Environment variable: `INFLUXDB_HTTP_MAX_BODY_SIZE` -#### `max-row-limit = 0` +#### `max-row-limit` + +Default is `0`. The default chunk size for result sets that should be chunked. The maximum number of rows that can be returned in a non-chunked query. @@ -899,7 +1101,9 @@ InfluxDB includes a `"partial":true` tag in the response body if query results e Environment variable: `INFLUXDB_HTTP_MAX_ROW_LIMIT` -#### `max-connection-limit = 0` +#### `max-connection-limit` + +Default is `0`. The maximum number of HTTP connections that may be open at once. New connections that would exceed this limit are dropped. @@ -907,33 +1111,43 @@ The default value of `0` disables the limit. Environment variable: `INFLUXDB_HTTP_MAX_CONNECTION_LIMIT` -#### `unix-socket-enabled = false` +#### `unix-socket-enabled` + +Default is `false`. Enables the HTTP service over the UNIX domain socket. Environment variable: `INFLUXDB_HTTP_UNIX_SOCKET_ENABLED` -#### `bind-socket = "/var/run/influxdb.sock"` +#### `bind-socket` + +Default is `"/var/run/influxdb.sock"`. The path of the UNIX domain socket. Environment variable: `INFLUXDB_HTTP_BIND_SOCKET` -#### `max-concurrent-write-limit = 0` +#### `max-concurrent-write-limit` + +Default is `0`. The maximum number of writes processed concurrently. The default value of `0` disables the limit. Environment variable: `INFLUXDB_HTTP_MAX_CONCURRENT_WRITE_LIMIT` -#### `max-enqueued-write-limit = 0` +#### `max-enqueued-write-limit` + +Default is `0`. The maximum number of writes queued for processing. The default value of `0` disables the limit. Environment variable: `INFLUXDB_HTTP_MAX_ENQUEUED_WRITE_LIMIT` -#### `enqueued-write-timeout = 0` +#### `enqueued-write-timeout` + +Default is `0`. The maximum duration for a write to wait in the queue to be processed. Setting this to `0` or setting `max-concurrent-write-limit` to `0` disables the limit. @@ -944,7 +1158,9 @@ Setting this to `0` or setting `max-concurrent-write-limit` to `0` disables the ### `[logging]` -#### `format = "logfmt"` +#### `format` + +Default is `"logfmt"`. Determines which log encoder to use for logs. Valid options are `auto`, `logfmt`, and `json`. @@ -953,13 +1169,17 @@ When the output is a non-TTY, `auto` will use `logfmt`. Environment variable: `INFLUXDB_LOGGING_FORMAT` -#### `level = "info"` +#### `level` + +Default is `"info"`. Determines which level of logs will be emitted. Environment variable: `INFLUXDB_LOGGING_LEVEL` -#### `suppress-logo = false` +#### `suppress-logo` + +Default is `false`. Suppresses the logo output that is printed when the program is started. @@ -973,45 +1193,59 @@ Environment variable: `INFLUXDB_LOGGING_SUPPRESS_LOGO` Controls the subscriptions, which can be used to fork a copy of all data received by the InfluxDB host. -#### `enabled = true` +#### `enabled` + +Default is `true`. Determines whether the subscriber service is enabled. Environment variable: `INFLUXDB_SUBSCRIBER_ENABLED` -#### `http-timeout = "30s"` +#### `http-timeout` + +Default is `"30s"`. The default timeout for HTTP writes to subscribers. Environment variable: `INFLUXDB_SUBSCRIBER_HTTP_TIMEOUT` -#### `insecure-skip-verify = false` +#### `insecure-skip-verify` + +Default is `false`. Allows insecure HTTPS connections to subscribers. This option is useful when testing with self-signed certificates. Environment variable: `INFLUXDB_SUBSCRIBER_INSECURE_SKIP_VERIFY` -#### `ca-certs = ""` +#### `ca-certs` + +Default is `""`. The path to the PEM-encoded CA certs file. If the set to the empty string (`""`), the default system certs will used. Environment variable: `INFLUXDB_SUBSCRIBER_CA_CERTS` -#### `write-concurrency = 40` +#### `write-concurrency` + +Default is `40`. The number of writer Goroutines processing the write channel. Environment variable: `INFLUXDB_SUBSCRIBER_WRITE_CONCURRENCY` -#### `write-buffer-size = 1000` +#### `write-buffer-size` + +Default is `1000`. The number of in-flight writes buffered in the write channel. Environment variable: `INFLUXDB_SUBSCRIBER_WRITE_BUFFER_SIZE` -#### `total-buffer-bytes = 0` +#### `total-buffer-bytes` + +Default is `0`. Total number of bytes allocated to buffering across all subscriptions. Each named subscription receives an equal share of the total. @@ -1029,7 +1263,9 @@ Environment variable: `INFLUXDB_SUBSCRIBER_TOTAL_BUFFER_BYTES` This section controls one or many listeners for Graphite data. For more information, see [Graphite protocol support in InfluxDB](/enterprise_influxdb/v1.9/supported_protocols/graphite/). -#### `enabled = false` +#### `enabled` + +Default is `false`. Determines whether the graphite endpoint is enabled. @@ -1045,27 +1281,39 @@ Batching will buffer points in memory if you have many coming in. # consistency-level = "one" ``` -#### `batch-size = 5000` +#### `batch-size` + +Default is `5000`. Flush if this many points get buffered. -#### `batch-pending = 10` +#### `batch-pending` + +Default is `10`. The number of batches that may be pending in memory. -#### `batch-timeout = "1s"` +#### `batch-timeout` + +Default is `"1s"`. Flush at least this often even if we haven't hit buffer limit. -#### `udp-read-buffer = 0` +#### `udp-read-buffer` + +Default is `0`. UDP Read buffer size, `0` means OS default. UDP listener will fail if set above OS max. -#### `separator = "."` +#### `separator` + +Default is `"."`. This string joins multiple matching 'measurement' values providing more control over the final measurement name. -#### `tags = ["region=us-east", "zone=1c"]` +#### `tags` + +Default is `["region=us-east", "zone=1c"]`. Default tags that will be added to all metrics. These can be overridden at the template level or by tags extracted from metric. @@ -1096,18 +1344,24 @@ For more information, see [CollectD protocol support in InfluxDB](/enterprise_in ### `[[collectd]]` ```toml -# enabled = false +# enabled` + +Default is `false. # bind-address = ":25826" # database = "collectd" # retention-policy = "" # typesdb = "/usr/share/collectd/types.db" ``` -#### `security-level = ""` +#### `security-level` + +Default is `""`. The collectd security level can be "" (or "none"), "sign", or "encrypt". -#### `auth-file = ""` +#### `auth-file` + +Default is `""`. The path to the `collectd` authorization file. Must be set if security level is sign or encrypt. @@ -1117,19 +1371,27 @@ These next lines control how batching works. You should have this enabled otherwise you could get dropped metrics or poor performance. Batching will buffer points in memory if you have many coming in. -#### `batch-size = 5000` +#### `batch-size` + +Default is `5000`. Flush if this many points get buffered. -#### `batch-pending = 10` +#### `batch-pending` + +Default is `10`. The number of batches that may be pending in memory. -#### `batch-timeout = "10s"` +#### `batch-timeout` + +Default is `"10s"`. Flush at least this often even if we haven't hit buffer limit. -#### `read-buffer = 0` +#### `read-buffer` + +Default is `0`. UDP Read buffer size, 0 means OS default. UDP listener will fail if set above OS max. @@ -1152,7 +1414,9 @@ For more information, see [OpenTSDB protocol support in InfluxDB](/enterprise_in # certificate= "/etc/ssl/influxdb.pem" ``` -#### `log-point-errors = true` +#### `log-point-errors` + +Default is `true`. Log an error for every malformed point. @@ -1162,15 +1426,21 @@ These next lines control how batching works. You should have this enabled otherwise you could get dropped metrics or poor performance. Only points metrics received over the telnet protocol undergo batching. -#### `batch-size = 1000` +#### `batch-size` + +Default is `1000`. Flush if this many points get buffered. -#### `batch-pending = 5` +#### `batch-pending` + +Default is `5`. The number of batches that may be pending in memory. -#### `batch-timeout = "1s"` +#### `batch-timeout` + +Default is `"1s"`. Flush at least this often even if we haven't hit buffer limit. @@ -1190,26 +1460,36 @@ For more information, see [UDP protocol support in InfluxDB](/enterprise_influxd # retention-policy = "" ``` -#### `precision = ""` +#### `precision` + +Default is `""`. InfluxDB precision for timestamps on received points ("" or "n", "u", "ms", "s", "m", "h") These next lines control how batching works. You should have this enabled otherwise you could get dropped metrics or poor performance. Batching will buffer points in memory if you have many coming in. -#### `batch-size = 5000` +#### `batch-size` + +Default is `5000`. Flush if this many points get buffered. -#### `batch-pending = 10` +#### `batch-pending` + +Default is `10`. The number of batches that may be pending in memory. -#### `batch-timeout = "1s"` +#### `batch-timeout` + +Default is `"1s"`. Will flush at least this often even if we haven't hit buffer limit. -#### `read-buffer = 0` +#### `read-buffer` + +Default is `0`. UDP Read buffer size, 0 means OS default. UDP listener will fail if set above OS max. @@ -1221,25 +1501,33 @@ UDP Read buffer size, 0 means OS default. UDP listener will fail if set above OS Controls how continuous queries are run within InfluxDB. -#### `enabled = true` +#### `enabled` + +Default is `true`. Determines whether the continuous query service is enabled. Environment variable: `INFLUXDB_CONTINUOUS_QUERIES_ENABLED` -#### `log-enabled = true` +#### `log-enabled` + +Default is `true`. Controls whether queries are logged when executed by the CQ service. Environment variable: `INFLUXDB_CONTINUOUS_QUERIES_LOG_ENABLED` -#### `query-stats-enabled = false` +#### `query-stats-enabled` + +Default is `false`. Write continuous query execution statistics to the default monitor store. Environment variable: `INFLUXDB_CONTINUOUS_QUERIES_QUERY_STATS_ENABLED` -#### `run-interval = "1s"` +#### `run-interval` + +Default is `"1s"`. The interval for how often continuous queries will be checked whether they need to run. @@ -1281,7 +1569,9 @@ max-version = "tls1.3" ``` -#### `min-version = "tls1.3"` +#### `min-version` + +Default is `"tls1.3"`. Minimum version of the TLS protocol that will be negotiated. Valid values include: `tls1.0`, `tls1.1`, and `tls1.3`. @@ -1290,7 +1580,9 @@ In this example, `tls1.3` specifies the minimum version as TLS 1.3. Environment variable: `INFLUXDB_TLS_MIN_VERSION` -#### `max-version = "tls1.3"` +#### `max-version` + +Default is `"tls1.3"`. The maximum version of the TLS protocol that will be negotiated. Valid values include: `tls1.0`, `tls1.1`, and `tls1.3`. @@ -1306,27 +1598,28 @@ Environment variable: `INFLUXDB_TLS_MAX_VERSION` This section contains configuration settings for Flux query management. For more on managing queries, see [Query Management](/enterprise_influxdb/v1.9/troubleshooting/query_management/). -#### query-concurrency +#### `query-concurrency` + Number of queries allowed to execute concurrently. `0` means unlimited. Default is `0`. -#### query-initial-memory-bytes +#### `query-initial-memory-bytes` Initial bytes of memory allocated for a query. `0` means unlimited. Default is `0`. -#### query-max-memory-bytes +#### `query-max-memory-bytes` Maximum total bytes of memory allowed for an individual query. `0` means unlimited. Default is `0`. -#### total-max-memory-bytes +#### `total-max-memory-bytes` Maximum total bytes of memory allowed for all running Flux queries. `0` means unlimited. Default is `0`. -#### query-queue-size +#### `query-queue-size` Maximum number of queries allowed in execution queue. When queue limit is reached, new queries are rejected. `0` means unlimited. diff --git a/content/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes.md b/content/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes.md index f1da803fe..0479edaeb 100644 --- a/content/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes.md +++ b/content/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes.md @@ -21,7 +21,9 @@ aliases: ### Global options -#### `reporting-disabled = false` +#### `reporting-disabled` + +Default is `false`. InfluxData, the company, relies on reported data from running nodes primarily to track the adoption rates of different InfluxDB versions. @@ -35,12 +37,16 @@ To disable reporting, set this option to `true`. > **Note:** No data from user databases are ever transmitted. -#### `bind-address = ""` +#### `bind-address` + +Default is `""`. This setting is not intended for use. It will be removed in future versions. -#### `hostname = ""` +#### `hostname` + +Default is `""`. The hostname of the [meta node](/enterprise_influxdb/v1.9/concepts/glossary/#meta-node). This must be resolvable and reachable by all other members of the cluster. @@ -55,7 +61,9 @@ Environment variable: `INFLUXDB_HOSTNAME` The `[enterprise]` section contains the parameters for the meta node's registration with the [InfluxData portal](https://portal.influxdata.com/). -#### `license-key = ""` +#### `license-key` + +Default is `""`. The license key created for you on [InfluxData portal](https://portal.influxdata.com). The meta node transmits the license key to @@ -72,7 +80,9 @@ Use the same key for all nodes in the same cluster. Environment variable: `INFLUXDB_ENTERPRISE_LICENSE_KEY` -#### `license-path = ""` +#### `license-path` + +Default is `""`. The local path to the permanent JSON license file that you received from InfluxData for instances that do not have access to the internet. @@ -96,13 +106,17 @@ Environment variable: `INFLUXDB_ENTERPRISE_LICENSE_PATH` #### `[meta]` -#### `dir = "/var/lib/influxdb/meta"` +#### `dir` + +Default is `"/var/lib/influxdb/meta"`. The directory where cluster meta data is stored. Environment variable: `INFLUXDB_META_DIR` -#### `bind-address = ":8089"` +#### `bind-address` + +Default is `":8089"`. The bind address(port) for meta node communication. For simplicity, InfluxData recommends using the same port on all meta nodes, @@ -110,13 +124,17 @@ but this is not necessary. Environment variable: `INFLUXDB_META_BIND_ADDRESS` -#### `http-bind-address = ":8091"` +#### `http-bind-address` + +Default is `":8091"`. The default address to bind the API to. Environment variable: `INFLUXDB_META_HTTP_BIND_ADDRESS` -#### `https-enabled = false` +#### `https-enabled` + +Default is `false`. Determines whether meta nodes use HTTPS to communicate with each other. By default, HTTPS is disabled. We strongly recommend enabling HTTPS. @@ -124,7 +142,9 @@ To enable HTTPS, set https-enabled to `true`, specify the path to the SSL certif Environment variable: `INFLUXDB_META_HTTPS_ENABLED` -#### `https-certificate = ""` +#### `https-certificate` + +Default is `""`. If HTTPS is enabled, specify the path to the SSL certificate. Use either: @@ -134,7 +154,9 @@ Use either: Environment variable: `INFLUXDB_META_HTTPS_CERTIFICATE` -#### `https-private-key = ""` +#### `https-private-key` + +Default is `""`. If HTTPS is enabled, specify the path to the SSL private key. Use either: @@ -144,43 +166,61 @@ Use either: Environment variable: `INFLUXDB_META_HTTPS_PRIVATE_KEY` -#### `https-insecure-tls = false` +#### `https-insecure-tls` + +Default is `false`. Whether meta nodes will skip certificate validation communicating with each other over HTTPS. This is useful when testing with self-signed certificates. Environment variable: `INFLUXDB_META_HTTPS_INSECURE_TLS` -#### `data-use-tls = false` +#### `data-use-tls` + +Default is `false`. Whether to use TLS to communicate with data nodes. -#### `data-insecure-tls = false` +#### `data-insecure-tls` + +Default is `false`. Whether meta nodes will skip certificate validation communicating with data nodes over TLS. This is useful when testing with self-signed certificates. -#### `gossip-frequency = "5s"` +#### `gossip-frequency` + +Default is `"5s"`. The default frequency with which the node will gossip its known announcements. -#### `announcement-expiration = "30s"` +#### `announcement-expiration` + +Default is `"30s"`. The default length of time an announcement is kept before it is considered too old. -#### `retention-autocreate = true` +#### `retention-autocreate` + +Default is `true`. Automatically create a default retention policy when creating a database. -#### `election-timeout = "1s"` +#### `election-timeout` + +Default is `"1s"`. The amount of time in candidate state without a leader before we attempt an election. -#### `heartbeat-timeout = "1s"` +#### `heartbeat-timeout` + +Default is `"1s"`. The amount of time in follower state without a leader before we attempt an election. -#### `leader-lease-timeout = "500ms"` +#### `leader-lease-timeout` + +Default is `"500ms"`. The leader lease timeout is the amount of time a Raft leader will remain leader if it does not hear from a majority of nodes. @@ -190,7 +230,9 @@ Clusters with high latency between nodes may want to increase this parameter to Environment variable: `INFLUXDB_META_LEADER_LEASE_TIMEOUT` -#### `commit-timeout = "50ms"` +#### `commit-timeout` + +Default is `"50ms"`. The commit timeout is the amount of time a Raft node will tolerate between commands before issuing a heartbeat to tell the leader it is alive. @@ -198,33 +240,53 @@ The default setting should work for most systems. Environment variable: `INFLUXDB_META_COMMIT_TIMEOUT` -#### `consensus-timeout = "30s"` +#### `consensus-timeout` + +Default is `"30s"`. Timeout waiting for consensus before getting the latest Raft snapshot. Environment variable: `INFLUXDB_META_CONSENSUS_TIMEOUT` -#### `cluster-tracing = false` +#### `cluster-tracing` -Cluster tracing toggles the logging of Raft logs on Raft nodes. -Enable this setting when debugging Raft consensus issues. +Default is `false`. + +Log all HTTP requests made to meta nodes. +Prints sanitized POST request information to show actual commands. + +**Sample log output:** + +``` +ts=2021-12-08T02:00:54.864731Z lvl=info msg=weblog log_id=0YHxBFZG001 service=meta-http host=172.18.0.1 user-id= username=admin method=POST uri=/user protocol=HTTP/1.1 command="{'{\"action\":\"create\",\"user\":{\"name\":\"fipple\",\"password\":[REDACTED]}}': ''}" status=307 size=0 referrer= user-agent=curl/7.68.0 request-id=ad87ce47-57ca-11ec-8026-0242ac120004 execution-time=63.571ms execution-time-readable=63.570738ms +ts=2021-12-08T02:01:00.070137Z lvl=info msg=weblog log_id=0YHxBEhl001 service=meta-http host=172.18.0.1 user-id= username=admin method=POST uri=/user protocol=HTTP/1.1 command="{'{\"action\":\"create\",\"user\":{\"name\":\"fipple\",\"password\":[REDACTED]}}': ''}" status=200 size=0 referrer= user-agent=curl/7.68.0 request-id=b09eb13a-57ca-11ec-800d-0242ac120003 execution-time=85.823ms execution-time-readable=85.823406ms +ts=2021-12-08T02:01:29.062313Z lvl=info msg=weblog log_id=0YHxBEhl001 service=meta-http host=172.18.0.1 user-id= username=admin method=POST uri=/user protocol=HTTP/1.1 command="{'{\"action\":\"create\",\"user\":{\"name\":\"gremch\",\"hash\":[REDACTED]}}': ''}" status=200 size=0 referrer= user-agent=curl/7.68.0 request-id=c1f3614a-57ca-11ec-8015-0242ac120003 execution-time=1.722ms execution-time-readable=1.722089ms +ts=2021-12-08T02:01:47.457607Z lvl=info msg=weblog log_id=0YHxBEhl001 service=meta-http host=172.18.0.1 user-id= username=admin method=POST uri=/user protocol=HTTP/1.1 command="{'{\"action\":\"create\",\"user\":{\"name\":\"gremchy\",\"hash\":[REDACTED]}}': ''}" status=400 size=37 referrer= user-agent=curl/7.68.0 request-id=ccea84b7-57ca-11ec-8019-0242ac120003 execution-time=0.154ms execution-time-readable=154.417µs +ts=2021-12-08T02:02:05.522571Z lvl=info msg=weblog log_id=0YHxBEhl001 service=meta-http host=172.18.0.1 user-id= username=admin method=POST uri=/user protocol=HTTP/1.1 command="{'{\"action\":\"create\",\"user\":{\"name\":\"thimble\",\"password\":[REDACTED]}}': ''}" status=400 size=37 referrer= user-agent=curl/7.68.0 request-id=d7af0082-57ca-11ec-801f-0242ac120003 execution-time=0.227ms execution-time-readable=227.853µs +``` Environment variable: `INFLUXDB_META_CLUSTER_TRACING` -#### `logging-enabled = true` +#### `logging-enabled` + +Default is `true`. Meta logging toggles the logging of messages from the meta service. Environment variable: `INFLUXDB_META_LOGGING_ENABLED` -#### `pprof-enabled = true` +#### `pprof-enabled` + +Default is `true`. Enables the `/debug/pprof` endpoint for troubleshooting. To disable, set the value to `false`. Environment variable: `INFLUXDB_META_PPROF_ENABLED` -#### `lease-duration = "1m0s"` +#### `lease-duration` + +Default is `"1m0s"`. The default duration of the leases that data nodes acquire from the meta nodes. Leases automatically expire after the `lease-duration` is met. @@ -238,35 +300,45 @@ For more details about `lease-duration` and its impact on continuous queries, se Environment variable: `INFLUXDB_META_LEASE_DURATION` -#### `auth-enabled = false` +#### `auth-enabled` + +Default is `false`. If true, HTTP endpoints require authentication. This setting must have the same value as the data nodes' meta.meta-auth-enabled configuration. -#### `ldap-allowed = false` +#### `ldap-allowed` + +Default is `false`. Whether LDAP is allowed to be set. If true, you will need to use `influxd ldap set-config` and set enabled=true to use LDAP authentication. -#### `shared-secret = ""` +#### `shared-secret` + +Default is `""`. The shared secret to be used by the public API for creating custom JWT authentication. -If you use this setting, set [`auth-enabled`](#auth-enabled-false) to `true`. +If you use this setting, set [`auth-enabled`](#auth-enabled) to `true`. Environment variable: `INFLUXDB_META_SHARED_SECRET` -#### `internal-shared-secret = ""` +#### `internal-shared-secret` + +Default is `""`. The shared secret used by the internal API for JWT authentication for inter-node communication within the cluster. Set this to a long pass phrase. This value must be the same value as the [`[meta] meta-internal-shared-secret`](/enterprise_influxdb/v1.9/administration/config-data-nodes#meta-internal-shared-secret) in the data node configuration file. -To use this option, set [`auth-enabled`](#auth-enabled-false) to `true`. +To use this option, set [`auth-enabled`](#auth-enabled) to `true`. Environment variable: `INFLUXDB_META_INTERNAL_SHARED_SECRET` -#### `password-hash = "bcrypt"` +#### `password-hash` + +Default is `"bcrypt"`. Specifies the password hashing scheme and its configuration. @@ -279,7 +351,7 @@ Optional sections after this are `key=value` password hash configuration options Each scheme has its own set of options. Any options not specified default to reasonable values as specified below. -This setting must have the same value as the data node option [`meta.password-hash`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#password-hash--bcrypt). +This setting must have the same value as the data node option [`meta.password-hash`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#password-hash). Environment variable: `INFLUXDB_META_PASSWORD_HASH` @@ -352,7 +424,9 @@ when used with appropriate `rounds` and `salt_len` options. * Must be greater than or equal to `16` for FIPS-readiness according to [NIST Special Publication 800-132] § 5.1. -#### `ensure-fips = false` +#### `ensure-fips` + +Default is `false`. If `ensure-fips` is set to `true`, then `influxd` and `influxd-meta` will refuse to start if they are not configured in a FIPS-ready manner. diff --git a/content/enterprise_influxdb/v1.9/administration/configure/ports.md b/content/enterprise_influxdb/v1.9/administration/configure/ports.md index 407593027..5ae29a6b9 100644 --- a/content/enterprise_influxdb/v1.9/administration/configure/ports.md +++ b/content/enterprise_influxdb/v1.9/administration/configure/ports.md @@ -20,7 +20,7 @@ aliases: The default port that runs the InfluxDB HTTP service. It is used for the primary public write and query API. Clients include the CLI, Chronograf, InfluxDB client libraries, Grafana, curl, or anything that wants to write and read time series data to and from InfluxDB. -[Configure this port](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#bind-address--8086) +[Configure this port](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#bind-address) in the data node configuration file. _See also: [API Reference](/enterprise_influxdb/v1.9/tools/api/)._ @@ -34,7 +34,7 @@ It's also used by meta nodes for cluster-type operations (e.g., tell a data node This is the default port used for RPC calls used for inter-node communication and by the CLI for backup and restore operations (`influxdb backup` and `influxd restore`). -[Configure this port](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#bind-address--8088) +[Configure this port](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#bind-address) in the configuration file. This port should not be exposed outside the cluster. diff --git a/content/enterprise_influxdb/v1.9/administration/configure/security/authentication.md b/content/enterprise_influxdb/v1.9/administration/configure/security/authentication.md index 7cef96fcd..e37ae1c39 100644 --- a/content/enterprise_influxdb/v1.9/administration/configure/security/authentication.md +++ b/content/enterprise_influxdb/v1.9/administration/configure/security/authentication.md @@ -47,7 +47,7 @@ For a more secure alternative to using passwords, include JWT tokens in requests InfluxDB Enterprise uses the shared secret to encode the JWT signature. By default, `shared-secret` is set to an empty string (no JWT authentication). - Add a custom shared secret in your [InfluxDB configuration file](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#shared-secret--) + Add a custom shared secret in your [InfluxDB configuration file](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#shared-secret) for each meta and data node. Longer strings are more secure: diff --git a/content/enterprise_influxdb/v1.9/administration/configure/security/configure-password-hashing.md b/content/enterprise_influxdb/v1.9/administration/configure/security/configure-password-hashing.md index 2e27a86a1..8e1e7c343 100644 --- a/content/enterprise_influxdb/v1.9/administration/configure/security/configure-password-hashing.md +++ b/content/enterprise_influxdb/v1.9/administration/configure/security/configure-password-hashing.md @@ -83,5 +83,5 @@ run: create server: passwordhash: not FIPS-ready: config: 'bcrypt' ``` [FIPS]: https://csrc.nist.gov/publications/detail/fips/140/3/final -[`password-hash`]: /enterprise_influxdb/v1.9/administration/config-meta-nodes/#password-hash--bcrypt -[`ensure-fips`]: /enterprise_influxdb/v1.9/administration/config-meta-nodes/#ensure-fips--false +[`password-hash`]: /enterprise_influxdb/v1.9/administration/config-meta-nodes/#password-hash +[`ensure-fips`]: /enterprise_influxdb/v1.9/administration/config-meta-nodes/#ensure-fips diff --git a/content/enterprise_influxdb/v1.9/concepts/file-system-layout.md b/content/enterprise_influxdb/v1.9/concepts/file-system-layout.md index 9ba97ed33..8ee209f71 100644 --- a/content/enterprise_influxdb/v1.9/concepts/file-system-layout.md +++ b/content/enterprise_influxdb/v1.9/concepts/file-system-layout.md @@ -28,19 +28,19 @@ The InfluxDB file structure includes the following: ### Data directory (**Data nodes only**) Directory path where InfluxDB Enterprise stores time series data (TSM files). -To customize this path, use the [`[data].dir`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#dir--varlibinfluxdbdata) +To customize this path, use the [`[data].dir`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#dir) configuration option. ### WAL directory (**Data nodes only**) Directory path where InfluxDB Enterprise stores Write Ahead Log (WAL) files. -To customize this path, use the [`[data].wal-dir`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#wal-dir--varlibinfluxdbwal) +To customize this path, use the [`[data].wal-dir`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#wal-dir) configuration option. ### Hinted handoff directory (**Data nodes only**) Directory path where hinted handoff (HH) queues are stored. -To customize this path, use the [`[hinted-handoff].dir`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#dir--varlibinfluxdbhh) +To customize this path, use the [`[hinted-handoff].dir`](/enterprise_influxdb/v1.9/administration/config-data-nodes/#dir) configuration option. ### Metastore directory @@ -48,10 +48,10 @@ Directory path of the InfluxDB Enterprise metastore, which stores information about the cluster, users, databases, retention policies, shards, and continuous queries. **On data nodes**, the metastore contains information about InfluxDB Enterprise meta nodes. -To customize this path, use the [`[meta].dir` configuration option in your data node configuration file](/enterprise_influxdb/v1.9/administration/config-data-nodes/#dir--varlibinfluxdbmeta). +To customize this path, use the [`[meta].dir` configuration option in your data node configuration file](/enterprise_influxdb/v1.9/administration/config-data-nodes/#dir). **On meta nodes**, the metastore contains information about the InfluxDB Enterprise RAFT cluster. -To customize this path, use the [`[meta].dir` configuration option in your meta node configuration file](/enterprise_influxdb/v1.9/administration/config-meta-nodes/#dir--varlibinfluxdbmeta). +To customize this path, use the [`[meta].dir` configuration option in your meta node configuration file](/enterprise_influxdb/v1.9/administration/config-meta-nodes/#dir). ### InfluxDB Enterprise configuration files InfluxDB Enterprise stores default data and meta node configuration file on disk. diff --git a/content/enterprise_influxdb/v1.9/concepts/storage_engine.md b/content/enterprise_influxdb/v1.9/concepts/storage_engine.md index f7cf01d5b..6c129482b 100644 --- a/content/enterprise_influxdb/v1.9/concepts/storage_engine.md +++ b/content/enterprise_influxdb/v1.9/concepts/storage_engine.md @@ -65,13 +65,13 @@ Deletes sent to the Cache will clear out the given key or the specific time rang The Cache exposes a few controls for snapshotting behavior. The two most important controls are the memory limits. -There is a lower bound, [`cache-snapshot-memory-size`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#cache-snapshot-memory-size--25m), which when exceeded will trigger a snapshot to TSM files and remove the corresponding WAL segments. +There is a lower bound, [`cache-snapshot-memory-size`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#cache-snapshot-memory-size), which when exceeded will trigger a snapshot to TSM files and remove the corresponding WAL segments. There is also an upper bound, [`cache-max-memory-size`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes#cache-max-memory-size-1g), which when exceeded will cause the Cache to reject new writes. These configurations are useful to prevent out of memory situations and to apply back pressure to clients writing data faster than the instance can persist it. The checks for memory thresholds occur on every write. The other snapshot controls are time based. -The idle threshold, [`cache-snapshot-write-cold-duration`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes#cache-snapshot-write-cold-duration--10m), forces the Cache to snapshot to TSM files if it hasn't received a write within the specified interval. +The idle threshold, [`cache-snapshot-write-cold-duration`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes#cache-snapshot-write-cold-duration), forces the Cache to snapshot to TSM files if it hasn't received a write within the specified interval. The in-memory Cache is recreated on restart by re-reading the WAL files on disk. diff --git a/content/enterprise_influxdb/v1.9/features/clustering-features.md b/content/enterprise_influxdb/v1.9/features/clustering-features.md index 714b41277..cbed7323c 100644 --- a/content/enterprise_influxdb/v1.9/features/clustering-features.md +++ b/content/enterprise_influxdb/v1.9/features/clustering-features.md @@ -14,8 +14,7 @@ menu: _For an overview of InfluxDB Enterprise security features, see ["InfluxDB Enterprise features - Security"](/enterprise_influxdb/v1.9/features/#security). To secure your InfluxDB Enterprise cluster, see -["Configure security"](/enterprise_influxdb/v1.9/administration/configure/security/) -and ["Manage security"](/enterprise_influxdb/v1.9/administration/manage/security/)_. +["Configure security"](/enterprise_influxdb/v1.9/administration/configure/security/). {{% /note %}} ## Entitlements @@ -59,11 +58,11 @@ Subscriptions used by Kapacitor work in a cluster. Writes to any node will be fo It is important to understand how to configure InfluxDB Enterprise and how this impacts the continuous queries (CQ) engine’s behavior: - **Data node configuration** `[continuous queries]` -[run-interval](/enterprise_influxdb/v1.9/administration/config-data-nodes#run-interval-1s) +[run-interval](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#run-interval) -- The interval at which InfluxDB checks to see if a CQ needs to run. Set this option to the lowest interval at which your CQs run. For example, if your most frequent CQ runs every minute, set run-interval to 1m. - **Meta node configuration** `[meta]` -[lease-duration](/enterprise_influxdb/v1.9/administration/config-meta-nodes#lease-duration-1m0s) +[lease-duration](/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes/#lease-duration) -- The default duration of the leases that data nodes acquire from the meta nodes. Leases automatically expire after the lease-duration is met. Leases ensure that only one data node is running something at a given time. For example, Continuous Queries use a lease so that all data nodes aren’t running the same CQs at once. diff --git a/content/enterprise_influxdb/v1.9/guides/authenticate.md b/content/enterprise_influxdb/v1.9/guides/authenticate.md index 49fad2a00..106cd990c 100644 --- a/content/enterprise_influxdb/v1.9/guides/authenticate.md +++ b/content/enterprise_influxdb/v1.9/guides/authenticate.md @@ -110,7 +110,7 @@ This is currently only possible through the [InfluxDB HTTP API](/enterprise_infl InfluxDB Enterprise uses the shared secret to encode the JWT signature. By default, `shared-secret` is set to an empty string, in which case no JWT authentication takes place. - Add a custom shared secret in your [InfluxDB configuration file](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#shared-secret--). + Add a custom shared secret in your [InfluxDB configuration file](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#shared-secret). The longer the secret string, the more secure it is: ```toml diff --git a/content/enterprise_influxdb/v1.9/guides/downsample_and_retain.md b/content/enterprise_influxdb/v1.9/guides/downsample_and_retain.md index 3fcd19824..4587fe898 100644 --- a/content/enterprise_influxdb/v1.9/guides/downsample_and_retain.md +++ b/content/enterprise_influxdb/v1.9/guides/downsample_and_retain.md @@ -214,7 +214,7 @@ data that reside in an RP other than the `DEFAULT` RP. Between checks, `orders` may have data that are older than two hours. The rate at which InfluxDB checks to enforce an RP is a configurable setting, see -[Database Configuration](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#check-interval--30m0s). +[Database Configuration](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#check-interval). Using a combination of RPs and CQs, we've successfully set up our database to automatically keep the high precision raw data for a limited time, create lower diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/_index.md b/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/_index.md deleted file mode 100644 index e842680d4..000000000 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/_index.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: Install and deploy InfluxDB Enterprise -description: Install InfluxDB Enterprise to on-premise or cloud providers, including Google Cloud Platform, Amazon Web Services, and Azure. -aliases: -- /enterprise_influxdb/v1.9/install-and-deploy/deploying/ -- /enterprise_influxdb/v1.9/install-and-deploy/ -- /enterprise_influxdb/v1.9/introduction/get-started/ -- /enterprise_influxdb/v1.9/production_installation/ -- /enterprise_influxdb/v1.9/introduction/installation/ - -menu: - enterprise_influxdb_1_9: - name: Install and deploy - weight: 30 - parent: Introduction ---- - -Install or deploy your InfluxDB Enterprise cluster in the environment of your choice: - -- Your own environment -- Your cloud provider - -## Your own environment - -Learn how to [install a cluster in your own environment](/enterprise_influxdb/v1.9/install-and-deploy/installation/). - -## Your cloud provider - -Learn how to deploy a cluster on the cloud provider of your choice: - - - [GCP](/enterprise_influxdb/v1.9/install-and-deploy/deploying/google-cloud-platform/) - - [AWS](/enterprise_influxdb/v1.9/install-and-deploy/deploying/aws/) - - [Azure](/enterprise_influxdb/v1.9/install-and-deploy/deploying/azure/) diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/_index.md b/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/_index.md deleted file mode 100644 index eee062c65..000000000 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/_index.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -title: Deploy InfluxDB Enterprise clusters -description: > - Install InfluxDB Enterprise to a cloud provider of your choice, including Google Cloud Platform, Amazon Web Services, and Azure. -aliases: - - /enterprise_influxdb/v1.9/other-options/ - - /enterprise_influxdb/v1.9/install-and-deploy/deploying/index/ -menu: - enterprise_influxdb_1_9: - name: Deploy in cloud - identifier: deploy-in-cloud-enterprise - weight: 30 - parent: Install and deploy ---- - -Deploy InfluxDB Enterprise clusters on the cloud provider of your choice. - -> **Note:** To install in your own environment, see [Install an InfluxDB Enterprise cluster in your own environment](/enterprise_influxdb/v1.9/install-and-deploy/installation/). - -{{< children hlevel="h2" >}} diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/_index.md b/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/_index.md deleted file mode 100644 index 3031ff526..000000000 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/_index.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -title: Deploy InfluxDB Enterprise clusters on Amazon Web Services -description: Deploy InfluxDB Enterprise clusters on Amazon Web Services (AWS). -aliases: - - /enterprise_influxdb/v1.9/other-options/ - - /enterprise_influxdb/v1.9/install-and-deploy/aws/ - - /enterprise_influxdb/v1.9/install-and-deploy/deploying/aws/ -menu: - enterprise_influxdb_1_9: - name: AWS - identifier: deploy-on-aws - weight: 30 - parent: deploy-in-cloud-enterprise ---- - -The following articles detail how to deploy InfluxDB clusters in AWS: - -- [Deploy an InfluxDB Enterprise cluster on Amazon Web Services](/enterprise_influxdb/v1.9/install-and-deploy/deploying/aws/setting-up-template) -- [AWS configuration options](/enterprise_influxdb/v1.9/install-and-deploy/deploying/aws/config-options) diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/config-options.md b/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/config-options.md deleted file mode 100644 index 088faadce..000000000 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/config-options.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: AWS configuration options -description: > - Configuration options when deploying InfluxDB Enterprise on Amazon Web Services (AWS). -aliases: - - /enterprise_influxdb/v1.9/install-and-deploy/aws/config-options/ - - /enterprise_influxdb/v1.9/install-and-deploy/aws/config-options/ - - /enterprise_influxdb/v1.9/install-and-deploy/deploying/aws/config-options/ -menu: - enterprise_influxdb_1_9: - name: AWS configuration options - weight: 30 - parent: deploy-on-aws ---- -When deploying InfluxDB Enterprise on AWS using the template described in [Deploy an InfluxDB Enterprise cluster on Amazon Web Services](/enterprise_influxdb/v1.9/install-and-deploy/aws/setting-up-template), the following configuration options are available: - -- **VPC ID**: The VPC ID of your existing Virtual Private Cloud (VPC). -- **Subnets**: A list of SubnetIds in your Virtual Private Cloud (VPC) where nodes will be created. The subnets must be in the same order as the availability zones they reside in. For a list of which availability zones correspond to which subnets, see the [Subnets section of your VPC dashboard](https://console.aws.amazon.com/vpc/home?region=us-east-1#subnets:sort=SubnetId). -- **Availability Zones**: Availability zones to correspond with your subnets above. The availability zones must be in the same order as their related subnets. For a list of which availability zones correspond to which subnets, see the [Subnets section of your VPC dashboard](https://console.aws.amazon.com/vpc/home?region=us-east-1#subnets:sort=SubnetId). -- **SSH Key Name**: An existing key pair to enable SSH access for the instances. For details on how to create a key pair, see [Creating a Key Pair Using Amazon EC2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#having-ec2-create-your-key-pair). -- **InfluxDB ingress CIDR**: The IP address range that can be used to connect to the InfluxDB API endpoint. To allow all traffic, enter 0.0.0.0/0. -- **SSH Access CIDR**: The IP address range that can be used to SSH into the EC2 instances. To allow all traffic, enter 0.0.0.0/0. -- **InfluxDB Enterprise License Key**: Your InfluxDB Enterprise license key. Applies only to BYOL. -- **InfluxDB Administrator Username**: Your InfluxDB administrator username. Applies only to BYOL. -- **InfluxDB Administrator Password**: Your InfluxDB administrator password. Applies only to BYOL. -- **InfluxDB Enterprise Version**: The version of InfluxDB. Defaults to current version. -- **Telegraf Version**: The version of Telegraf. Defaults to current version. -- **InfluxDB Data Node Disk Size**: The size in GB of the EBS io1 volume each data node. Defaults to 250. -- **InfluxDB Data Node Disk IOPS**: The IOPS of the EBS io1 volume on each data node. Defaults to 1000. -- **DataNodeInstanceType**: The instance type of the data node. Defaults to m5.large. -- **MetaNodeInstanceType**: The instance type of the meta node. Defaults to t3.small. -- **MonitorInstanceType**: The instance type of the monitor node. Defaults to t3.large. diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/setting-up-template.md b/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/setting-up-template.md deleted file mode 100644 index 6ff0f6583..000000000 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/aws/setting-up-template.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -title: Deploy an InfluxDB Enterprise cluster on Amazon Web Services -description: Deploy an InfluxDB Enterprise cluster on Amazon Web Services (AWS). -aliases: - - /enterprise_influxdb/v1.9/install-and-deploy/aws/setting-up-template/ - - /enterprise_influxdb/v1.9/install-and-deploy/deploying/aws/setting-up-template/ -menu: - enterprise_influxdb_1_9: - name: Deploy on Amazon Web Services - weight: 20 - parent: deploy-on-aws ---- - -Follow these steps to deploy an InfluxDB Enterprise cluster on AWS. - -## Step 1: Specify template - -After you complete the marketplace flow, you'll be directed to the Cloud Formation Template. - -1. In the Prepare template section, select **Template is ready**. -2. In the Specify template section, the **Amazon S3 URL** field is automatically populated with either the BYOL or integrated billing template, depending on the option you selected in the marketplace. -3. Click **Next**. - -## Step 2: Specify stack details - -1. In the Stack name section, enter a name for your stack. -2. Complete the Network Configuration section: - - **VPC ID**: Click the dropdown menu to fill in your VPC. - - **Subnets**: Select three subnets. - - **Availability Zones**: Select three availability zones to correspond with your subnets above. The availability zones must be in the same order as their related subnets. For a list of which availability zones correspond to which subnets, see the [Subnets section of your VPC dashboard](https://console.aws.amazon.com/vpc/home?region=us-east-1#subnets:sort=SubnetId). - - **SSH Key Name**: Select an existing key pair to enable SSH access for the instances. For details on how to create a key pair, see [Creating a Key Pair Using Amazon EC2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#having-ec2-create-your-key-pair). - - **InfluxDB ingress CIDR**: Enter the IP address range that can be used to connect to the InfluxDB API endpoint. To allow all traffic, enter 0.0.0.0/0. - - **SSH Access CIDR**: Enter the IP address range that can be used to SSH into the EC2 instances. To allow all traffic, enter 0.0.0.0/0. -3. Complete the **InfluxDB Configuration** section: - - **InfluxDB Enterprise License Key**: Applies only to BYOL. Enter your InfluxDB Enterprise license key. - - **InfluxDB Administrator Username**: Applies only to BYOL. Enter your InfluxDB administrator username. - - **InfluxDB Administrator Password**: Applies only to BYOL. Enter your InfluxDB administrator password. - - **InfluxDB Enterprise Version**: Defaults to current version. - - **Telegraf Version**: Defaults to current version. - - **InfluxDB Data Node Disk Size**: The size in GB of the EBS io1 volume each data node. Defaults to 250. - - **InfluxDB Data Node Disk IOPS**: The IOPS of the EBS io1 volume on each data node. Defaults to 1000. -4. Review the **Other Parameters** section and modify if needed. The fields in this section are all automatically populated and shouldn't require changes. - - **DataNodeInstanceType**: Defaults to m5.large. - - **MetaNodeInstanceType**: Defaults to t3.small. - - **MonitorInstanceType**: Defaults to t3.large. -5. Click **Next**. - -## Step 3: Configure stack options - -1. In the **Tags** section, enter any key-value pairs you want to apply to resources in the stack. -2. Review the **Permissions** and **Advanced options** sections. In most cases, there's no need to modify anything in these sections. -3. Click **Next**. - -## Step 4: Review - -1. Review the configuration options for all of the above sections. -2. In the **Capabilities** section, check the box acknowledging that AWS CloudFormation might create IAM resources. -3. Click **Create stack**. diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/azure.md b/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/azure.md deleted file mode 100644 index fecc7dff7..000000000 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/azure.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -title: Deploy an InfluxDB Enterprise cluster on Azure Cloud Platform -description: > - Deploy an InfluxDB Enterprise cluster on Microsoft Azure cloud computing service. -aliases: - - /enterprise_influxdb/v1.9/install-and-deploy/azure/ - - /enterprise_influxdb/v1.9/install-and-deploy/deploying/azure/ -menu: - enterprise_influxdb_1_9: - name: Azure - weight: 20 - parent: deploy-in-cloud-enterprise ---- - -For deploying InfluxDB Enterprise clusters on Microsoft Azure cloud computing service, InfluxData provides an [InfluxDB Enterprise application](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/influxdata.influxdb-enterprise-cluster) on the [Azure Marketplace](https://azuremarketplace.microsoft.com/) that makes the installation and setup process easy and straightforward. Clusters are deployed through an Azure Marketplace subscription and are ready for production. Billing occurs through your Azure subscription. - -> Please submit issues and feature requests for the Azure Marketplace deployment [through the related GitHub repository](https://github.com/influxdata/azure-resource-manager-influxdb-enterprise/issues/new) (requires a GitHub account) or by contacting [InfluxData Support](mailto:support@influxdata.com). - -## Prerequisites - -This guide requires the following: - -- Microsoft Azure account with access to the [Azure Marketplace](https://azuremarketplace.microsoft.com/). -- SSH access to cluster instances. - -To deploy InfluxDB Enterprise clusters on platforms other than Azure, see [Deploy InfluxDB Enterprise](/enterprise_influxdb/v1.9/install-and-deploy/). - -## Deploy a cluster - -1. Log in to your Azure Cloud Platform account and navigate to [InfluxData's InfluxDB Enterprise (Official Version) application](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/influxdata.influxdb-enterprise-cluster) on Azure Marketplace. - -2. Click **Get It Now**, read and agree to the terms of use, and then click **Continue**. Once in the Azure Portal, click **Create**. - -3. Select the subscription to use for your InfluxDB Enterprise cluster. Then select a resource group and region where the cluster resources will be deployed. - - > **Tip:** If you do not know which resource group to use, we recommend creating a new one for the InfluxDB Enterprise cluster. - -4. In the Instance Details section, set the OS username and SSH authentication type you will use to access the cluster VMs. For password authentication, enter a username and password. For SSH public key authentication, copy an SSH public key. The cluster VMs are built from an Ubuntu base image. - -5. Click **Next: Cluster Configuration**, and then enter details including the InfluxDB admin username and password, the number of meta and data nodes, and the VM size for both meta and data nodes. We recommend using the default VM sizes and increasing the data node VM size if you anticipate needing more resources for your workload. - - > **Note:** Make sure to save the InfluxDB admin credentials. They will be required to access InfluxDB. - -6. Click **Next: External Access & Chronograf**, and then do the following: - - - To create a separate instance to monitor the cluster and run [Chronograf](https://www.influxdata.com/time-series-platform/chronograf/), select **Yes**. Otherwise, select **No**. - - > **Note:** Adding a Chronograf instance will also configure that instance as an SSH bastion. All cluster instances will only be accessible through the Chronograf instance. - - - Select the appropriate access for the InfluxDB load balancer: **External** to allow external Internet access; otherwise, select **Internal**. - - {{% warn %}}The cluster uses HTTP by default. You must configure HTTPS after the cluster has been deployed.{{% /warn %}} - -7. Click **Next: Review + create** to validate your cluster configuration details. If validation passes, your InfluxDB Enterprise cluster is deployed. - - > **Note:** Some Azure accounts may have vCPU quotas limited to 10 vCPUs available in certain regions. Selecting VM sizes larger than the default can cause a validation error for exceeding the vCPU limit for the region. - -## Access InfluxDB - -Once the cluster is created, access the InfluxDB API at the IP address associated with the load balancer resource (`lb-influxdb`). If external access was configured during setup, the load balancer is publicly accessible. Otherwise, the load balancer is only accessible to the cluster's virtual network. - -Use the load balancer IP address and the InfluxDB admin credentials entered during the cluster creation to interact with InfluxDB Enterprise via the [`influx` CLI](/enterprise_influxdb/v1.9/tools/influx-cli/use-influx/) or use the InfluxDB's [query](/enterprise_influxdb/v1.9/guides/query_data/) and [write](/enterprise_influxdb/v1.9/guides/write_data/) HTTP APIs. - -## Access the cluster - -The InfluxDB Enterprise cluster's VMs are only reachable within the virtual network using the SSH credentails provided during setup. - -If a Chronograf instance has been added to the cluster, the Chronograf instance is publically accessible via SSH. The other VMs can then be reached from the Chronograf VM. - -## Testing - -Azure Resource Manager (ARM) templates used in the InfluxDB Enterprise offering on Azure Marketplace are [available for testing purposes](https://github.com/influxdata/azure-resource-manager-influxdb-enterprise). **Please note, these templates are under active development and not recommended for production.** - -### Next steps - -For an introduction to the InfluxDB database and the InfluxData Platform, see [Getting started with InfluxDB](/platform/introduction/getting-started). diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/google-cloud-platform.md b/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/google-cloud-platform.md deleted file mode 100644 index cd457f5c8..000000000 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/deploying/google-cloud-platform.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -title: Deploy an InfluxDB Enterprise cluster on Google Cloud Platform -description: > - Deploy an InfluxDB Enterprise cluster on Google Cloud Platform (GCP). -aliases: - - /enterprise_influxdb/v1.9/other-options/google-cloud/ - - /enterprise_influxdb/v1.9/install-and-deploy/google-cloud-platform/ - - /enterprise_influxdb/v1.9/install-and-deploy/deploying/google-cloud-platform/ -menu: - enterprise_influxdb_1_9: - name: GCP - weight: 30 - parent: deploy-in-cloud-enterprise ---- - -Complete the following steps to deploy an InfluxDB Enterprise cluster on Google Cloud Platform (GCP): - -1. [Verify prerequistes](#verify-prerequisites). -2. [Deploy a cluster](#deploy-a-cluster). -3. [Access the cluster](#access-the-cluster). - -After deploying your cluster, see [Getting started with InfluxDB](/platform/introduction/getting-started) for an introduction to InfluxDB database and the InfluxData platform. - ->**Note:** InfluxDB Enterprise on GCP is a self-managed product. For a fully managed InfluxDB experience, check out [InfluxDB Cloud](/influxdb/cloud/sign-up/). - -## Verify prerequisites - -Before deploying an InfluxDB Enterprise cluster on GCP, verify you have the following prerequisites: - -- A [Google Cloud Platform (GCP)](https://cloud.google.com/) account with access to the [GCP Marketplace](https://cloud.google.com/marketplace/). -- Access to [GCP Cloud Shell](https://cloud.google.com/shell/) or the [`gcloud` SDK and command line tools](https://cloud.google.com/sdk/). - -## Deploy a cluster - -1. Log in to your Google Cloud Platform account and go to [InfluxDB Enterprise](https://console.cloud.google.com/marketplace/details/influxdata-public/influxdb-enterprise-vm). - -2. Click **Launch** to create or select a project to open up your cluster's configuration page. - -3. Adjust cluster fields as needed, including: - - - Deployment name: Enter a name for the InfluxDB Enterprise cluster. - - InfluxDB Enterprise admin username: Enter the username of your cluster administrator. - - Zone: Select a region for your cluster. - - Network: Select a network for your cluster. - - Subnetwork: Select a subnetwork for your cluster, if applicable. - - > **Note:** The cluster is only accessible within the network (or subnetwork, if specified) where it's deployed. - -4. Adjust data node fields as needed, including: - - - Data node instance count: Enter the number of data nodes to include in your cluster (we recommend starting with the default, 2). - - Data node machine type: Select the virtual machine type to use for data nodes (by default, 4 vCPUs). Use the down arrow to scroll through list. Notice the amount of memory available for the selected machine. To alter the number of cores and memory for your selected machine type, click the **Customize** link. - - - - - (Optional) By default, the data node disk type is SSD Persistent Disk and the disk size is 250 GB. To alter these defaults, click More and update if needed. - - > **Note:** Typically, fields in collapsed sections don't need to be altered. - -5. Adjust meta node fields as needed, including: - - - Meta node instance count: Enter the number of meta nodes to include in your cluster (we recommend using the default, 3). - - Meta node machine type: Select the virtual machine type to use for meta nodes (by default, 1 vCPUs). Use the down arrow to scroll through list. Notice the amount of memory available for the selected machine. To alter the number of cores and memory for your selected machine type, click the **Customize** link. - - By default, the meta node disk type is SSD Persistent Disk and the disk size is 10 GB. Alter these defaults if needed. - -6. (Optional) Adjust boot disk options fields is needed. By default the boot disk type is Standard Persistent disk and boot disk is 10 GB . - -7. Accept terms and conditions by selecting both check boxes, and then click **Deploy** to launch the InfluxDB Enterprise cluster. - -The cluster may take a few minutes to fully deploy. If the deployment does not complete or reports an error, read through the list of [common deployment errors](https://cloud.google.com/marketplace/docs/troubleshooting). - -> **Important:** Make sure you save the "Admin username", "Admin password", and "Connection internal IP" values displayed on the screen. They are required to access the cluster. - -## Access the cluster - -Access the cluster's IP address from the GCP network (or subnetwork) specified when you deployed the cluster. A cluster can only be reached from instances or services in the same GCP network or subnetwork. - -1. In the GCP Cloud Shell or `gcloud` CLI, create a new instance to access the InfluxDB Enterprise cluster. - - ``` - gcloud compute instances create influxdb-access --image-family ubuntu-1804-lts --image-project ubuntu-os-cloud - ``` - -2. SSH into the instance. - - ``` - gcloud compute ssh influxdb-access - ``` - -3. On the instance, install the `influx` command line tool via the InfluxDB open source package. - - ``` - wget https://dl.influxdata.com/influxdb/releases/influxdb_{{< latest-patch >}}_amd64.deb - sudo dpkg -i influxdb_{{< latest-patch >}}_amd64.deb - ``` - -4. Access the InfluxDB Enterprise cluster using the following command with "Admin username", "Admin password", and "Connection internal IP" values from the deployment screen substituted for ``. - - ``` - influx -username -password -host -execute "CREATE DATABASE test" - - influx -username -password -host -execute "SHOW DATABASES" - ``` diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/_index.md b/content/enterprise_influxdb/v1.9/introduction/installation/_index.md similarity index 55% rename from content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/_index.md rename to content/enterprise_influxdb/v1.9/introduction/installation/_index.md index 7fa21d69a..447c5f8f3 100644 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/_index.md +++ b/content/enterprise_influxdb/v1.9/introduction/installation/_index.md @@ -1,14 +1,15 @@ --- -title: Install an InfluxDB Enterprise cluster in your own environment +title: Install an InfluxDB Enterprise cluster description: Install InfluxDB Enterprise in your own on-premise environment. aliases: - /enterprise_influxdb/v1.9/installation/ - /enterprise_influxdb/v1.9/install-and-deploy/installation/ + - /enterprise_influxdb/v1.9/introduction/install-and-deploy/ menu: enterprise_influxdb_1_9: - name: Install in your environment - weight: 10 - parent: Install and deploy + name: Install + weight: 103 + parent: Introduction --- Complete the following steps to install an InfluxDB Enterprise cluster in your own environment: @@ -16,5 +17,3 @@ Complete the following steps to install an InfluxDB Enterprise cluster in your o 1. [Install InfluxDB Enterprise meta nodes](/enterprise_influxdb/v1.9/install-and-deploy/installation/meta_node_installation/) 2. [Install InfluxDB data nodes](/enterprise_influxdb/v1.9/install-and-deploy/installation/data_node_installation/) 3. [Install Chronograf](/enterprise_influxdb/v1.9/install-and-deploy/installation/chrono_install/) - -> **Note:** If you're looking for cloud infrastructure and services, check out how to deploy InfluxDB Enterprise (production-ready) on a cloud provider of your choice: [Azure](/enterprise_influxdb/v1.9/install-and-deploy/deploying/azure/), [GCP](/enterprise_influxdb/v1.9/install-and-deploy/deploying/google-cloud-platform/), or [AWS](/enterprise_influxdb/v1.9/install-and-deploy/deploying/aws/). diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/chrono_install.md b/content/enterprise_influxdb/v1.9/introduction/installation/chrono_install.md similarity index 93% rename from content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/chrono_install.md rename to content/enterprise_influxdb/v1.9/introduction/installation/chrono_install.md index 1c9eb2e0d..1aa94629d 100644 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/chrono_install.md +++ b/content/enterprise_influxdb/v1.9/introduction/installation/chrono_install.md @@ -7,7 +7,7 @@ menu: enterprise_influxdb_1_9: name: Install Chronograf weight: 30 - parent: Install in your environment + parent: Install identifier: chrono_install --- diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/data_node_installation.md b/content/enterprise_influxdb/v1.9/introduction/installation/data_node_installation.md similarity index 99% rename from content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/data_node_installation.md rename to content/enterprise_influxdb/v1.9/introduction/installation/data_node_installation.md index 98495bab0..2cc156d76 100644 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/data_node_installation.md +++ b/content/enterprise_influxdb/v1.9/introduction/installation/data_node_installation.md @@ -7,7 +7,7 @@ menu: enterprise_influxdb_1_9: name: Install data nodes weight: 20 - parent: Install in your environment + parent: Install --- InfluxDB Enterprise offers highly scalable clusters on your infrastructure @@ -316,7 +316,7 @@ Once your data nodes are part of your cluster, do the following: - Set up [authentication](/enterprise_influxdb/v1.9/administration/configure/security/authentication/). Once you cluster is configured for authentication, if you want to add more users in addition to admin user, - see [Manage users and permissions](/enterprise_influxdb/v1.9/administration/manage/security/). + see [Manage users and permissions](/enterprise_influxdb/v1.9/administration/manage/users-and-permissions/). - [Enable TLS](/enterprise_influxdb/v1.9/guides/enable-tls/). - [Set up Chronograf](/enterprise_influxdb/v1.9/install-and-deploy/installation/chrono_install) for UI visualization, dashboards, and management. diff --git a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/meta_node_installation.md b/content/enterprise_influxdb/v1.9/introduction/installation/meta_node_installation.md similarity index 99% rename from content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/meta_node_installation.md rename to content/enterprise_influxdb/v1.9/introduction/installation/meta_node_installation.md index 2affd5c2c..8d8dcd4f2 100644 --- a/content/enterprise_influxdb/v1.9/introduction/install-and-deploy/installation/meta_node_installation.md +++ b/content/enterprise_influxdb/v1.9/introduction/installation/meta_node_installation.md @@ -7,7 +7,7 @@ menu: enterprise_influxdb_1_9: name: Install meta nodes weight: 10 - parent: Install in your environment + parent: Install --- InfluxDB Enterprise offers highly scalable clusters on your infrastructure diff --git a/content/enterprise_influxdb/v1.9/introduction/installation_requirements.md b/content/enterprise_influxdb/v1.9/introduction/installation_requirements.md index d94c8a71d..8a26c0dab 100644 --- a/content/enterprise_influxdb/v1.9/introduction/installation_requirements.md +++ b/content/enterprise_influxdb/v1.9/introduction/installation_requirements.md @@ -12,7 +12,7 @@ menu: parent: Introduction --- -Review the installation requirements below, and then check out available options to [install and deploy InfluxDB Enterprise](/enterprise_influxdb/v1.9/install-and-deploy/). For an overview of the architecture and concepts in an InfluxDB Enterprise cluster, review [Clustering in InfluxDB Enterprise](/enterprise_influxdb/v1.9/concepts/clustering/). +Review the installation requirements below, and then check out available options to [install and deploy InfluxDB Enterprise](/enterprise_influxdb/v1.9/introduction/installation/). For an overview of the architecture and concepts in an InfluxDB Enterprise cluster, review [Clustering in InfluxDB Enterprise](/enterprise_influxdb/v1.9/concepts/clustering/). ## Requirements for InfluxDB Enterprise clusters diff --git a/content/enterprise_influxdb/v1.9/query_language/manage-database.md b/content/enterprise_influxdb/v1.9/query_language/manage-database.md index ad52cf8d1..6b9c22ab0 100644 --- a/content/enterprise_influxdb/v1.9/query_language/manage-database.md +++ b/content/enterprise_influxdb/v1.9/query_language/manage-database.md @@ -87,7 +87,7 @@ If you attempt to create a database that already exists, InfluxDB does nothing a ``` The query creates a database called `NOAA_water_database`. -[By default](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#retention-autocreate--true), InfluxDB also creates the `autogen` retention policy and associates it with the `NOAA_water_database`. +[By default](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#retention-autocreate), InfluxDB also creates the `autogen` retention policy and associates it with the `NOAA_water_database`. ##### Create a database with a specific retention policy @@ -229,7 +229,7 @@ exist. The following sections cover how to create, alter, and delete retention policies. Note that when you create a database, InfluxDB automatically creates a retention policy named `autogen` which has infinite retention. -You may disable its auto-creation in the [configuration file](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#retention-autocreate--true). +You may disable its auto-creation in the [configuration file](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#retention-autocreate). ### Create retention policies with CREATE RETENTION POLICY diff --git a/content/enterprise_influxdb/v1.9/supported_protocols/prometheus.md b/content/enterprise_influxdb/v1.9/supported_protocols/prometheus.md index 668def969..cb1dd915c 100644 --- a/content/enterprise_influxdb/v1.9/supported_protocols/prometheus.md +++ b/content/enterprise_influxdb/v1.9/supported_protocols/prometheus.md @@ -89,7 +89,7 @@ made to match the InfluxDB data structure: * Prometheus labels become InfluxDB tags. * All `# HELP` and `# TYPE` lines are ignored. * [v1.8.6 and later] Prometheus remote write endpoint drops unsupported Prometheus values (`NaN`,`-Inf`, and `+Inf`) rather than reject the entire batch. - * If [write trace logging is enabled (`[http] write-tracing = true`)](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#write-tracing--false), then summaries of dropped values are logged. + * If [write trace logging is enabled (`[http] write-tracing = true`)](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#write-tracing), then summaries of dropped values are logged. * If a batch of values contains values that are subsequently dropped, HTTP status code `204` is returned. ### Example: Parse Prometheus to InfluxDB diff --git a/content/enterprise_influxdb/v1.9/tools/api.md b/content/enterprise_influxdb/v1.9/tools/api.md index af3027f4a..b6eacf8fb 100644 --- a/content/enterprise_influxdb/v1.9/tools/api.md +++ b/content/enterprise_influxdb/v1.9/tools/api.md @@ -427,7 +427,7 @@ A successful [`CREATE DATABASE` query](/enterprise_influxdb/v1.9/query_language/ | u=\ | Optional if you haven't [enabled authentication](/enterprise_influxdb/v1.9/administration/authentication_and_authorization/#set-up-authentication). Required if you've enabled authentication.* | Sets the username for authentication if you've enabled authentication. The user must have read access to the database. Use with the query string parameter `p`. | \* InfluxDB does not truncate the number of rows returned for requests without the `chunked` parameter. -That behavior is configurable; see the [`max-row-limit`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#max-row-limit--0) +That behavior is configurable; see the [`max-row-limit`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#max-row-limit) configuration option for more information. \** The InfluxDB API also supports basic authentication. @@ -951,7 +951,7 @@ Errors are returned in JSON. | 400 Bad Request | Unacceptable request. Can occur with an InfluxDB line protocol syntax error or if a user attempts to write values to a field that previously accepted a different value type. The returned JSON offers further information. | | 401 Unauthorized | Unacceptable request. Can occur with invalid authentication credentials. | | 404 Not Found | Unacceptable request. Can occur if a user attempts to write to a database that does not exist. The returned JSON offers further information. | -| 413 Request Entity Too Large | Unaccetable request. It will occur if the payload of the POST request is bigger than the maximum size allowed. See [`max-body-size`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#max-body-size--25000000) parameter for more details. +| 413 Request Entity Too Large | Unaccetable request. It will occur if the payload of the POST request is bigger than the maximum size allowed. See [`max-body-size`](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#max-body-size) parameter for more details. | 500 Internal Server Error | The system is overloaded or significantly impaired. Can occur if a user attempts to write to a retention policy that does not exist. The returned JSON offers further information. | #### Examples diff --git a/content/enterprise_influxdb/v1.9/tools/influxd-ctl.md b/content/enterprise_influxdb/v1.9/tools/influxd-ctl.md index ff09fecde..15ed1ce05 100644 --- a/content/enterprise_influxdb/v1.9/tools/influxd-ctl.md +++ b/content/enterprise_influxdb/v1.9/tools/influxd-ctl.md @@ -110,7 +110,9 @@ influxd-ctl -auth-type jwt -secret oatclusters show The `influxd-ctl` utility uses JWT authentication with the shared secret `oatclusters`. -If authentication is enabled in the cluster's [meta node configuration files](/enterprise_influxdb/v1.9/administration/config-meta-nodes#auth-enabled-false) and [data node configuration files](/enterprise_influxdb/v1.9/administration/config-data-nodes#meta-auth-enabled-false) and the `influxd-ctl` command does not include authentication details, the system returns: +If authentication is enabled in the cluster's [meta node configuration files](/enterprise_influxdb/v1.9/administration/configure/config-meta-nodes/#auth-enabled) +and [data node configuration files](/enterprise_influxdb/v1.9/administration/config-data-nodes/#meta-auth-enabled) +and the `influxd-ctl` command does not include authentication details, the system returns: ```bash Error: unable to parse authentication credentials. @@ -132,7 +134,10 @@ In the following example, the `influxd-ctl` utility uses basic authentication fo influxd-ctl -auth-type basic -user admini -pwd mouse show ``` -If authentication is enabled in the cluster's [meta node configuration files](/enterprise_influxdb/v1.9/administration/config-meta-nodes#auth-enabled-false) and [data node configuration files](/enterprise_influxdb/v1.9/administration/config-data-nodes#meta-auth-enabled-false) and the `influxd-ctl` command does not include authentication details, the system returns: +If authentication is enabled in the cluster's +[meta node configuration files](/enterprise_influxdb/v1.9/administration/config-meta-nodes/#auth-enabled-false) +and [data node configuration files](/enterprise_influxdb/v1.9/administration/configure/config-data-nodes/#meta-auth-enabled) +and the `influxd-ctl` command does not include authentication details, the system returns: ```bash Error: unable to parse authentication credentials. @@ -246,6 +251,20 @@ Optional arguments are in brackets. Name of the single database to back up. +###### [ `-estimate` ] + +Provide estimated backup size and progress messages during backup. + +**Sample output:** + +``` +Backing up node backup_data_0_1:8088, db stress, rp autogen, shard 14 +Files: 8 / 9 Bytes: 189543424 / 231921095 Completed: 82% in 22s Estimated remaining: 3s +Files: 8 / 9 Bytes: 189543424 / 231921095 Completed: 82% in 23s Estimated remaining: 2s +Files: 9 / 9 Bytes: 231736320 / 231921095 Completed: 100% in 24s Estimated remaining: 447µs +Done backing up node backup_data_0_1:8088, db stress, rp autogen, shard 14 in 67ms: 42192896 bytes transferred +``` + ###### [ `-from ` ] TCP address of the target data node. @@ -351,7 +370,10 @@ Copied shard 22 from cluster-data-node-01:8088 to cluster-data-node-02:8088 ### `copy-shard-status` -Shows all in-progress [copy shard](#copy-shard) operations, including the shard's source node, destination node, database, [retention policy](/enterprise_influxdb/v1.9/concepts/glossary/#retention-policy-rp), shard ID, total size, current size, and the operation's start time. +Shows all in-progress [copy shard](#copy-shard) operations, +including the shard's source node, destination node, database, +[retention policy](/enterprise_influxdb/v1.9/concepts/glossary/#retention-policy-rp), +shard ID, total size, current size, and the operation's start time. #### Syntax @@ -935,6 +957,19 @@ To restore metadata, [restore a metadata backup](#restore-from-a-metadata-backup Show the contents of the backup. +###### [ `-meta-only-overwrite-force` ] + +Restore *metadata only* from a backup. + +{{% warn %}} +Only use this flag to restore from backups of the target cluster. +If you use this flag with metadata from a different cluster, you will lose data +(since metadata includes shard assignments to data nodes). + +See ["Back up and restore"](/enterprise_influxdb/v1.9/administration/backup-and-restore/#restore-overwrite-metadata-from-a-full-or-incremental-backup-to-fix-damaged-metadata) +for instructions on using this flag. +{{% /warn %}} + ###### [ `-newdb ` ] Name of the new database to restore to. diff --git a/content/enterprise_influxdb/v1.9/troubleshooting/frequently-asked-questions.md b/content/enterprise_influxdb/v1.9/troubleshooting/frequently-asked-questions.md index a429ba366..dd3eb381f 100644 --- a/content/enterprise_influxdb/v1.9/troubleshooting/frequently-asked-questions.md +++ b/content/enterprise_influxdb/v1.9/troubleshooting/frequently-asked-questions.md @@ -177,7 +177,7 @@ an RP every 30 minutes. You may need to wait for the next RP check for InfluxDB to drop data that are outside the RP's new `DURATION` setting. The 30 minute interval is -[configurable](/enterprise_influxdb/v1.9/administration/config-data-nodes/#check-interval--30m0s). +[configurable](/enterprise_influxdb/v1.9/administration/config-data-nodes/#check-interval). Second, altering both the `DURATION` and `SHARD DURATION` of an RP can result in unexpected data retention. diff --git a/content/platform/monitoring/influxdata-platform/tools/measurements-internal.md b/content/platform/monitoring/influxdata-platform/tools/measurements-internal.md index 940f29830..63b781da4 100644 --- a/content/platform/monitoring/influxdata-platform/tools/measurements-internal.md +++ b/content/platform/monitoring/influxdata-platform/tools/measurements-internal.md @@ -278,6 +278,12 @@ The number of remote node requests for information about the fields and associat #### iteratorCostReq The number of internal requests for iterator cost. +#### openConnections + +Tracks the number of open connections +being handled by the data node +(including counting logical connections multiplexed onto a single yamux connection). + #### removeShardReq The number of internal requests to delete a shard from this data node. Exclusively incremented by use of the `influxd-ctl remove shard` command.