diff --git a/content/influxdb/cloud-dedicated/admin/databases/_index.md b/content/influxdb/cloud-dedicated/admin/databases/_index.md index e7e271c1a..39ecc8024 100644 --- a/content/influxdb/cloud-dedicated/admin/databases/_index.md +++ b/content/influxdb/cloud-dedicated/admin/databases/_index.md @@ -71,7 +71,8 @@ cluster in the following ways: {{% expand "**May improve query performance** <em style='opacity:.5;font-weight:normal;'>View more info</em>" %}} Schemas with many measurements that contain -[focused sets of tags and fields](/influxdb/cloud-dedicated/write-data/best-practices/schema-design/#design-for-performance) can make it easier for the query engine to +[focused sets of tags and fields](/influxdb/cloud-dedicated/write-data/best-practices/schema-design/#design-for-performance) +can make it easier for the query engine to identify what partitions contain the queried data, resulting in better query performance. @@ -100,33 +101,32 @@ operating cost of your cluster. ### Column limit -**Default maximum number of columns**: 1000 +**Default maximum number of columns**: 250 -A table can contain **up to 1000 columns**. Each row must include a time column, with the remaining columns representing tags and fields. -As a result, a table can have one time column and up to 999 field and tag columns. - -When creating or updating a database, you can configure the table column limit to be -lower than 1000, based on your requirements. -After you update the column limit for a database, the limit applies to newly -created tables; it doesn't override the column limit for existing tables. +As a result, a table can have one time column and up to 249 field and tag columns. If you attempt to write to a table and exceed the column limit, the write request fails and InfluxDB returns an error. +When creating or updating a database, you can configure the table column limit to be +lower than the default or up to 1000, based on your requirements. +After you update the column limit for a database, the limit applies to newly +created tables; it doesn't override the column limit for existing tables. + Increasing your column limit affects your {{% product-name omit=" Clustered" %}} cluster in the following ways: {{< expand-wrapper >}} {{% expand "May adversely affect system performance" %}} -InfluxData identified 1000 columns as the safe limit for maintaining system +InfluxData identified 250 columns as the safe limit for maintaining system performance and stability. Exceeding this threshold can result in [wide schemas](/influxdb/cloud-dedicated/write-data/best-practices/schema-design/#avoid-wide-schemas), which can negatively impact performance and resource use, -depending on the shape of your schema and data types in the schema. +depending on your queries, the shape of your schema, and data types in the schema. {{% /expand %}} {{< /expand-wrapper >}} diff --git a/content/influxdb/cloud-dedicated/reference/cli/influxctl/database/create.md b/content/influxdb/cloud-dedicated/reference/cli/influxctl/database/create.md index 858e66428..6ef6e4f8e 100644 --- a/content/influxdb/cloud-dedicated/reference/cli/influxctl/database/create.md +++ b/content/influxdb/cloud-dedicated/reference/cli/influxctl/database/create.md @@ -102,9 +102,9 @@ influxctl database create [flags] <DATABASE_NAME> | Flag | | Description | | :--- | :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------- | -| | `--retention-period` | Database retention period (default is `0s`, infinite) | -| | `--max-tables` | Maximum tables per database (default is 500, `0` uses default) | -| | `--max-columns` | Maximum columns per table (default is 1000, `0` uses default) | +| | `--retention-period` | [Database retention period ](/influxdb/cloud-dedicated/admin/databases/#retention-periods)(default is `0s`, infinite) | +| | `--max-tables` | [Maximum tables per database](/influxdb/cloud-dedicated/admin/databases/#table-limit) (default is 500, `0` uses default) | +| | `--max-columns` | [Maximum columns per table](/influxdb/cloud-dedicated/admin/databases/#column-limit) (default is 250, `0` uses default) | | | `--template-tag` | Tag to add to partition template (can include multiple of this flag) | | | `--template-tag-bucket` | Tag and number of buckets to partition tag values into separated by a comma--for example: `tag1,100` (can include multiple of this flag) | | | `--template-timeformat` | Timestamp format for partition template (default is `%Y-%m-%d`) | diff --git a/content/influxdb/cloud-dedicated/reference/cli/influxctl/database/update.md b/content/influxdb/cloud-dedicated/reference/cli/influxctl/database/update.md index 1f076a98e..a743650b3 100644 --- a/content/influxdb/cloud-dedicated/reference/cli/influxctl/database/update.md +++ b/content/influxdb/cloud-dedicated/reference/cli/influxctl/database/update.md @@ -14,6 +14,8 @@ table (measurement), or column limits in InfluxDB. ## Usage +<!-- pytest.mark.skip --> + ```sh influxctl database update [flags] <DATABASE_NAME> ``` @@ -28,9 +30,9 @@ influxctl database update [flags] <DATABASE_NAME> | Flag | | Description | | :--- | :------------------- | :----------------------------------------------------------- | -| | `--retention-period` | Database retention period (default is 0s or infinite) | -| | `--max-tables` | Maximum tables per database (default is 500, 0 uses default) | -| | `--max-columns` | Maximum columns per table (default is 250, 0 uses default) | +| | `--retention-period` | [Database retention period ](/influxdb/cloud-dedicated/admin/databases/#retention-periods)(default is `0s` or infinite) | +| | `--max-tables` | [Maximum tables per database](/influxdb/cloud-dedicated/admin/databases/#table-limit) (default is 500, 0 uses default) | +| | `--max-columns` | [Maximum columns per table](/influxdb/cloud-dedicated/admin/databases/#column-limit) (default is 250, 0 uses default) | | `-h` | `--help` | Output command help | {{% caption %}} diff --git a/content/influxdb/cloud-dedicated/write-data/best-practices/schema-design.md b/content/influxdb/cloud-dedicated/write-data/best-practices/schema-design.md index 3455ffae5..fab4ed0a7 100644 --- a/content/influxdb/cloud-dedicated/write-data/best-practices/schema-design.md +++ b/content/influxdb/cloud-dedicated/write-data/best-practices/schema-design.md @@ -134,11 +134,12 @@ As a result, a table can have the following: If you attempt to write to a table and exceed the column limit, then the write request fails and InfluxDB returns an error. -InfluxData identified 1000 columns as the safe limit for maintaining system +InfluxData identified the default maximum as the safe limit for maintaining system performance and stability. Exceeding this threshold can result in [wide schemas](#avoid-wide-schemas), which can negatively impact performance -and resource use, depending on the shape and data types in your schema. +and resource use, [depending on your queries](#avoid-non-specific-queries), +the shape of your schema, and data types in the schema. --- @@ -162,32 +163,13 @@ Wide schemas can lead to the following issues: - Increased resource usage for persisting and compacting data during ingestion. - Reduced sorting performance due to complex primary keys with [too many tags](#avoid-too-many-tags). -- Reduced query performance when [using non-specific queries](#avoid-non-specific-queries). +- Reduced query performance when + [selecting too many columns](/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/optimize-queries/#select-only-columns-you-need). To prevent wide schema issues, limit the number of tags and fields stored in a table. If you need to store more than the [maximum number of columns](/influxdb/cloud-dedicated/admin/databases/), consider segmenting your fields into separate tables. -#### Avoid non-specific queries - -Because InfluxDB v3 is a columnar database, it only processes the columns -selected in a query, which can mitigate the query performance impact of wide schemas. -If you [query only the data that you need](/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/optimize-queries/#strategies-for-improving-query-performance), -then a wide schema might not impact query performance. - -However, a non-specific query that retrieves a large number of columns from a -wide schema -is slower and less efficient than a more targeted query--for example, consider -the following queries: - -- `SELECT time,a,b,c` -- `SELECT *` - -If the table contains 10 columns, the difference in performance between the -two queries is minimal. -In a table with over 1000 columns, the `SELECT *` query is slower and -less efficient. - #### Avoid too many tags In InfluxDB, the primary key for a row is the combination of the point's diff --git a/content/influxdb/cloud-serverless/admin/buckets/_index.md b/content/influxdb/cloud-serverless/admin/buckets/_index.md index 321456b84..50085343b 100644 --- a/content/influxdb/cloud-serverless/admin/buckets/_index.md +++ b/content/influxdb/cloud-serverless/admin/buckets/_index.md @@ -59,6 +59,7 @@ Each measurement is represented by a table. Time, fields, and tags are each represented by a column. **Maximum number of tables**: 500 + **Maximum number of columns**: 200 The following articles provide information about managing buckets: diff --git a/content/influxdb/cloud-serverless/query-data/troubleshoot-and-optimize/optimize-queries.md b/content/influxdb/cloud-serverless/query-data/troubleshoot-and-optimize/optimize-queries.md index 0eaa274a3..c4856774a 100644 --- a/content/influxdb/cloud-serverless/query-data/troubleshoot-and-optimize/optimize-queries.md +++ b/content/influxdb/cloud-serverless/query-data/troubleshoot-and-optimize/optimize-queries.md @@ -22,6 +22,7 @@ Learn how to use observability tools to analyze query execution and view metrics - [Why is my query slow?](#why-is-my-query-slow) - [Strategies for improving query performance](#strategies-for-improving-query-performance) + - [Query only the data you need](#query-only-the-data-you-need) - [Analyze and troubleshoot queries](#analyze-and-troubleshoot-queries) ## Why is my query slow? diff --git a/content/influxdb/cloud-serverless/write-data/best-practices/schema-design.md b/content/influxdb/cloud-serverless/write-data/best-practices/schema-design.md index ec7525344..b4d834774 100644 --- a/content/influxdb/cloud-serverless/write-data/best-practices/schema-design.md +++ b/content/influxdb/cloud-serverless/write-data/best-practices/schema-design.md @@ -135,32 +135,13 @@ Wide schemas can lead to the following issues: - Increased resource usage for persisting and compacting data during ingestion. - Reduced sorting performance due to complex primary keys with [too many tags](#avoid-too-many-tags). -- Reduced query performance when [using non-specific queries](#avoid-non-specific-queries). +- Reduced query performance when + [selecting too many columns](/influxdb/cloud-dedicated/query-data/troubleshoot-and-optimize/optimize-queries/#select-only-columns-you-need). To prevent wide schema issues, limit the number of tags and fields stored in a measurement. If you need to store more than the [maximum number of columns](/influxdb/cloud-serverless/admin/buckets/), consider segmenting your fields into separate measurements. -#### Avoid non-specific queries - -Because InfluxDB v3 is a columnar database, it only processes the columns -selected in a query, which can mitigate the query performance impact of wide schemas. -If you [query only the data that you need](/influxdb/cloud-serverless/query-data/troubleshoot-and-optimize/optimize-queries/#strategies-for-improving-query-performance), -then a wide schema might not impact query performance. - -However, a non-specific query that retrieves a large number of columns from a -wide schema -is slower and less efficient than a more targeted query--for example, consider -the following queries: - -- `SELECT time,a,b,c` -- `SELECT *` - -If the measurement contains 10 columns, the difference in performance between the -two queries is minimal. -In a measurement with over 1000 columns, the `SELECT *` query is slower and -less efficient. - #### Avoid too many tags In InfluxDB, the primary key for a row is the combination of the point's timestamp and _tag set_ - the collection of [tag keys](/influxdb/cloud-serverless/reference/glossary/#tag-key) and [tag values](/influxdb/cloud-serverless/reference/glossary/#tag-value) on the point. diff --git a/content/influxdb/clustered/admin/databases/_index.md b/content/influxdb/clustered/admin/databases/_index.md index e2907628f..569ac23f3 100644 --- a/content/influxdb/clustered/admin/databases/_index.md +++ b/content/influxdb/clustered/admin/databases/_index.md @@ -27,7 +27,8 @@ Each InfluxDB database has a [retention period](#retention-periods). {{% note %}} **If coming from InfluxDB v1**, the concepts of databases and retention policies have been combined into a single concept--database. Retention policies are no -longer part of the InfluxDB data model. However, {{% product-name %}} does +longer part of the InfluxDB data model. +However, {{% product-name %}} does support InfluxQL, which requires databases and retention policies. See [InfluxQL DBRP naming convention](/influxdb/clustered/admin/databases/create/#influxql-dbrp-naming-convention). @@ -70,7 +71,8 @@ cluster in the following ways: {{% expand "**May improve query performance** <em style='opacity:.5;font-weight:normal;'>View more info</em>" %}} Schemas with many measurements that contain -[focused sets of tags and fields](/influxdb/clustered/write-data/best-practices/schema-design/#design-for-performance) can make it easier for the query engine to +[focused sets of tags and fields](/influxdb/clustered/write-data/best-practices/schema-design/#design-for-performance) +can make it easier for the query engine to identify what partitions contain the queried data, resulting in better query performance. @@ -82,7 +84,7 @@ data by measurement and time range and stores each partition as a Parquet file in your cluster's object store. By increasing the number of measurements (tables) you can store in your database, you also increase the potential for more `PUT` requests into your object store as InfluxDB creates more partitions. -Each `PUT` request incurs a monetary cost and will increase the operating cost of +Each `PUT` request incurs a monetary cost and increases the operating cost of your cluster. {{% /expand %}} @@ -99,33 +101,32 @@ operating cost of your cluster. ### Column limit -**Default maximum number of columns**: 1000 +**Default maximum number of columns**: 250 -A table can contain **up to 1000 columns**. Each row must include a time column, with the remaining columns representing tags and fields. -As a result, a table can have one time column and up to 999 field and tag columns. - -When creating or updating a database, you can configure the column limit to be -lower than 1000, based on your requirements. -After you update the column limit for a database, the limit applies to newly -created tables; doesn't override the column limit for existing tables. +As a result, a table can have one time column and up to 249 field and tag columns. If you attempt to write to a table and exceed the column limit, the write request fails and InfluxDB returns an error. +When creating or updating a database, you can configure the table column limit to be +lower than the default or up to 1000, based on your requirements. +After you update the column limit for a database, the limit applies to newly +created tables; it doesn't override the column limit for existing tables. + Increasing your column limit affects your {{% product-name omit=" Clustered" %}} cluster in the following ways: {{< expand-wrapper >}} {{% expand "May adversely affect system performance" %}} -InfluxData identified 1000 columns as the safe limit for maintaining system +InfluxData identified 250 columns as the safe limit for maintaining system performance and stability. Exceeding this threshold can result in -[wide schemas](/influxdb/cloud-dedicated/write-data/best-practices/schema-design/#avoid-wide-schemas), +[wide schemas](/influxdb/clustered/write-data/best-practices/schema-design/#avoid-wide-schemas), which can negatively impact performance and resource use, -depending on the shape of your schema and data types in the schema. +depending on your queries, the shape of your schema, and data types in the schema. {{% /expand %}} {{< /expand-wrapper >}} diff --git a/content/influxdb/clustered/reference/cli/influxctl/database/create.md b/content/influxdb/clustered/reference/cli/influxctl/database/create.md index 1a6f4d1ed..d18d7cbb0 100644 --- a/content/influxdb/clustered/reference/cli/influxctl/database/create.md +++ b/content/influxdb/clustered/reference/cli/influxctl/database/create.md @@ -101,9 +101,9 @@ influxctl database create [flags] <DATABASE_NAME> | Flag | | Description | | :--- | :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------- | -| | `--retention-period` | Database retention period (default is `0s`, infinite) | -| | `--max-tables` | Maximum tables per database (default is 500, `0` uses default) | -| | `--max-columns` | Maximum columns per table (default is 1000, `0` uses default) | +| | `--retention-period` | [Database retention period ](/influxdb/clustered/admin/databases/#retention-periods)(default is `0s`, infinite) | +| | `--max-tables` | [Maximum tables per database](/influxdb/clustered/admin/databases/#table-limit) (default is 500, `0` uses default) | +| | `--max-columns` | [Maximum columns per table](/influxdb/clustered/admin/databases/#column-limit) (default is 250, `0` uses default) | | | `--template-tag` | Tag to add to partition template (can include multiple of this flag) | | | `--template-tag-bucket` | Tag and number of buckets to partition tag values into separated by a comma--for example: `tag1,100` (can include multiple of this flag) | | | `--template-timeformat` | Timestamp format for partition template (default is `%Y-%m-%d`) | diff --git a/content/influxdb/clustered/reference/cli/influxctl/database/update.md b/content/influxdb/clustered/reference/cli/influxctl/database/update.md index 1afce5b97..d8935f24a 100644 --- a/content/influxdb/clustered/reference/cli/influxctl/database/update.md +++ b/content/influxdb/clustered/reference/cli/influxctl/database/update.md @@ -14,6 +14,8 @@ table (measurement), or column limits in InfluxDB. ## Usage +<!-- pytest.mark.skip --> + ```sh influxctl database update [flags] <DATABASE_NAME> ``` @@ -28,9 +30,9 @@ influxctl database update [flags] <DATABASE_NAME> | Flag | | Description | | :--- | :------------------- | :----------------------------------------------------------- | -| | `--retention-period` | Database retention period (default is 0s or infinite) | -| | `--max-tables` | Maximum tables per database (default is 500, 0 uses default) | -| | `--max-columns` | Maximum columns per table (default is 250, 0 uses default) | +| | `--retention-period` | [Database retention period ](/influxdb/clustered/admin/databases/#retention-periods)(default is `0s` or infinite) | +| | `--max-tables` | [Maximum tables per database](/influxdb/clustered/admin/databases/#table-limit) (default is 500, `0` uses default) | +| | `--max-columns` | [Maximum columns per table](/influxdb/clustered/admin/databases/#column-limit) (default is 250, `0` uses default) | | `-h` | `--help` | Output command help | {{% caption %}} diff --git a/content/influxdb/clustered/write-data/best-practices/schema-design.md b/content/influxdb/clustered/write-data/best-practices/schema-design.md index 1b08b0204..87f2594fb 100644 --- a/content/influxdb/clustered/write-data/best-practices/schema-design.md +++ b/content/influxdb/clustered/write-data/best-practices/schema-design.md @@ -134,11 +134,12 @@ As a result, a table can have the following: If you attempt to write to a table and exceed the column limit, then the write request fails and InfluxDB returns an error. -InfluxData identified 1000 columns as the safe limit for maintaining system +InfluxData identified the default maximum as the safe limit for maintaining system performance and stability. Exceeding this threshold can result in [wide schemas](#avoid-wide-schemas), which can negatively impact performance -and resource use, depending on the shape and data types in your schema. +and resource use, [depending on your queries](#avoid-non-specific-queries), +the shape of your schema, and data types in the schema. --- @@ -162,32 +163,13 @@ Wide schemas can lead to the following issues: - Increased resource usage for persisting and compacting data during ingestion. - Reduced sorting performance due to complex primary keys with [too many tags](#avoid-too-many-tags). -- Reduced query performance when [using non-specific queries](#avoid-non-specific-queries). +- Reduced query performance when + [selecting too many columns](/influxdb/clustered/query-data/troubleshoot-and-optimize/optimize-queries/#select-only-columns-you-need). To prevent wide schema issues, limit the number of tags and fields stored in a table. If you need to store more than the [maximum number of columns](/influxdb/clustered/admin/databases/), consider segmenting your fields into separate tables. -#### Avoid non-specific queries - -Because InfluxDB v3 is a columnar database, it only processes the columns -selected in a query, which can mitigate the query performance impact of wide schemas. -If you [query only the data that you need](/influxdb/clustered/query-data/troubleshoot-and-optimize/optimize-queries/#strategies-for-improving-query-performance), -then a wide schema might not impact query performance. - -However, a non-specific query that retrieves a large number of columns from a -wide schema -is slower and less efficient than a more targeted query--for example, consider -the following queries: - -- `SELECT time,a,b,c` -- `SELECT *` - -If the table contains 10 columns, the difference in performance between the -two queries is minimal. -In a table with over 1000 columns, the `SELECT *` query is slower and -less efficient. - #### Avoid too many tags In InfluxDB, the primary key for a row is the combination of the point's