diff --git a/content/influxdb/cloud-dedicated/guides/api-compatibility/v1/_index.md b/content/influxdb/cloud-dedicated/guides/api-compatibility/v1/_index.md index c5d91ce1a..b0bc5f648 100644 --- a/content/influxdb/cloud-dedicated/guides/api-compatibility/v1/_index.md +++ b/content/influxdb/cloud-dedicated/guides/api-compatibility/v1/_index.md @@ -310,7 +310,7 @@ Replace the following: ##### Other Telegraf configuration options -`influx_uint_support`: supported in InfluxDB IOx. +`influx_uint_support`: supported in InfluxDB v3. For more plugin options, see [`influxdb`](https://github.com/influxdata/telegraf/blob/master/plugins/outputs/influxdb/README.md) on GitHub. diff --git a/content/influxdb/cloud-dedicated/guides/migrate-data/_index.md b/content/influxdb/cloud-dedicated/guides/migrate-data/_index.md index 253b9b79a..b949137e6 100644 --- a/content/influxdb/cloud-dedicated/guides/migrate-data/_index.md +++ b/content/influxdb/cloud-dedicated/guides/migrate-data/_index.md @@ -36,7 +36,7 @@ The following questions will help guide your decision to migrate. **Yes, you should migrate**. Series cardinality is a major limiting factor with the InfluxDB TSM storage engine. The more unique series in your data, the less performant your database. -The IOx storage engine supports near limitless series cardinality and is, without +The InfluxDB v3 storage engine supports limitless series cardinality and is, without question, the better solution for high series cardinality workloads. #### Do you want to use SQL to query your data? @@ -62,8 +62,8 @@ from the following providers: {{< cloud_regions type=iox-list >}} If your deployment requires other cloud providers or regions, you may need to -wait until the IOx storage engine is available in a region that meets your requirements. -We are currently working to make InfluxDB IOx available on more providers and +wait until the v3 storage engine is available in a region that meets your requirements. +We are currently working to make InfluxDB v3 available on more providers and in more regions around the world. #### Are you reliant on Flux queries and Flux tasks? diff --git a/content/influxdb/cloud-serverless/get-started/write.md b/content/influxdb/cloud-serverless/get-started/write.md index 22fe26884..cd0aa1572 100644 --- a/content/influxdb/cloud-serverless/get-started/write.md +++ b/content/influxdb/cloud-serverless/get-started/write.md @@ -385,7 +385,7 @@ Write data with your existing workloads that already use the InfluxDB v1 `/write {{% note %}} -If migrating data from InfluxDB 1.x, see the [Migrate data from InfluxDB 1.x to InfluxDB {{% product-name %}}](/influxdb/cloud-serverless/guides/migrate-data/migrate-1x-to-iox/) guide. +If migrating data from InfluxDB 1.x, see the [Migrate data from InfluxDB 1.x to InfluxDB {{% product-name %}}](/influxdb/cloud-serverless/guides/migrate-data/migrate-1x-to-serverless/) guide. {{% /note %}} diff --git a/content/influxdb/cloud-serverless/guides/migrate-data/migrate-1x-to-iox.md b/content/influxdb/cloud-serverless/guides/migrate-data/migrate-1x-to-serverless.md similarity index 99% rename from content/influxdb/cloud-serverless/guides/migrate-data/migrate-1x-to-iox.md rename to content/influxdb/cloud-serverless/guides/migrate-data/migrate-1x-to-serverless.md index c1fd77336..3a2c00c25 100644 --- a/content/influxdb/cloud-serverless/guides/migrate-data/migrate-1x-to-iox.md +++ b/content/influxdb/cloud-serverless/guides/migrate-data/migrate-1x-to-serverless.md @@ -10,7 +10,8 @@ menu: parent: Migrate data weight: 103 aliases: - - /influxdb/cloud-serverless/write-data/migrate-data/migrate-1x-to-iox + - /influxdb/cloud-serverless/write-data/migrate-data/migrate-1x-to-iox/ + - /influxdb/cloud-serverless/guides/migrate-data/migrate-1x-to-iox/ --- To migrate data from an InfluxDB 1.x OSS or Enterprise instance to InfluxDB Cloud diff --git a/content/influxdb/cloud-serverless/guides/migrate-data/migrate-tsm-to-iox.md b/content/influxdb/cloud-serverless/guides/migrate-data/migrate-tsm-to-serverless.md similarity index 93% rename from content/influxdb/cloud-serverless/guides/migrate-data/migrate-tsm-to-iox.md rename to content/influxdb/cloud-serverless/guides/migrate-data/migrate-tsm-to-serverless.md index 5f5e1f0e4..ed9dae147 100644 --- a/content/influxdb/cloud-serverless/guides/migrate-data/migrate-tsm-to-iox.md +++ b/content/influxdb/cloud-serverless/guides/migrate-data/migrate-tsm-to-serverless.md @@ -1,5 +1,5 @@ --- -title: Migrate data from TSM to IOx in InfluxDB Cloud +title: Migrate data from TSM to InfluxDB Cloud Serverless description: > To migrate data from a TSM-powered InfluxDB Cloud organization to an InfluxDB Cloud Serverless organization powered by the v3 storage engine, query the data in @@ -12,6 +12,7 @@ menu: weight: 102 aliases: - /influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-iox + - /influxdb/cloud-serverless/guides/migrate-data/migrate-tsm-to-iox alt_links: cloud: /influxdb/cloud/write-data/migrate-data/migrate-cloud-to-cloud/ --- @@ -339,6 +340,7 @@ to allow for variation between batches. So in this example, **it would be best to set your `batchInterval` to `4d`**. ##### Important things to note + - This assumes no other queries are running in your source InfluxDB Cloud organization. - This assumes no other writes are happening in your destination InfluxDB Cloud Serverless organization. {{% /expand %}} @@ -346,7 +348,8 @@ So in this example, **it would be best to set your `batchInterval` to `4d`**. {{< /expand-wrapper >}} ## Monitor the migration progress -The [InfluxDB TSM to IOx Migration Community template](https://github.com/influxdata/community-templates/tree/master/influxdb-tsm-iox-migration/) + +The [InfluxDB TSM to Serverless Migration Community template](https://github.com/influxdata/community-templates/tree/master/influxdb-tsm-iox-migration/) installs the migration task outlined in this guide as well as a dashboard for monitoring running data migrations. @@ -355,14 +358,17 @@ for monitoring running data migrations. Install the InfluxDB Cloud Migration template ## Troubleshoot migration task failures + If the migration task fails, [view your task logs](/influxdb/cloud/process-data/manage-tasks/task-run-history/) to identify the specific error. Below are common causes of migration task failures. - [Exceeded rate limits](#exceeded-rate-limits) - [Invalid API token](#invalid-api-token) - [Query timeout](#query-timeout) +- [Batch size is too large](#batch-size-is-too-large) ### Exceeded rate limits + If your data migration causes you to exceed your InfluxDB Cloud organization's limits and quotas, the task will return an error similar to: @@ -375,20 +381,22 @@ too many requests a smaller interval. Each batch will then query less data. ### Invalid API token -If the API token you add as the `INFLUXDB_SERVERLESS_SECRET` doesn't have write access -to your InfluxDB Cloud Serverless bucket, the task will return an error similar to: + +If the API token you add as the `INFLUXDB_CLOUD_SECRET` doesn't have read access to +your InfluxDB Cloud bucket, the task will return an error similar to: ``` unauthorized access ``` **Possible solutions**: -- Ensure the API token has write access to your InfluxDB Cloud Serverless bucket. -- Generate a new API token with write access to the bucket you want to migrate to. - Then, update the `INFLUXDB_SERVERLESS_TOKEN` secret in your InfluxDB Cloud (TSM) - instance with the new token. +- Ensure the API token has read access to your InfluxDB Cloud bucket. +- Generate a new InfluxDB Cloud API token with read access to the bucket you + want to migrate. Then, update the `INFLUXDB_CLOUD_TOKEN` secret in your + InfluxDB OSS instance with the new token. ### Query timeout + The InfluxDB Cloud query timeout is 90 seconds. If it takes longer than this to return the data from the batch interval, the query will time out and the task will fail. @@ -397,3 +405,15 @@ task will fail. - Update the `migration.batchInterval` setting in your migration task to use a smaller interval. Each batch will then query less data and take less time to return results. + +### Batch size is too large + +If your batch size is too large, the task returns an error similar to the following: + +``` +internal error: error calling function "metadata" @97:1-97:11: error calling function "findRecord" @67:32-67:69: wrong number of fields +``` + +**Possible solutions**: +- Update the `migration.batchInterval` setting in your migration task to use + a smaller interval and retrieve less data per batch. diff --git a/content/influxdb/cloud/account-management/switch-org.md b/content/influxdb/cloud/account-management/switch-org.md index 508b4e06d..e16f7b714 100644 --- a/content/influxdb/cloud/account-management/switch-org.md +++ b/content/influxdb/cloud/account-management/switch-org.md @@ -23,18 +23,19 @@ To switch {{< product-name "short" >}} organizations: 3. Select the organization you want to switch to from the drop-down list. {{% note %}} -#### Migrate to IOx +#### Migrate to InfluxDB Cloud Serverless -To benefit from IOx's unlimited cardinality and support for SQL, [migrate your data to an InfluxDB Cloud Serverless organization](/influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-iox/). +To unlock the benefits of the InfluxDB v3 storage engine, including unlimited +cardinality and SQL, [migrate your data to an InfluxDB Cloud Serverless organization](/influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-iox/). -All InfluxDB Cloud [accounts](/influxdb/cloud-serverless/admin/accounts/) and [organizations](/influxdb/cloud-serverless/admin/organizations/) created through +All InfluxDB Cloud [accounts](/influxdb/cloud-serverless/admin/accounts/) and +[organizations](/influxdb/cloud-serverless/admin/organizations/) created through [cloud2.influxdata.com](https://cloud2.influxdata.com) on or after **January 31, 2023** -are powered by the InfluxDB IOx storage engine. +are on InfluxDB Cloud Serverless and are powered by the InfluxDB v3 storage engine. -To see which storage engine your organization uses, -find the **InfluxDB Cloud powered by** link in your -[InfluxDB Cloud organization homepage](https://cloud2.influxdata.com) version information. -If your organization is using TSM, you'll see **TSM** followed by the version number. -If IOx, you'll see -**InfluxDB Cloud Serverless** followed by the version number. +To see which storage engine your organization uses, find the **InfluxDB Cloud powered by** +link in your [InfluxDB Cloud organization homepage](https://cloud2.influxdata.com) +version information. If your organization is using TSM, you'll see **TSM** +followed by the version number. If Serverless, you'll see **InfluxDB Cloud Serverless** +followed by the version number. {{% /note %}} diff --git a/content/influxdb/cloud/admin/organizations/_index.md b/content/influxdb/cloud/admin/organizations/_index.md index b65285351..49381b663 100644 --- a/content/influxdb/cloud/admin/organizations/_index.md +++ b/content/influxdb/cloud/admin/organizations/_index.md @@ -26,18 +26,19 @@ The following articles provide information about managing organizations: {{< children >}} {{% note %}} -#### Migrate to IOx +#### Migrate to InfluxDB Cloud Serverless -To benefit from IOx's unlimited cardinality and support for SQL, [migrate your data to an InfluxDB Cloud Serverless organization](/influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-iox/). +To unlock the benefits of the InfluxDB v3 storage engine, including unlimited +cardinality and SQL, [migrate your data to an InfluxDB Cloud Serverless organization](/influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-serverless/). -All InfluxDB Cloud [accounts](/influxdb/cloud-serverless/admin/accounts/) and [organizations](/influxdb/cloud-serverless/admin/organizations/) created through +All InfluxDB Cloud [accounts](/influxdb/cloud-serverless/admin/accounts/) and +[organizations](/influxdb/cloud-serverless/admin/organizations/) created through [cloud2.influxdata.com](https://cloud2.influxdata.com) on or after **January 31, 2023** -are powered by the InfluxDB IOx storage engine. +are on InfluxDB Cloud Serverless and are powered by the InfluxDB v3 storage engine. -To see which storage engine your organization is using, -find the **InfluxDB Cloud powered by** link in your -[InfluxDB Cloud organization homepage](https://cloud2.influxdata.com) version information. -If your organization is using TSM, you'll see **TSM** followed by the version number. -If IOx, you'll see -**InfluxDB Cloud Serverless** followed by the version number. -{{% /note %}} \ No newline at end of file +To see which storage engine your organization uses, find the **InfluxDB Cloud powered by** +link in your [InfluxDB Cloud organization homepage](https://cloud2.influxdata.com) +version information. If your organization is using TSM, you'll see **TSM** +followed by the version number. If Serverless, you'll see **InfluxDB Cloud Serverless** +followed by the version number. +{{% /note %}} diff --git a/content/influxdb/cloud/migrate-regions.md b/content/influxdb/cloud/migrate-regions.md index 40b33abb4..7cdd1564c 100644 --- a/content/influxdb/cloud/migrate-regions.md +++ b/content/influxdb/cloud/migrate-regions.md @@ -22,20 +22,21 @@ walks through the migration. The specific process varies depending on whether your destination account is powered by our current database engine, [Time-Structured Merge Tree (TSM)](/influxdb/v2/reference/internals/storage-engine/#time-structured-merge-tree-tsm) -or [our new database engine, InfluxDB IOx](/blog/announcing-general-availability-new-database-engine/). +or [our new database engine, InfluxDB v3](/blog/announcing-general-availability-new-database-engine/). -To benefit from IOx's unlimited cardinality and support for SQL, migrate your data to IOx. +To benefit from the InfluxDB v3 storage engine's unlimited cardinality and +support for SQL, migrate your data to InfluxDB Cloud Serverless. -- [Migrate data TSM to IOx](/influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-iox/) +- [Migrate data TSM to Serverless](/influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-serverless/) - [Migrate data from TSM to TSM](/influxdb/cloud/write-data/migrate-data/migrate-cloud-to-cloud/). -To see which storage engine your organization is using, -find the **InfluxDB Cloud powered by** link in your -[InfluxDB Cloud organization homepage](https://cloud2.influxdata.com) version information. -If your organization is using TSM, you'll see **TSM** followed by the version number. -If IOx, you'll see -**InfluxDB Cloud Serverless** followed by the version number. +To see which storage engine your organization uses, find the **InfluxDB Cloud powered by** +link in your [InfluxDB Cloud organization homepage](https://cloud2.influxdata.com) +version information. If your organization is using TSM, you'll see **TSM** +followed by the version number. If Serverless, you'll see **InfluxDB Cloud Serverless** +followed by the version number. {{% note %}} + #### Dual write into both organizations Depending on the duration of your retention policy for storing data it may be diff --git a/content/influxdb/cloud/upgrade/v2-to-cloud.md b/content/influxdb/cloud/upgrade/v2-to-cloud.md index 010c6826f..3b8151b97 100644 --- a/content/influxdb/cloud/upgrade/v2-to-cloud.md +++ b/content/influxdb/cloud/upgrade/v2-to-cloud.md @@ -11,20 +11,21 @@ weight: 11 --- {{% note %}} -#### Migrate to IOx +#### Migrate to InfluxDB Cloud Serverless -To benefit from IOx's unlimited cardinality and support for SQL, [migrate your data to an InfluxDB Cloud Serverless organization](/influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-iox/). +To unlock the benefits of the InfluxDB v3 storage engine, including unlimited +cardinality and SQL, [migrate your data to an InfluxDB Cloud Serverless organization](/influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-serverless/). -All InfluxDB Cloud [accounts](/influxdb/cloud-serverless/admin/accounts/) and [organizations](/influxdb/cloud-serverless/admin/organizations/) created through +All InfluxDB Cloud [accounts](/influxdb/cloud-serverless/admin/accounts/) and +[organizations](/influxdb/cloud-serverless/admin/organizations/) created through [cloud2.influxdata.com](https://cloud2.influxdata.com) on or after **January 31, 2023** -are powered by the InfluxDB IOx storage engine. +are on InfluxDB Cloud Serverless and are powered by the InfluxDB v3 storage engine. -To see which storage engine your organization uses, -find the **InfluxDB Cloud powered by** link in your -[InfluxDB Cloud organization homepage](https://cloud2.influxdata.com) version information. -If your organization is using TSM, you'll see **TSM** followed by the version number. -If IOx, you'll see -**InfluxDB Cloud Serverless** followed by the version number. +To see which storage engine your organization uses, find the **InfluxDB Cloud powered by** +link in your [InfluxDB Cloud organization homepage](https://cloud2.influxdata.com) +version information. If your organization is using TSM, you'll see **TSM** +followed by the version number. If Serverless, you'll see **InfluxDB Cloud Serverless** +followed by the version number. {{% /note %}} To upgrade from **InfluxDB OSS 2.x** to **InfluxDB Cloud**: @@ -40,6 +41,7 @@ To upgrade from **InfluxDB OSS 2.x** to **InfluxDB Cloud**: {{% note %}} #### Consider when upgrading + - InfluxDB Cloud requires token authentication, and you must create all new API tokens. - InfluxDB Cloud does not support: - Multiple [organizations](/influxdb/cloud/reference/glossary/#organization) per account. @@ -51,12 +53,14 @@ To upgrade from **InfluxDB OSS 2.x** to **InfluxDB Cloud**: {{% /note %}} ## Create an InfluxDB Cloud account + Do one of the following to create an InfluxDB Cloud account: - [Subscribe through InfluxData](/influxdb/cloud/sign-up/#subscribe-through-influxdata) and start for free. - [Subscribe through your cloud provider](/influxdb/cloud/sign-up/#subscribe-through-a-cloud-provider). ## Create an All-Access API token + InfluxDB API tokens are unique to each organization. Create an **All-Access** token in your InfluxDB Cloud user interface (UI) to use for the upgrade process. @@ -76,6 +80,7 @@ You cannot migrate tokens from InfluxDB 2.x to InfluxDB Cloud. _For more information about managing tokens and token types, see [Manage tokens](/influxdb/cloud/admin/tokens/)._ ## Set up influx CLI connection configurations + The `influx` command line interface (CLI) lets you create connection configurations that automatically provides **host**, **organization**, and **API token** credentials to CLI commands. @@ -100,6 +105,7 @@ Include the following flags for each configuration: - **InfluxDB OSS 2.x**: Provide an [Operator token](/influxdb/v2/admin/tokens/#operator-token). ##### Create an InfluxDB Cloud connection configuration + ```sh # Example cloud connection configuration influx config create \ @@ -110,6 +116,7 @@ influx config create \ ``` ##### Create an InfluxDB 2.x connection configuration + {{< keep-url >}} ```sh # Example 2.x connection configuration @@ -121,6 +128,7 @@ influx config create \ ``` ## Use templates to migrate InfluxDB resources + [InfluxDB templates](/influxdb/cloud/tools/influxdb-templates/) let you export InfluxDB [resources](/influxdb/cloud/tools/influxdb-templates/#template-resources) such as buckets, dashboards, labels, tasks, and more and import them into another InfluxDB instance. @@ -128,7 +136,9 @@ Export resources from your **InfluxDB 2.x** instance and migrate them to your **InfluxDB Cloud** instance. {{% note %}} + #### InfluxDB Cloud Free Plan resource limits + If upgrading to an InfluxDB Cloud Free Plan, you are only able to create a [limited number of resources](/influxdb/cloud/account-management/limits/#free-plan-limits). If your exported template exceeds these limits, the resource migration will fail. {{% /note %}} @@ -179,7 +189,9 @@ influx apply --active-config cloud and [`influx export all`](/influxdb/cloud/reference/cli/influx/export/all#examples) documentation. {{% note %}} + #### Update hardcoded InfluxDB URLs + If any of your migrated resources contain hardcoded InfluxDB URLs (`http://localhost:8086`), do one of the following to update these URLs to your [InfluxDB Cloud region URL](/influxdb/cloud/reference/regions/): - Migrate your resources to InfluxDB Cloud, and then update URLs in the InfluxDB Cloud UI. @@ -187,6 +199,7 @@ If any of your migrated resources contain hardcoded InfluxDB URLs (`http://local {{% /note %}} ## Migrate DBRP mappings + InfluxDB database and retention policy (DBRP) mappings let you query InfluxDB Cloud buckets with InfluxQL and the InfluxDB 1.x DBRP convention. **If you have DBRP mappings in your InfluxDB 2.x instance**, migrate them @@ -223,11 +236,13 @@ to your InfluxDB Cloud instance. {{< /expand-wrapper >}} ## Dual write to InfluxDB 2.x and InfluxDB Cloud + Update external clients to write to your InfluxDB Cloud instance. **We recommend writing data to both InfluxDB 2.x and InfluxDB Cloud until you finish [migrating your existing time series data](#migrate-time-series-data)**. For step-by-step instructions, see [Dual write](/influxdb/cloud/write-data/no-code/use-telegraf/dual-write/). ## Migrate time series data + To migrate your time series data from your InfluxDB 2.x instance to your InfluxDB Cloud instance, do the following: @@ -278,7 +293,9 @@ InfluxDB Cloud instance, do the following: 4. Repeat steps 2-3 for each bucket. {{% note %}} + #### InfluxDB Cloud write rate limits + Write requests are subject to rate limits associated with your [InfluxDB Cloud pricing plan](/influxdb/cloud/account-management/pricing-plans/). If your exported line protocol size potentially exceeds your rate limits, @@ -313,6 +330,7 @@ rate limits track the payload size of the **uncompressed** line protocol. {{% /note %}} #### Migrate system buckets + InfluxDB [system buckets](/influxdb/cloud/reference/internals/system-buckets/) contain data related to the InfluxDB monitoring and alerting system. Although the retention period for system buckets in both InfluxDB Cloud and @@ -320,6 +338,7 @@ InfluxDB 2.x is only seven days, if you want to migrate this data, use the same method described above [to migrate time series data](#migrate-time-series-data). #### Export and write data in a single command + If your data and rate limits allow, you can export and write data in a single command without writing a line protocol export file to disk. The `influxd inspect export-lp` command can output to **stdout** and the `influx write` @@ -343,5 +362,6 @@ influx write \ {{< /expand-wrapper >}} ## Collaborate with other users + To collaborate with other users in your InfluxDB Cloud organization, [invite users to join your organization](/influxdb/cloud/admin/organizations/users/#invite-a-user-to-your-organization/). diff --git a/content/influxdb/cloud/write-data/migrate-data/migrate-cloud-to-cloud.md b/content/influxdb/cloud/write-data/migrate-data/migrate-cloud-to-cloud.md index 5d99f6f6c..a708f403d 100644 --- a/content/influxdb/cloud/write-data/migrate-data/migrate-cloud-to-cloud.md +++ b/content/influxdb/cloud/write-data/migrate-data/migrate-cloud-to-cloud.md @@ -12,7 +12,7 @@ aliases: - /influxdb/cloud/migrate-data/migrate-cloud-to-cloud/ weight: 102 alt_links: - cloud-serverless: /influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-iox/ + cloud-serverless: /influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-serverless/ --- To migrate data from one InfluxDB Cloud organization to another, query the @@ -37,7 +37,7 @@ All query and write requests are subject to your InfluxDB Cloud organization's - [Configuration help](#configuration-help) - [Monitor the migration progress](#monitor-the-migration-progress) - [Troubleshoot migration task failures](#troubleshoot-migration-task-failures) -- [Migrate to InfluxDB Cloud Serverless powered by IOx](#migrate-to-influxdb-cloud-serverless-powered-by-iox) +- [Migrate to InfluxDB Cloud Serverless](#migrate-to-influxdb-cloud-serverless) ## Set up the migration @@ -326,6 +326,7 @@ to allow for variation between batches. So in this example, **it would be best to set your `batchInterval` to `4d`**. ##### Important things to note + - This assumes no other queries are running in your source InfluxDB Cloud organization. - This assumes no other writes are happening in your destination InfluxDB Cloud organization. {{% /expand %}} @@ -333,6 +334,7 @@ So in this example, **it would be best to set your `batchInterval` to `4d`**. {{< /expand-wrapper >}} ## Monitor the migration progress + The [InfluxDB Cloud Migration Community template](https://github.com/influxdata/community-templates/tree/master/influxdb-cloud-oss-migration/) installs the migration task outlined in this guide as well as a dashboard for monitoring running data migrations. @@ -342,14 +344,17 @@ for monitoring running data migrations. Install the InfluxDB Cloud Migration template ## Troubleshoot migration task failures + If the migration task fails, [view your task logs](/influxdb/cloud/process-data/manage-tasks/task-run-history/) to identify the specific error. Below are common causes of migration task failures. - [Exceeded rate limits](#exceeded-rate-limits) - [Invalid API token](#invalid-api-token) - [Query timeout](#query-timeout) +- [Batch size is too large](#batch-size-is-too-large) ### Exceeded rate limits + If your data migration causes you to exceed your InfluxDB Cloud organization's limits and quotas, the task will return an error similar to: @@ -362,6 +367,7 @@ too many requests a smaller interval. Each batch will then query less data. ### Invalid API token + If the API token you add as the `INFLUXDB_CLOUD_SECRET` doesn't have read access to your InfluxDB Cloud bucket, the task will return an error similar to: @@ -376,6 +382,7 @@ unauthorized access InfluxDB OSS instance with the new token. ### Query timeout + The InfluxDB Cloud query timeout is 90 seconds. If it takes longer than this to return the data from the batch interval, the query will time out and the task will fail. @@ -385,17 +392,30 @@ task will fail. a smaller interval. Each batch will then query less data and take less time to return results. -### Migrate to InfluxDB Cloud Serverless powered by IOx +### Batch size is too large -To unlock the benefits of the IOx storage engine, including unlimited cardinality and SQL, [migrate your data to an InfluxDB Cloud Serverless organization](/influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-iox/). +If your batch size is too large, the task returns an error similar to the following: -All InfluxDB Cloud [accounts](/influxdb/cloud-serverless/admin/accounts/) and [organizations](/influxdb/cloud-serverless/admin/organizations/) created through +``` +internal error: error calling function "metadata" @97:1-97:11: error calling function "findRecord" @67:32-67:69: wrong number of fields +``` + +**Possible solutions**: +- Update the `migration.batchInterval` setting in your migration task to use + a smaller interval and retrieve less data per batch. + +### Migrate to InfluxDB Cloud Serverless + +To unlock the benefits of the InfluxDB v3 storage engine, including unlimited +cardinality and SQL, [migrate your data to an InfluxDB Cloud Serverless organization](/influxdb/cloud-serverless/write-data/migrate-data/migrate-tsm-to-serverless/). + +All InfluxDB Cloud [accounts](/influxdb/cloud-serverless/admin/accounts/) and +[organizations](/influxdb/cloud-serverless/admin/organizations/) created through [cloud2.influxdata.com](https://cloud2.influxdata.com) on or after **January 31, 2023** -are powered by the InfluxDB IOx storage engine. +are on InfluxDB Cloud Serverless and are powered by the InfluxDB v3 storage engine. -To see which storage engine your organization uses, -find the **InfluxDB Cloud powered by** link in your -[InfluxDB Cloud organization homepage](https://cloud2.influxdata.com) version information. -If your organization is using TSM, you'll see **TSM** followed by the version number. -If IOx, you'll see -**InfluxDB Cloud Serverless** followed by the version number. +To see which storage engine your organization uses, find the **InfluxDB Cloud powered by** +link in your [InfluxDB Cloud organization homepage](https://cloud2.influxdata.com) +version information. If your organization is using TSM, you'll see **TSM** +followed by the version number. If Serverless, you'll see **InfluxDB Cloud Serverless** +followed by the version number. diff --git a/content/influxdb/v2/write-data/migrate-data/migrate-cloud-to-oss.md b/content/influxdb/v2/write-data/migrate-data/migrate-cloud-to-oss.md index 85e3b99ac..a4ea7e3a3 100644 --- a/content/influxdb/v2/write-data/migrate-data/migrate-cloud-to-oss.md +++ b/content/influxdb/v2/write-data/migrate-data/migrate-cloud-to-oss.md @@ -35,6 +35,7 @@ All queries against data in InfluxDB Cloud are subject to your organization's - [Troubleshoot migration task failures](#troubleshoot-migration-task-failures) ## Set up the migration + 1. [Install and set up InfluxDB OSS](/influxdb/{{< current-version-link >}}/install/). 2. **In InfluxDB Cloud**, [create an API token](/influxdb/cloud/admin/tokens/create-token/) @@ -69,6 +70,7 @@ Batch range is beyond the migration range. Migration is complete. ## Migration task ### Configure the migration + 1. Specify how often you want the task to run using the `task.every` option. _See [Determine your task interval](#determine-your-task-interval)._ @@ -313,6 +315,7 @@ to allow for variation between batches. So in this example, **it would be best to set your `batchInterval` to `35d`**. ##### Important things to note + - This assumes no other queries are running in your InfluxDB Cloud organization. - You should also consider your network speeds and whether a batch can be fully downloaded within the [task interval](#determine-your-task-interval). @@ -322,6 +325,7 @@ So in this example, **it would be best to set your `batchInterval` to `35d`**. {{< /expand-wrapper >}} ## Monitor the migration progress + The [InfluxDB Cloud Migration Community template](https://github.com/influxdata/community-templates/tree/master/influxdb-cloud-oss-migration/) installs the migration task outlined in this guide as well as a dashboard for monitoring running data migrations. @@ -331,14 +335,17 @@ for monitoring running data migrations. Install the InfluxDB Cloud Migration template ## Troubleshoot migration task failures + If the migration task fails, [view your task logs](/influxdb/v2/process-data/manage-tasks/task-run-history/) to identify the specific error. Below are common causes of migration task failures. - [Exceeded rate limits](#exceeded-rate-limits) - [Invalid API token](#invalid-api-token) - [Query timeout](#query-timeout) +- [Batch size is too large](#batch-size-is-too-large) ### Exceeded rate limits + If your data migration causes you to exceed your InfluxDB Cloud organization's limits and quotas, the task will return an error similar to: @@ -351,6 +358,7 @@ too many requests a smaller interval. Each batch will then query less data. ### Invalid API token + If the API token you add as the `INFLUXDB_CLOUD_SECRET` doesn't have read access to your InfluxDB Cloud bucket, the task will return an error similar to: @@ -365,6 +373,7 @@ unauthorized access InfluxDB OSS instance with the new token. ### Query timeout + The InfluxDB Cloud query timeout is 90 seconds. If it takes longer than this to return the data from the batch interval, the query will time out and the task will fail. @@ -373,3 +382,15 @@ task will fail. - Update the `migration.batchInterval` setting in your migration task to use a smaller interval. Each batch will then query less data and take less time to return results. + +### Batch size is too large + +If your batch size is too large, the task returns an error similar to the following: + +``` +internal error: error calling function "metadata" @97:1-97:11: error calling function "findRecord" @67:32-67:69: wrong number of fields +``` + +**Possible solutions**: +- Update the `migration.batchInterval` setting in your migration task to use + a smaller interval and retrieve less data per batch.