From fec12d0b2adb3e6ccc3e9dca06d57556b148e0b8 Mon Sep 17 00:00:00 2001 From: Scott Anderson Date: Mon, 31 Jul 2023 17:20:04 -0600 Subject: [PATCH] Product/feature EOL and IOx data retention (#5064) * addes EOL procedures, adds data retention info to serverless and dedicated * Apply suggestions from code review Co-authored-by: Jason Stirnaman * updated DR retention language for dedicated --------- Co-authored-by: Jason Stirnaman --- .../reference/internals/data-retention.md | 45 ++++++++++ .../reference/policies/_index.md | 11 +++ .../reference/policies/end-of-life.md | 85 +++++++++++++++++++ .../reference/internals/data-retention.md | 45 ++++++++++ .../reference/policies/_index.md | 11 +++ .../reference/policies/end-of-life.md | 85 +++++++++++++++++++ content/influxdb/cloud/reference/glossary.md | 2 +- .../cloud/reference/policies/_index.md | 10 +++ .../cloud/reference/policies/end-of-life.md | 85 +++++++++++++++++++ 9 files changed, 378 insertions(+), 1 deletion(-) create mode 100644 content/influxdb/cloud-dedicated/reference/internals/data-retention.md create mode 100644 content/influxdb/cloud-dedicated/reference/policies/_index.md create mode 100644 content/influxdb/cloud-dedicated/reference/policies/end-of-life.md create mode 100644 content/influxdb/cloud-serverless/reference/internals/data-retention.md create mode 100644 content/influxdb/cloud-serverless/reference/policies/_index.md create mode 100644 content/influxdb/cloud-serverless/reference/policies/end-of-life.md create mode 100644 content/influxdb/cloud/reference/policies/_index.md create mode 100644 content/influxdb/cloud/reference/policies/end-of-life.md diff --git a/content/influxdb/cloud-dedicated/reference/internals/data-retention.md b/content/influxdb/cloud-dedicated/reference/internals/data-retention.md new file mode 100644 index 000000000..6228efd7b --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/internals/data-retention.md @@ -0,0 +1,45 @@ +--- +title: Data retention in InfluxDB Cloud Dedicated +description: > + InfluxDB Cloud Dedicated enforces database retention periods at query time + and, to optimize storage, routinely deletes [Parquet](https://parquet.apache.org/) + files containing only expired data. +weight: 103 +menu: + influxdb_cloud_dedicated: + name: Data retention + parent: InfluxDB internals +influxdb/cloud-dedicated/tags: [internals] +--- + +{{< cloud-name >}} enforces database retention periods at query time. +Any points with timestamps beyond a database's [retention period](#database-retention-period) +are filtered out of query results, even though the data may still exist. + +- [Database retention period](#database-retention-period) +- [When does data actually get deleted?](#when-does-data-actually-get-deleted) + +## Database retention period + +A **database retention period** is the duration of time that a database retains data. +Retention periods are designed to automatically delete expired data and optimize +storage without any user intervention. + +Retention periods can be as short as an hour or infinite. +[Points](/influxdb/cloud-dedicated/reference/glossary/#point) in a database with +timestamps beyond the defined retention period (relative to now) are not queryable, +but may still exist in storage until [fully deleted](#when-does-data-actually-get-deleted). + +{{% note %}} +#### View database retention periods + +Use the [`influxctl database list` command](/influxdb/cloud-dedicated/reference/cli/influxctl/database/list/) +to view your databases' retention periods. +{{% /note %}} + +## When does data actually get deleted? + +InfluxDB routinely deletes [Parquet](https://parquet.apache.org/) files containing only expired data. +InfluxDB retains expired Parquet files for approximately 100 days for disaster recovery. +After the disaster recovery period, expired Parquet files are permanently deleted +and can't be recovered. diff --git a/content/influxdb/cloud-dedicated/reference/policies/_index.md b/content/influxdb/cloud-dedicated/reference/policies/_index.md new file mode 100644 index 000000000..bee6ded9a --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/policies/_index.md @@ -0,0 +1,11 @@ +--- +title: Policies and procedures +description: InfluxData product policies and procedures. +menu: + influxdb_cloud_dedicated: + name: Policies & procedures + parent: Reference +weight: 108 +--- + +{{< children >}} diff --git a/content/influxdb/cloud-dedicated/reference/policies/end-of-life.md b/content/influxdb/cloud-dedicated/reference/policies/end-of-life.md new file mode 100644 index 000000000..d5b17caf1 --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/policies/end-of-life.md @@ -0,0 +1,85 @@ +--- +title: Product and feature end-of-life procedures +description: > + InfluxData adheres to the following process for any End-of-Life (EOL) of + products and features, including the shutdown of InfluxDB Cloud regions. +menu: + influxdb_cloud_dedicated: + parent: Policies & procedures + name: End-of-life procedures +weight: 201 +--- + +InfluxData adheres to the following process for any End-of-Life (EOL) of +products and features, including the shutdown of InfluxDB Cloud regions. + +1. **Email Notices**: Customers are given at least six months notice of the + change in service via the following communication methods: + + - Email notifications are sent to all users and billing contacts + associated with the account. + - Emails are sent as "Service Notification" emails that customers and + users cannot opt out of. Service Notifications are limited to critical + service changes or updates and come from a support or formal alert alias. + - Initial email notifications are followed by at least two additional reminders. + For longer notice periods (one year or more), reminders are sent at + least quarterly. As the date approaches, additional reminders are sent. + Any new accounts established after the date of the first email notification + that are impacted by the service shutdown are included in the subsequent + reminders. + +2. **Non-Email Notification**: Notifications and reminders are added to the + following areas: + + - In-app notification for InfluxDB products that include a user interface + (UI) that users must read or acknowledge in order to proceed in the app. + - For annual accounts or accounts billing more than $500 USD per month, an + additional personal outreach is attempted by a Sales Team Member or + Technical Support Team member. + - [docs.influxdata.com](https://docs.influxdata.com) notification informing + visitors of the upcoming end-of-life. The notification remains from the + date of the first email through the end-of-life date. + - Reminders published in the InfluxDB Community Slack channel starting on the + day of the initial email and the day before the event. + - For any planned InfluxDB Cloud cluster removal, a banner notification is + added to the InfluxData status page located at [status.influxdata.com](https://status.influxdata.com) + at the same time the first notification is sent (at least six months in + advance of the shutdown). + +3. **Fail-Safe Controls**: Because the above communication methods may not be + 100% effective, InfluxData implements the following fail-safe controls to + allow for a "scream test" with the ability to notify customers via an outage + and wait to see who responds before shutting down the service or feature + and before deleting any data. At least 30 days before the scheduled service + removal, InfluxData temporarily disables the service (in a fully + reversible manner) for up to 24 hours so that all users relying on the + service should be able to detect the service loss and get assistance. + After 24 hours, the service is returned to normal operation. + Depending on the results of the first "scream test", InfluxData may perform + additional scream tests. + + - As soon as the first "scream test" is started, a banner is added to the + top of the [status.influxdata.com](https://status.influxdata.com) page + advising users about the service removal. This banner stays in place until + the service is fully removed. + - _Feature Flagging_: For reversible actions and other changes that can be + feature flagged, InfluxData adds a feature flag, which enables flexibility + to turn off the service for everyone, but still allow it for individual + accounts in case of emergency. + - _System Backup_: Back up the system at the point in time of execution of + the end-of-life, including data and configuration when appropriate. + - _Export Capability_: For any end-of-life event that would cause customer + data to be deleted, an export capability exists to give the customer a + reasonable method of exporting their data. + - _Waiting Period_: There is at least a 30-day waiting period from disablement + of service before the service is actually removed. This allows 30 days + for customers and users to report a service outage and notify them if they + were not aware of the end-of-life. + +4. **Data Recovery**: If a customer contacts us within the 30-day waiting period, + InfluxData restores the service when possible, or provides backups of data + if a restore of the service is not possible. + +5. **Data Retention**: Data retention in InfluxDB Cloud is described in + InfluxData’s [Data retention documentation](/influxdb/cloud-dedicated/reference/internals/data-retention/) + and SOC-2 Statement. \ No newline at end of file diff --git a/content/influxdb/cloud-serverless/reference/internals/data-retention.md b/content/influxdb/cloud-serverless/reference/internals/data-retention.md new file mode 100644 index 000000000..1b00a9e03 --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/internals/data-retention.md @@ -0,0 +1,45 @@ +--- +title: Data retention in InfluxDB Cloud Serverless +description: > + InfluxDB Cloud Serverless enforces bucket retention periods at query time + and, to optimize storage, routinely deletes [Parquet](https://parquet.apache.org/) + files containing only expired data. +weight: 103 +menu: + influxdb_cloud_serverless: + name: Data retention + parent: InfluxDB Cloud internals +influxdb/cloud-serverless/tags: [internals] +--- + +{{< cloud-name >}} enforces bucket retention periods at query time. +Any points with timestamps beyond a bucket's [retention period](#bucket-retention-period) +are filtered out of query results, even though the data may still exist. + +- [Bucket retention period](#bucket-retention-period) +- [When does data actually get deleted?](#when-does-data-actually-get-deleted) + +## Bucket retention period + +A **bucket retention period** is the duration of time that a bucket retains data. +Retention periods are designed to automatically delete expired data and optimize +storage without any user intervention. + +Retention periods can be as short as an hour or infinite. +[Points](/influxdb/cloud-serverless/reference/glossary/#point) in a bucket with +timestamps beyond the defined retention period (relative to now) are not queryable, +but may still exist in storage until [fully deleted](#when-does-data-actually-get-deleted). + +{{% note %}} +#### View bucket retention periods + +Use the [`influx bucket list` command](/influxdb/cloud-serverless/reference/cli/influx/bucket/list/) +to view your buckets' retention periods. +{{% /note %}} + +## When does data actually get deleted? + +InfluxDB routinely deletes [Parquet](https://parquet.apache.org/) files containing only expired data. +InfluxDB retains expired Parquet files for at least 100 days for disaster recovery. +After the disaster recovery period, expired Parquet files are permanently deleted +and can't be recovered. diff --git a/content/influxdb/cloud-serverless/reference/policies/_index.md b/content/influxdb/cloud-serverless/reference/policies/_index.md new file mode 100644 index 000000000..03c1dc63c --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/policies/_index.md @@ -0,0 +1,11 @@ +--- +title: Policies and procedures +description: InfluxData product policies and procedures. +menu: + influxdb_cloud_serverless: + name: Policies & procedures + parent: Reference +weight: 108 +--- + +{{< children >}} diff --git a/content/influxdb/cloud-serverless/reference/policies/end-of-life.md b/content/influxdb/cloud-serverless/reference/policies/end-of-life.md new file mode 100644 index 000000000..100681ba1 --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/policies/end-of-life.md @@ -0,0 +1,85 @@ +--- +title: Product and feature end-of-life procedures +description: > + InfluxData adheres to the following process for any End-of-Life (EOL) of + products and features, including the shutdown of InfluxDB Cloud regions. +menu: + influxdb_cloud_serverless: + parent: Policies & procedures + name: End-of-life procedures +weight: 201 +--- + +InfluxData adheres to the following process for any End-of-Life (EOL) of +products and features, including the shutdown of InfluxDB Cloud regions. + +1. **Email Notices**: Customers are given at least six months notice of the + change in service via the following communication methods: + + - Email notifications are sent to all users and billing contacts + associated with the account. + - Emails are sent as "Service Notification" emails that customers and + users cannot opt out of. Service Notifications are limited to critical + service changes or updates and come from a support or formal alert alias. + - Initial email notifications are followed by at least two additional reminders. + For longer notice periods (one year or more), reminders are sent at + least quarterly. As the date approaches, additional reminders are sent. + Any new accounts established after the date of the first email notification + that are impacted by the service shutdown are included in the subsequent + reminders. + +2. **Non-Email Notification**: Notifications and reminders are added to the + following areas: + + - In-app notification for InfluxDB products that include a user interface + (UI) that users must read or acknowledge in order to proceed in the app. + - For annual accounts or accounts billing more than $500 USD per month, an + additional personal outreach is attempted by a Sales Team Member or + Technical Support Team member. + - [docs.influxdata.com](https://docs.influxdata.com) notification informing + visitors of the upcoming end-of-life. The notification remains from the + date of the first email through the end-of-life date. + - Reminders published in the InfluxDB Community Slack channel starting on the + day of the initial email and the day before the event. + - For any planned InfluxDB Cloud cluster removal, a banner notification is + added to the InfluxData status page located at [status.influxdata.com](https://status.influxdata.com) + at the same time the first notification is sent (at least six months in + advance of the shutdown). + +3. **Fail-Safe Controls**: Because the above communication methods may not be + 100% effective, InfluxData implements the following fail-safe controls to + allow for a "scream test" with the ability to notify customers via an outage + and wait to see who responds before shutting down the service or feature + and before deleting any data. At least 30 days before the scheduled service + removal, InfluxData temporarily disables the service (in a fully + reversible manner) for up to 24 hours so that all users relying on the + service should be able to detect the service loss and can get assistance. + After 24 hours, the service is returned to normal operation. + Depending on the results of the first "scream test", InfluxData may perform + additional scream tests. + + - As soon as the first "scream test" is started, a banner is added to the + top of the [status.influxdata.com](https://status.influxdata.com) page + advising users about the service removal. This banner stays in place until + the service is fully removed. + - _Feature Flagging_: For reversible actions and other changes that can be + feature flagged, InfluxData adds a feature flag, which enables flexibility + to turn off the service for everyone, but still allow it for individual + accounts in case of emergency. + - _System Backup_: Back up the system at the point in time of execution of + the end-of-life, including data and configuration when appropriate. + - _Export Capability_: For any end-of-life event that would cause customer + data to be deleted, an export capability exists to give the customer a + reasonable method of exporting their data. + - _Waiting Period_: There is at least a 30-day waiting period from disablement + of service before the service is actually removed. This allows 30 days + for customers and users to report a service outage and notify them if they + were not aware of the end-of-life. + +4. **Data Recovery**: If a customer contacts us within the 30-day waiting period, + InfluxData restores the service when possible, or provides backups of data + if a restore of the service is not possible. + +5. **Data Retention**: Data retention in InfluxDB Cloud is described in + InfluxData’s [Data retention documentation](/influxdb/cloud-serverless/reference/internals/data-retention/) + and SOC-2 Statement. \ No newline at end of file diff --git a/content/influxdb/cloud/reference/glossary.md b/content/influxdb/cloud/reference/glossary.md index 5cbdbeb83..aa1e0f169 100644 --- a/content/influxdb/cloud/reference/glossary.md +++ b/content/influxdb/cloud/reference/glossary.md @@ -2,7 +2,7 @@ title: Glossary description: > Terms related to InfluxData products and platforms. -weight: 9 +weight: 10 menu: influxdb_cloud_ref: name: Glossary diff --git a/content/influxdb/cloud/reference/policies/_index.md b/content/influxdb/cloud/reference/policies/_index.md new file mode 100644 index 000000000..990fecac6 --- /dev/null +++ b/content/influxdb/cloud/reference/policies/_index.md @@ -0,0 +1,10 @@ +--- +title: Policies and procedures +description: InfluxData product policies and procedures. +menu: + influxdb_cloud_ref: + name: Policies & procedures +weight: 9 +--- + +{{< children >}} diff --git a/content/influxdb/cloud/reference/policies/end-of-life.md b/content/influxdb/cloud/reference/policies/end-of-life.md new file mode 100644 index 000000000..4e5c7e895 --- /dev/null +++ b/content/influxdb/cloud/reference/policies/end-of-life.md @@ -0,0 +1,85 @@ +--- +title: Product and feature end-of-life procedures +description: > + InfluxData adheres to the following process for any End-of-Life (EOL) of + products and features, including the shutdown of InfluxDB Cloud regions. +menu: + influxdb_cloud_ref: + parent: Policies & procedures + name: End-of-life procedures +weight: 101 +--- + +InfluxData adheres to the following process for any End-of-Life (EOL) of +products and features, including the shutdown of InfluxDB Cloud regions. + +1. **Email Notices**: Customers are given at least six months notice of the + change in service via the following communication methods: + + - Email notifications are sent to all users and billing contacts + associated with the account. + - Emails are sent as "Service Notification" emails that customers and + users cannot opt out of. Service Notifications are limited to critical + service changes or updates and come from a support or formal alert alias. + - Initial email notifications are followed by at least two additional reminders. + For longer notice periods (one year or more), reminders are sent at + least quarterly. As the date approaches, additional reminders are sent. + Any new accounts established after the date of the first email notification + that are impacted by the service shutdown are included in the subsequent + reminders. + +2. **Non-Email Notification**: Notifications and reminders are added to the + following areas: + + - In-app notification for InfluxDB products that include a user interface + (UI) that users must read or acknowledge in order to proceed in the app. + - For annual accounts or accounts billing more than $500 USD per month, an + additional personal outreach is attempted by a Sales Team Member or + Technical Support Team member. + -[docs.influxdata.com](https://docs.influxdata.com) notification informing + visitors of the upcoming end-of-life. The notification remains from the + date of the first email through the end-of-life date. + - Reminders published in the InfluxDB Community Slack channel starting on the + day of the initial email and the day before the event. + - For any planned InfluxDB Cloud cluster removal, a banner notification is + added to the InfluxData status page located at [status.influxdata.com](https://status.influxdata.com) + at the same time the first notification is sent (at least six months in + advance of the shutdown). + +3. **Fail-Safe Controls**: Because the above communication methods may not be + 100% effective, InfluxData implements the following fail-safe controls to + allow for a "scream test" with the ability to notify customers via an outage + and wait to see who responds before shutting down the service or feature + and before deleting any data. At least 30 days before the scheduled service + removal, InfluxData temporarily disables the service (in a fully + reversible manner) for up to 24 hours so that all users relying on the + service should be able to detect the service loss and can get assistance. + After 24 hours, the service is returned to normal operation. + Depending on the results of the first "scream test," InfluxData may perform + additional scream tests. + + - As soon as the first "scream test" is started, a banner is added to the + top of the [status.influxdata.com](https://status.influxdata.com) page + advising users about the service removal. This banner stays in place until + the service is fully removed. + - _Feature Flagging_: For reversible actions and other changes that can be + feature flagged, InfluxData adds a feature flag, which enables flexibility + to turn off the service for everyone, but still allow it for individual + accounts in case of emergency. + - _System Backup_: Back up the system at the point in time of execution of + the end-of-life, including data and configuration when appropriate. + - _Export Capability_: For any end-of-life event that would cause customer + data to be deleted, an export capability exists to give the customer a + reasonable method of exporting their data. + - _Waiting Period_: There is at least a 30-day waiting period from disablement + of service before the service is actually removed. This allows 30 days + for customers and users to report a service outage and notify them if they + were not aware of the end-of-life. + +4. **Data Recovery**: If a customer contacts us within the 30-day waiting period, + InfluxData restores the service when possible, or provides backups of data + if a restore of the service is not possible. + +5. **Data Retention**: Data retention in InfluxDB Cloud is described in + InfluxData’s [Data retention documentation](/influxdb/cloud/reference/internals/data-retention/) + and SOC-2 Statement. \ No newline at end of file