From 624eb0205b4a766c58b86a634cd0d5c89e814625 Mon Sep 17 00:00:00 2001 From: Jason Stirnaman Date: Fri, 15 Dec 2023 09:27:57 -0600 Subject: [PATCH] =?UTF-8?q?Revert=20"feature(v3):=20Write:=20partial=20wri?= =?UTF-8?q?tes=20write=20good=20lines=20and=20reject=20bad=20li=E2=80=A6?= =?UTF-8?q?=20(#5169)"=20(#5280)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This reverts commit 90948e67dc6dac6797bf0f555a778756e46f37e8. --- api-docs/cloud-serverless/ref.yml | 182 +++++------------- .../buckets/manage-explicit-bucket-schemas.md | 2 +- .../write-data/troubleshoot.md | 72 ++----- 3 files changed, 68 insertions(+), 188 deletions(-) diff --git a/api-docs/cloud-serverless/ref.yml b/api-docs/cloud-serverless/ref.yml index d408ad857..6071a461c 100644 --- a/api-docs/cloud-serverless/ref.yml +++ b/api-docs/cloud-serverless/ref.yml @@ -13839,37 +13839,24 @@ paths: Use this endpoint to send data in [line protocol](/influxdb/cloud-serverless/reference/syntax/line-protocol/) format to InfluxDB. - InfluxDB Cloud Serverless does the following when you send a write request: + InfluxDB does the following when you send a write request: - 1. Validates the request. - 2. If successful, attempts to [ingest the data](/influxdb/cloud-serverless/reference/internals/durability/#data-ingest); [error](/influxdb/cloud-serverless/write-data/troubleshoot/#review-http-status-codes) otherwise. - 3. If some or all points in the batch are written, responds with an HTTP `204 "No Content"` status code, acknowledging that data is written and queryable; error otherwise. + 1. Validates the request + 2. If successful, attempts to [ingest the data](/influxdb/cloud-serverless/reference/internals/durability/#data-ingest); _error_ otherwise. + 3. If successful, responds with _success_ (HTTP `204` status code), acknowledging that the data is written and queryable; _error_ otherwise. - - `204 "No Content"`: Data in the batch is written and queryable. - - `400 "Bad Request"`: The entire batch is rejected. - - If some points in the batch are rejected, the response body contains details about the [rejected points](/influxdb/cloud-serverless/write-data/troubleshoot/#troubleshoot-rejected-points). - - To ensure that InfluxDB handles writes in the order you request them, - wait for the response before you send the next request. - - #### Write endpoints - - The [`/write`](#operation/GetLegacyQuery) and [`/api/v2/write`](#operation/PostWrite) endpoints are functionally equivalent for writing data to InfluxDB Cloud Serverless. - - - Use the [`/write` endpoint](#operation/GetLegacyQuery) for InfluxDB v1 parameter compatibility. - - Use [`/api/v2/write` endpoint](#operation/PostWrite) for InfluxDB v2 parameter compatibility. + To ensure that InfluxDB Cloud handles writes in the order you request them, + wait for a success response (HTTP `2xx` status code) before you send the next request. #### Rate limits - _Write_ rate limits apply. + `write` rate limits apply. For more information, see [limits and adjustable quotas](/influxdb/cloud-serverless/admin/billing/limits/). #### Related guides - - [Get started writing data](/influxdb/cloud-serverless/get-started/write/) - [Write data with the InfluxDB API](/influxdb/cloud-serverless/get-started/write/) - - [Best practices for writing data](/influxdb/cloud-serverless/write-data/best-practices/) + - [Optimize writes to InfluxDB](/influxdb/cloud-serverless/write-data/best-practices/optimize-writes/) - [Troubleshoot issues writing data](/influxdb/cloud-serverless/write-data/troubleshoot/) operationId: PostWrite parameters: @@ -13992,45 +13979,23 @@ paths: responses: '204': description: | - Success. - Data in the batch is written and queryable. - - If some points in the batch are rejected, the response body contains details about the [rejected points](/influxdb/cloud-serverless/write-data/troubleshoot/#troubleshoot-rejected-points), up to 100 points. - content: - application/json: - examples: - partialWriteErrorWithRejectedPoints: - summary: Partial write rejects points with syntax errors - value: - code: invalid - line: 2 - message: "failed to parse line protocol: - errors encountered on line(s): - error message for first rejected point - error message for second rejected point - error message for Nth rejected point (up to 100 rejected points)" - schema: - $ref: "#/components/schemas/LineProtocolError" + Success. Data is written and queryable. '400': - description: | - All data in the batch was rejected and not written. - - The response body contains details about the [rejected points](/influxdb/cloud-serverless/write-data/troubleshoot/#troubleshoot-rejected-points). content: application/json: examples: - rejectedAllPoints: - summary: Rejected all points + measurementSchemaFieldTypeConflict: + summary: field type conflict thrown by an explicit bucket schema value: code: invalid - line: 2 - message: "failed to parse line protocol: - errors encountered on line(s): - error message for first rejected point - error message for second rejected point - error message for Nth rejected point (up to 100 rejected points)" + message: 'failed to parse line protocol: error writing line 2: Unable to insert iox::column_type::field::integer type into column temp with type iox::column_type::field::string' schema: $ref: '#/components/schemas/LineProtocolError' + description: | + Bad request. The response body contains detail about the error. + + InfluxDB returns this error if the line protocol data in the request is malformed or contains a database schema conflict. + The response body contains the first malformed line in the data, and indicates what was expected. '401': $ref: '#/components/responses/AuthorizationError' '404': @@ -14064,7 +14029,9 @@ paths: The request payload is too large. InfluxDB rejected the batch and did not write any data. - InfluxDB returns this error if the payload exceeds the 50MB size limit. + #### InfluxDB Cloud: + + - Returns this error if the payload exceeds the 50MB size limit. '429': description: | Too many requests. @@ -14270,42 +14237,6 @@ paths: /write: post: operationId: PostLegacyWrite - description: | - Writes data to a bucket. - - Use this endpoint to send data in [line protocol](/influxdb/cloud-serverless/reference/syntax/line-protocol/) format to InfluxDB. - - InfluxDB Cloud Serverless does the following when you send a write request: - - 1. Validates the request. - 2. If successful, attempts to [ingest the data](/influxdb/cloud-serverless/reference/internals/durability/#data-ingest); [error](/influxdb/cloud-serverless/write-data/troubleshoot/#review-http-status-codes) otherwise. - 3. If some or all points in the batch are written, responds with an HTTP `204 "No Content"` status code, acknowledging that data is written and queryable; error otherwise. - - - `204 "No Content"`: Data in the batch is written and queryable. - - `400 "Bad Request"`: The entire batch is rejected. - - If some points in the batch are rejected, the response body contains details about the [rejected points](/influxdb/cloud-serverless/write-data/troubleshoot/#troubleshoot-rejected-points). - - To ensure that InfluxDB handles writes in the order you request them, - wait for the response before you send the next request. - - #### Write endpoints - - The [`/write`](#operation/GetLegacyQuery) and [`/api/v2/write`](#operation/PostWrite) endpoints are functionally equivalent for writing data to InfluxDB Cloud Serverless. - - - Use the [`/write` endpoint](#operation/GetLegacyQuery) for InfluxDB v1 parameter compatibility. - - Use [`/api/v2/write` endpoint](#operation/PostWrite) for InfluxDB v2 parameter compatibility. - - #### Rate limits - - _Write_ rate limits apply. - For more information, see [limits and adjustable quotas](/influxdb/cloud-serverless/admin/billing/limits/). - - #### Related guides - - - [Write data with the InfluxDB API](/influxdb/cloud-serverless/get-started/write/) - - [Optimize writes to InfluxDB](/influxdb/cloud-serverless/write-data/best-practices/optimize-writes/) - - [Troubleshoot issues writing data](/influxdb/cloud-serverless/write-data/troubleshoot/) parameters: - $ref: '#/components/parameters/TraceSpan' - description: The InfluxDB 1.x username to authenticate the request. @@ -14334,7 +14265,7 @@ paths: name: precision schema: type: string - - description: When present, indicates that compression is applied to the line protocol body. + - description: When present, its value indicates to the database that compression is applied to the line protocol body. in: header name: Content-Encoding schema: @@ -14352,47 +14283,14 @@ paths: description: Line protocol body required: true responses: - "204": - description: | - Success. - Data in the batch is written and queryable. - - If some points in the batch are rejected, the response body contains details about the [rejected points](/influxdb/cloud-serverless/write-data/troubleshoot/#troubleshoot-rejected-points), up to 100 points. - content: - application/json: - examples: - partialWriteErrorWithRejectedPoints: - summary: Partial write rejects points with syntax errors - value: - code: invalid - line: 2 - message: "failed to parse line protocol: - errors encountered on line(s): - error message for first rejected point - error message for second rejected point - error message for Nth rejected point (up to 100 rejected points)" - schema: - $ref: "#/components/schemas/LineProtocolError" + '204': + description: Write data is correctly formatted and accepted for writing to the bucket. '400': - description: | - All data in the batch is rejected and not written. - - The response body contains details about the [rejected points](/influxdb/cloud-serverless/write-data/troubleshoot/#troubleshoot-rejected-points). content: application/json: - examples: - rejectsAllPoints: - summary: Rejected all points - value: - code: invalid - line: 2 - message: "failed to parse line protocol: - errors encountered on line(s): - error message for first rejected point - error message for second rejected point - error message for Nth rejected point (up to 100 rejected points)" schema: $ref: '#/components/schemas/LineProtocolError' + description: Line protocol poorly formed and no points were written. Response can be used to determine the first malformed line in the body line-protocol. All data in body was rejected and not written. '401': content: application/json: @@ -14433,9 +14331,32 @@ paths: schema: $ref: '#/components/schemas/Error' description: Internal server error + description: | + Writes data to a bucket. + + Use this InfluxDB v1-compatible endpoint to send data in [line protocol](/influxdb/cloud-serverless/reference/syntax/line-protocol/) format to InfluxDB using v1 API parameters and authorization. + + InfluxDB does the following when you send a write request: + + 1. Validates the request + 2. If successful, attempts to [ingest the data](/influxdb/cloud-serverless/reference/internals/durability/#data-ingest); _error_ otherwise. + 3. If successful, responds with _success_ (HTTP `204` status code), acknowledging that the data is written and queryable; _error_ otherwise. + + To ensure that InfluxDB handles writes in the order you request them, + wait for a success response (HTTP `2xx` status code) before you send the next request. + + #### Rate limits + + `write` rate limits apply. + For more information, see [limits and adjustable quotas](/influxdb/cloud-serverless/admin/billing/limits/). + + #### Related guides + + - [Write data with the InfluxDB API](/influxdb/cloud-serverless/get-started/write/) + - [Optimize writes to InfluxDB](/influxdb/cloud-serverless/write-data/best-practices/optimize-writes/) + - [Troubleshoot issues writing data](/influxdb/cloud-serverless/write-data/troubleshoot/) summary: Write data using the InfluxDB v1 API tags: - - Data I/O endpoints - Write security: - TokenAuthentication: [] @@ -14574,7 +14495,7 @@ tags: |:------------------------ |:--------------------- |:-------------------------------------------| | `Accept` | string | The content type that the client can understand. | | `Authorization` | string | The authorization scheme and credential. | - | `Content-Length` | integer | The size of the entity-body, in bytes. | + | `Content-Length` | integer | The size of the entity-body, in bytes, sent to the database. | | `Content-Type` | string | The format of the data in the request body. | name: Headers x-traitTag: true @@ -14680,9 +14601,8 @@ tags: |  Code  | Status | Description | |:-----------:|:------------------------ |:--------------------- | | `200` | Success | | - | `201` | Created | Successfully created a resource. The response body may contain details. | - | `204` | No content | The request succeeded, but InfluxDB doesn't typically return a response body for the operation. | - | `400` | Bad request | InfluxDB can't parse the request due to an incorrect parameter or bad syntax. For _writes_, the error may indicate one of the following problems: | + | `204` | Success. No content | InfluxDB doesn't return data for the request. For example, a successful write request returns `204` status code, acknowledging that data is written and queryable. | + | `400` | Bad request | InfluxDB can't parse the request due to an incorrect parameter or bad syntax. For _writes_, the error may indicate one of the following problems: | | `401` | Unauthorized | May indicate one of the following: | | `404` | Not found | Requested resource was not found. `message` in the response body provides details about the requested resource. | | `405` | Method not allowed | The API path doesn't support the HTTP method used in the request--for example, you send a `POST` request to an endpoint that only allows `GET`. | diff --git a/content/influxdb/cloud-serverless/admin/buckets/manage-explicit-bucket-schemas.md b/content/influxdb/cloud-serverless/admin/buckets/manage-explicit-bucket-schemas.md index 84e0a0eb4..2889deb5e 100644 --- a/content/influxdb/cloud-serverless/admin/buckets/manage-explicit-bucket-schemas.md +++ b/content/influxdb/cloud-serverless/admin/buckets/manage-explicit-bucket-schemas.md @@ -19,7 +19,7 @@ alt_links: {{% warn %}} #### Don't use explicit schemas with InfluxDB v3 -Don't use **explicit bucket schemas** with InfluxDB v3. +We don't recommend using **explicit bucket schemas** with InfluxDB v3. The sections on this page provide help for managing and troubleshooting `explicit` buckets you may already have. {{% /warn %}} diff --git a/content/influxdb/cloud-serverless/write-data/troubleshoot.md b/content/influxdb/cloud-serverless/write-data/troubleshoot.md index 7a669ed88..f6425927b 100644 --- a/content/influxdb/cloud-serverless/write-data/troubleshoot.md +++ b/content/influxdb/cloud-serverless/write-data/troubleshoot.md @@ -3,9 +3,7 @@ title: Troubleshoot issues writing data seotitle: Troubleshoot issues writing data to InfluxDB weight: 106 description: > - Troubleshoot issues writing data. - Find response codes for failed writes. - Discover how writes fail, from exceeding rate or payload limits, to syntax errors and schema conflicts. + Troubleshoot issues writing data. Find response codes for failed writes. Discover how writes fail, from exceeding rate or payload limits, to syntax errors and schema conflicts. menu: influxdb_cloud_serverless: name: Troubleshoot issues @@ -19,24 +17,23 @@ related: Learn how to avoid unexpected results and recover from errors when writing to {{% product-name %}}. + - [Handle write responses](#handle-write-responses) - [Review HTTP status codes](#review-http-status-codes) - [Troubleshoot failures](#troubleshoot-failures) - [Troubleshoot rejected points](#troubleshoot-rejected-points) + + ## Handle write responses In {{% product-name %}}, writes are synchronous. -After InfluxDB validates the request and attempts to write the data, it responds with a _success_ or _error_ message that indicates the final status of the write. +After InfluxDB validates the request and ingests the data, it sends a _success_ response (HTTP `204` status code) as an acknowledgement that the data is written and queryable. +To ensure that InfluxDB handles writes in the order you request them, wait for the acknowledgement before you send the next request. -A success response (HTTP `204` status code) acknowledges that all data in the batch is written and queryable. - -If InfluxDB responds with an [HTTP error status code](#review-http-status-codes), one or more points weren't written. -The response body contains information about [rejected points](#troubleshoot-rejected-points). -Written points are queryable. - -To ensure that InfluxDB handles writes in the order you request them, wait for the response before you send the next request. +If InfluxDB successfully writes all the request data to the database, it returns _success_ (HTTP `204` status code). +The first rejected point in a batch causes InfluxDB to reject the entire batch and respond with an [HTTP error status](#review-http-status-codes). ### Review HTTP status codes @@ -44,17 +41,17 @@ InfluxDB uses conventional HTTP status codes to indicate the success or failure The `message` property of the response body may contain additional details about the error. Write requests return the following status codes: -| HTTP response code | Response body | Description | -| :-------------------------------| :--------------------------------------------------------------- | :------------- | -| `204 "No Content"` | error details about rejected points, up to 100 points: `line` contains the first rejected line, `message` describes rejected points | If InfluxDB ingested some or all of the data | -| `400 "Bad request"` | `line` contains the first malformed line, `message` describes rejected points | If request data is malformed | +| HTTP response code | Message | Description | +| :-------------------------------| :-----------------------------------------------------------------------| :------------- | +| `204 "Success"` | | If InfluxDB ingested the data | +| `400 "Bad request"` | `message` contains the first malformed line | If request data is malformed | | `401 "Unauthorized"` | | If the `Authorization` header is missing or malformed or if the [token](/influxdb/cloud-serverless/admin/tokens/) doesn't have [permission](/influxdb/cloud-serverless/admin/tokens/create-token/) to write to the bucket. See [examples using credentials](/influxdb/cloud-serverless/get-started/write/#write-line-protocol-to-influxdb) in write requests. | | `403 "Forbidden"` | `message` contains details about the error | If the data isn't allowed (for example, falls outside of the bucket's retention period). | `404 "Not found"` | requested **resource type** (for example, "organization" or "bucket"), and **resource name** | If a requested resource (for example, organization or bucket) wasn't found | | `413 “Request too large”` | cannot read data: points in batch is too large | If a request exceeds the maximum [global limit](/influxdb/cloud-serverless/admin/billing/limits/) | -| `429 “Too many requests”` | | If the number of requests exceeds the [adjustable service quota](/influxdb/cloud-serverless/admin/billing/limits/#adjustable-service-quotas). The `Retry-After` header contains the number of seconds to wait before trying the write again. | If a request exceeds your plan's [adjustable service quotas](/influxdb/cloud-serverless/admin/billing/limits/#adjustable-service-quotas) +| `429 “Too many requests”` | `Retry-After` header: xxx (seconds to wait before retrying the request) | If a request exceeds your plan's [adjustable service quotas](/influxdb/cloud-serverless/admin/billing/limits/#adjustable-service-quotas) | | `500 "Internal server error"` | | Default status for an error | -| `503 "Service unavailable"` | | If the server is temporarily unavailable to accept writes. The `Retry-After` header contains the number of seconds to wait before trying the write again. +| `503` `"Service unavailable"` | | If the server is temporarily unavailable to accept writes. The `Retry-After` header describes when to try the write again. The `message` property of the response body may contain additional details about the error. If your data did not write to the bucket, see how to [troubleshoot rejected points](#troubleshoot-rejected-points). @@ -71,43 +68,6 @@ If you notice data is missing in your database, do the following: ## Troubleshoot rejected points -When writing points from a batch, InfluxDB rejects points that have syntax errors or schema conflicts. +InfluxDB rejects points that fall within the same partition (measurement and day) as existing bucket data and have a different data type for an existing field. -If some points in the batch are written and some are rejected, the InfluxDB HTTP API responds with an HTTP `204 No Content` status code; -if the entire batch is rejected, an HTTP `400 Bad Request` status code. - -If some or all points in the batch are rejected, the HTTP response body contains the following properties: - -- `code`: the status code description for rejected writes is `"invalid"`. -- `line`: the line number of the _first_ rejected point in the batch. -- `message`: a string that contains line-separated error messages, one message for each rejected point in the batch, up to 100 rejected points. - -InfluxDB rejects points for the following reasons: - -- a line protocol parsing error -- an invalid timestamp -- a schema conflict with existing tags or fields in the measurement - -Schema conflicts occur when you try to write data that contains the following: - -- a wrong data type: the point falls within the same partition (default partitioning is measurement and day) as existing bucket data and contains a different data type for an existing field -- a tag and a field that use the same key - -For example, a _partial write_ may occur when InfluxDB writes all points that conform to a series in your bucket, but rejects points that have a different data type in a field. - -### Example - -The following example shows a response body for a write request that contains two rejected points: - -```json -{ - "code": "invalid", - "line": 2, - "message": "failed to parse line protocol: - errors encountered on line(s): - error parsing line 2 (1-based): Invalid measurement was provided - error parsing line 4 (1-based): Unable to parse timestamp value '123461000000000000000000000000'" -} -``` - -Check for [field data type](/influxdb/cloud-serverless/reference/syntax/line-protocol/#data-types-and-format) differences between the rejected data point and points within the same database and partition--for example, did you attempt to write `string` data to an `int` field? +Check for [field data type](/influxdb/cloud-serverless/reference/syntax/line-protocol/#data-types-and-format) differences between the rejected data point and points in the bucket that have the same measurement and day--for example, did you attempt to write `string` data to an `int` field?