docs-v2/content/shared/influxdb3-write-guides/api-client-libraries.md

577 lines
18 KiB
Markdown

Use the `/api/v3/write_lp` HTTP API endpoint and InfluxDB v3 API clients to write points as line protocol data to {{% product-name %}}.
- [Use the /api/v3/write\_lp endpoint](#use-the-apiv3write_lp-endpoint)
- [Example: write data using the /api/v3 HTTP API](#example-write-data-using-the-apiv3-http-api)
- [Write responses](#write-responses)
- [Use no\_sync for immediate write responses](#use-no_sync-for-immediate-write-responses)
- [Use API client libraries](#use-api-client-libraries)
- [Construct line protocol](#construct-line-protocol)
- [Set up your project](#set-up-your-project)
## Use the /api/v3/write_lp endpoint
{{% product-name %}} adds the `/api/v3/write_lp` endpoint.
{{<api-endpoint endpoint="/api/v3/write_lp?db=mydb&precision=nanosecond&accept_partial=true&no_sync=false" method="post" >}}
This endpoint accepts the same line protocol syntax as [previous versions](/influxdb3/version/write-data/compatibility-apis/),
and supports the following parameters:
- `?accept_partial=<BOOLEAN>`: Accept or reject partial writes (default is `true`).
- `?no_sync=<BOOLEAN>`: Control when writes are acknowledged:
- `no_sync=true`: Acknowledge writes before WAL persistence completes.
- `no_sync=false`: Acknowledges writes after WAL persistence completes (default).
- `?precision=<PRECISION>`: Specify the precision of the timestamp. The default is nanosecond precision.
For more information about the parameters, see [Write data](/influxdb3/version/write-data/).
InfluxData provides supported InfluxDB 3 client libraries that you can integrate with your code
to construct data as time series points, and then write them as line protocol to an {{% product-name %}} database.
For more information, see how to [use InfluxDB client libraries to write data](/influxdb3/version/write-data/client-libraries/).
### Example: write data using the /api/v3 HTTP API
The following examples show how to write data using `curl` and the `/api/3/write_lp` HTTP endpoint.
To show the difference between accepting and rejecting partial writes, line `2` in the example contains a string value (`"hi"`) for a float field (`temp`).
#### Partial write of line protocol occurred
With `accept_partial=true` (default):
```bash
curl -v "http://{{< influxdb/host >}}/api/v3/write_lp?db=sensors&precision=auto" \
--data-raw 'home,room=Sunroom temp=96
home,room=Sunroom temp="hi"'
```
The response is the following:
```
< HTTP/1.1 400 Bad Request
...
{
"error": "partial write of line protocol occurred",
"data": [
{
"original_line": "home,room=Sunroom temp=hi",
"line_number": 2,
"error_message": "invalid column type for column 'temp', expected iox::column_type::field::float, got iox::column_type::field::string"
}
]
}
```
Line `1` is written and queryable.
Line `2` is rejected.
The response is an HTTP error (`400`) status, and the response body contains the error message `partial write of line protocol occurred` with details about the problem line.
#### Parsing failed for write_lp endpoint
With `accept_partial=false`:
```bash
curl -v "http://{{< influxdb/host >}}/api/v3/write_lp?db=sensors&precision=auto&accept_partial=false" \
--data-raw 'home,room=Sunroom temp=96
home,room=Sunroom temp="hi"'
```
The response is the following:
```
< HTTP/1.1 400 Bad Request
...
{
"error": "parsing failed for write_lp endpoint",
"data": {
"original_line": "home,room=Sunroom temp=hi",
"line_number": 2,
"error_message": "invalid column type for column 'temp', expected iox::column_type::field::float, got iox::column_type::field::string"
}
}
```
InfluxDB rejects all points in the batch.
The response is an HTTP error (`400`) status, and the response body contains `parsing failed for write_lp endpoint` and details about the problem line.
For more information about the ingest path and data flow, see [Data durability](/influxdb3/version/reference/internals/durability/).
### Write responses
By default, InfluxDB acknowledges writes after flushing the WAL file to the Object store (occurring every second).
For high write throughput, you can send multiple concurrent write requests.
### Use no_sync for immediate write responses
To reduce the latency of writes, use the `no_sync` write option, which acknowledges writes _before_ WAL persistence completes.
When `no_sync=true`, InfluxDB validates the data, writes the data to the WAL, and then immediately responds to the client, without waiting for persistence to the Object store.
Using `no_sync=true` is best when prioritizing high-throughput writes over absolute durability.
- Default behavior (`no_sync=false`): Waits for data to be written to the Object store before acknowledging the write. Reduces the risk of data loss, but increases the latency of the response.
- With `no_sync=true`: Reduces write latency, but increases the risk of data loss in case of a crash before WAL persistence.
#### Immediate write using the HTTP API
The `no_sync` parameter controls when writes are acknowledged--for example:
```bash
curl "http://localhost:8181/api/v3/write_lp?db=sensors&precision=auto&no_sync=true" \
--data-raw "home,room=Sunroom temp=96"
```
## Use API client libraries
Use InfluxDB 3 client libraries that integrate with your code to construct data
as time series points, and
then write them as line protocol to an {{% product-name %}} database.
- [Construct line protocol](#construct-line-protocol)
- [Example home schema](#example-home-schema)
- [Set up your project](#set-up-your-project)
- [Construct points and write line protocol](#construct-points-and-write-line-protocol)
### Construct line protocol
With a [basic understanding of line protocol](/influxdb3/version/write-data/#line-protocol),
you can construct line protocol data and write it to {{% product-name %}}.
All InfluxDB client libraries write data in line protocol format to InfluxDB.
Client library `write` methods let you provide data as raw line protocol or as
`Point` objects that the client library converts to line protocol. If your
program creates the data you write to InfluxDB, use the client library `Point`
interface to take advantage of type safety in your program.
#### Example home schema
Consider a use case where you collect data from sensors in your home. Each
sensor collects temperature, humidity, and carbon monoxide readings.
To collect this data, use the following schema:
<!-- vale InfluxDataDocs.v3Schema = NO -->
- **table**: `home`
- **tags**
- `room`: Living Room or Kitchen
- **fields**
- `temp`: temperature in °C (float)
- `hum`: percent humidity (float)
- `co`: carbon monoxide in parts per million (integer)
- **timestamp**: Unix timestamp in _second_ precision
<!-- vale InfluxDataDocs.v3Schema = YES -->
The following example shows how to construct and write points that follow the
`home` schema.
### Set up your project
After setting up {{< product-name >}} and your project, you should have the following:
- {{< product-name >}} credentials:
- [Database](/influxdb3/version/admin/databases/)
- Authorization token
> [!Note]
> While in beta, {{< product-name >}} does not require an authorization token.
- {{% product-name %}} URL
- A directory for your project.
- Credentials stored as environment variables or in a project configuration
file--for example, a `.env` ("dotenv") file.
- Client libraries installed for writing data to {{< product-name >}}.
The following examples use InfluxDB 3 client libraries to show how to construct
`Point` objects that follow the [example `home` schema](#example-home-schema),
and then write the data as line protocol to an {{% product-name %}} database.
{{< tabs-wrapper >}}
{{% tabs %}}
[Go](#)
[Node.js](#)
[Python](#)
{{% /tabs %}}
{{% tab-content %}}
The following steps set up a Go project using the
[InfluxDB 3 Go client](https://github.com/InfluxCommunity/influxdb3-go/):
<!-- BEGIN GO PROJECT SETUP -->
1. Install [Go 1.13 or later](https://golang.org/doc/install).
1. Create a directory for your Go module and change to the directory--for
example:
```sh
mkdir iot-starter-go && cd $_
```
1. Initialize a Go module--for example:
```sh
go mod init iot-starter
```
1. Install [`influxdb3-go`](https://github.com/InfluxCommunity/influxdb3-go/),
which provides the InfluxDB `influxdb3` Go client library module.
```sh
go get github.com/InfluxCommunity/influxdb3-go/v2
```
<!-- END GO SETUP PROJECT -->
{{% /tab-content %}} {{% tab-content %}}
<!-- BEGIN NODE.JS PROJECT SETUP -->
The following steps set up a JavaScript project using the
[InfluxDB 3 JavaScript client](https://github.com/InfluxCommunity/influxdb3-js/).
1. Install [Node.js](https://nodejs.org/en/download/).
1. Create a directory for your JavaScript project and change to the
directory--for example:
```sh
mkdir -p iot-starter-js && cd $_
```
1. Initialize a project--for example, using `npm`:
<!-- pytest.mark.skip -->
```sh
npm init
```
1. Install the `@influxdata/influxdb3-client` InfluxDB 3 JavaScript client
library.
```sh
npm install @influxdata/influxdb3-client
```
<!-- END NODE.JS SETUP PROJECT -->
{{% /tab-content %}} {{% tab-content %}}
<!-- BEGIN PYTHON SETUP PROJECT -->
The following steps set up a Python project using the
[InfluxDB 3 Python client](https://github.com/InfluxCommunity/influxdb3-python/):
1. Install [Python](https://www.python.org/downloads/)
1. Inside of your project directory, create a directory for your Python module
and change to the module directory--for example:
```sh
mkdir -p iot-starter-py && cd $_
```
1. **Optional, but recommended**: Use
[`venv`](https://docs.python.org/3/library/venv.html) or
[`conda`](https://docs.continuum.io/anaconda/install/) to activate a virtual
environment for installing and executing code--for example, enter the
following command using `venv` to create and activate a virtual environment
for the project:
```bash
python3 -m venv envs/iot-starter && source ./envs/iot-starter/bin/activate
```
1. Install
[`influxdb3-python`](https://github.com/InfluxCommunity/influxdb3-python),
which provides the InfluxDB `influxdb_client_3` Python client library module
and also installs the
[`pyarrow` package](https://arrow.apache.org/docs/python/index.html) for
working with Arrow data.
```sh
pip install influxdb3-python
```
<!-- END PYTHON SETUP PROJECT -->
{{% /tab-content %}}
{{< /tabs-wrapper >}}
#### Construct points and write line protocol
Client libraries provide one or more `Point` constructor methods. Some libraries
support language-native data structures, such as Go's `struct`, for creating
points.
{{< tabs-wrapper >}}
{{% tabs %}}
[Go](#)
[Node.js](#)
[Python](#)
{{% /tabs %}}
{{% tab-content %}}
<!-- BEGIN GO SETUP SAMPLE -->
1. Create a file for your module--for example: `main.go`.
2. In `main.go`, enter the following sample code:
```go
package main
import (
"context"
"os"
"fmt"
"time"
"github.com/InfluxCommunity/influxdb3-go/v2/influxdb3"
"github.com/influxdata/line-protocol/v2/lineprotocol"
)
func Write() error {
url := os.Getenv("INFLUX_HOST")
token := os.Getenv("INFLUX_TOKEN")
database := os.Getenv("INFLUX_DATABASE")
// To instantiate a client, call New() with InfluxDB credentials.
client, err := influxdb3.New(influxdb3.ClientConfig{
Host: url,
Token: token,
Database: database,
})
/** Use a deferred function to ensure the client is closed when the
* function returns.
**/
defer func (client *influxdb3.Client) {
err = client.Close()
if err != nil {
panic(err)
}
}(client)
/** Use the NewPoint method to construct a point.
* NewPoint(measurement, tags map, fields map, time)
**/
point := influxdb3.NewPoint("home",
map[string]string{
"room": "Living Room",
},
map[string]any{
"temp": 24.5,
"hum": 40.5,
"co": 15i},
time.Now(),
)
/** Use the NewPointWithMeasurement method to construct a point with
* method chaining.
**/
point2 := influxdb3.NewPointWithMeasurement("home").
SetTag("room", "Living Room").
SetField("temp", 23.5).
SetField("hum", 38.0).
SetField("co", 16i).
SetTimestamp(time.Now())
fmt.Println("Writing points")
points := []*influxdb3.Point{point, point2}
/** Write points to InfluxDB.
* You can specify WriteOptions, such as Gzip threshold,
* default tags, and timestamp precision. Default precision is lineprotocol.Nanosecond
**/
err = client.WritePoints(context.Background(), points,
influxdb3.WithPrecision(lineprotocol.Second))
return nil
}
func main() {
Write()
}
```
3. To run the module and write the data to your {{% product-name %}} database,
enter the following command in your terminal:
<!-- pytest.mark.skip -->
```sh
go run main.go
```
<!-- END GO SAMPLE -->
{{% /tab-content %}}
{{% tab-content %}}
<!-- BEGIN NODE.JS SETUP SAMPLE -->
1. Create a file for your module--for example: `write-points.js`.
2. In `write-points.js`, enter the following sample code:
```js
// write-points.js
import { InfluxDBClient, Point } from '@influxdata/influxdb3-client';
/**
* Set InfluxDB credentials.
*/
const host = process.env.INFLUX_HOST ?? '';
const database = process.env.INFLUX_DATABASE;
const token = process.env.INFLUX_TOKEN;
/**
* Write line protocol to InfluxDB using the JavaScript client library.
*/
export async function writePoints() {
/**
* Instantiate an InfluxDBClient.
* Provide the host URL and the database token.
*/
const client = new InfluxDBClient({ host, token });
/** Use the fluent interface with chained methods to construct Points. */
const point = Point.measurement('home')
.setTag('room', 'Living Room')
.setFloatField('temp', 22.2)
.setFloatField('hum', 35.5)
.setIntegerField('co', 7)
.setTimestamp(new Date().getTime() / 1000);
const point2 = Point.measurement('home')
.setTag('room', 'Kitchen')
.setFloatField('temp', 21.0)
.setFloatField('hum', 35.9)
.setIntegerField('co', 0)
.setTimestamp(new Date().getTime() / 1000);
/** Write points to InfluxDB.
* The write method accepts an array of points, the target database, and
* an optional configuration object.
* You can specify WriteOptions, such as Gzip threshold, default tags,
* and timestamp precision. Default precision is lineprotocol.Nanosecond
**/
try {
await client.write([point, point2], database, '', { precision: 's' });
console.log('Data has been written successfully!');
} catch (error) {
console.error(`Error writing data to InfluxDB: ${error.body}`);
}
client.close();
}
writePoints();
```
3. To run the module and write the data to your {{\< product-name >}} database,
enter the following command in your terminal:
<!-- pytest.mark.skip -->
```sh
node writePoints.js
```
<!-- END NODE.JS SAMPLE -->
{{% /tab-content %}}
{{% tab-content %}}
<!-- BEGIN PYTHON SETUP SAMPLE -->
1. Create a file for your module--for example: `write-points.py`.
2. In `write-points.py`, enter the following sample code to write data in
batching mode:
```python
import os
from influxdb_client_3 import (
InfluxDBClient3, InfluxDBError, Point, WritePrecision,
WriteOptions, write_client_options)
host = os.getenv('INFLUX_HOST')
token = os.getenv('INFLUX_TOKEN')
database = os.getenv('INFLUX_DATABASE')
# Create an array of points with tags and fields.
points = [Point("home")
.tag("room", "Kitchen")
.field("temp", 25.3)
.field('hum', 20.2)
.field('co', 9)]
# With batching mode, define callbacks to execute after a successful or
# failed write request.
# Callback methods receive the configuration and data sent in the request.
def success(self, data: str):
print(f"Successfully wrote batch: data: {data}")
def error(self, data: str, exception: InfluxDBError):
print(f"Failed writing batch: config: {self}, data: {data} due: {exception}")
def retry(self, data: str, exception: InfluxDBError):
print(f"Failed retry writing batch: config: {self}, data: {data} retry: {exception}")
# Configure options for batch writing.
write_options = WriteOptions(batch_size=500,
flush_interval=10_000,
jitter_interval=2_000,
retry_interval=5_000,
max_retries=5,
max_retry_delay=30_000,
exponential_base=2)
# Create an options dict that sets callbacks and WriteOptions.
wco = write_client_options(success_callback=success,
error_callback=error,
retry_callback=retry,
write_options=write_options)
# Instantiate a synchronous instance of the client with your
# InfluxDB credentials and write options, such as Gzip threshold, default tags,
# and timestamp precision. Default precision is nanosecond ('ns').
with InfluxDBClient3(host=host,
token=token,
database=database,
write_client_options=wco) as client:
client.write(points, write_precision='s')
```
3. To run the module and write the data to your {{< product-name >}} database,
enter the following command in your terminal:
<!-- pytest.mark.skip -->
```sh
python write-points.py
```
<!-- END PYTHON SETUP PROJECT -->
{{% /tab-content %}} {{< /tabs-wrapper >}}
The sample code does the following:
<!-- vale InfluxDataDocs.v3Schema = NO -->
1. Instantiates a client configured with the InfluxDB URL and API token.
2. Constructs `home`table `Point` objects.
3. Sends data as line protocol format to InfluxDB and waits for the response.
4. If the write succeeds, logs the success message to stdout; otherwise, logs
the failure message and error details.
5. Closes the client to release resources.
<!-- vale InfluxDataDocs.v3Schema = YES -->