fix: replaced bad anchor ref (#2490)
Jason Stirnaman
parent
c529ecd958
commit
316c6b2573
|
|
@ -16,24 +16,26 @@ related:
|
|||
- /influxdb/v2.0/reference/syntax/annotated-csv/
|
||||
- /influxdb/v2.0/reference/cli/influx/write/
|
||||
---
|
||||
|
||||
Write CSV data with the following methods:
|
||||
|
||||
- [Flux](#flux)
|
||||
- [influx write command](#influx-write-command)
|
||||
|
||||
### Flux
|
||||
|
||||
Use the [csv.from()](/influxdb/v2.0/reference/flux/stdlib/csv/from/) and [to()](/influxdb/v2.0/reference/flux/stdlib/built-in/outputs/to/) Flux functions to write an annotated CSV to the bucket of your choice.
|
||||
|
||||
|
||||
{{< youtube wPKZ9i0DulQ >}}
|
||||
|
||||
The experimental [csv.from()](/influxdb/v2.0/reference/flux/stdlib/csv/from/) function lets you write CSV from a URL.
|
||||
The example below writes [NOAA water sample data](/influxdb/v2.0/reference/sample-data/#noaa-water-sample-data) to an example `noaa` bucket in an example organization:
|
||||
|
||||
```js
|
||||
import "experimental/csv"
|
||||
import "experimental/csv";
|
||||
|
||||
csv.from(url: "https://influx-testdata.s3.amazonaws.com/noaa.csv")
|
||||
|> to(bucket: "noaa", org: "example-org")
|
||||
csv.from((url: "https://influx-testdata.s3.amazonaws.com/noaa.csv"))
|
||||
|> to((bucket: "noaa"), (org: "example-org"));
|
||||
```
|
||||
|
||||
### influx write command
|
||||
|
|
@ -45,6 +47,7 @@ Include annotations in the CSV file or inject them using the `--header` flag of
|
|||
the `influx write` command.
|
||||
|
||||
##### On this page
|
||||
|
||||
- [CSV Annotations](#csv-annotations)
|
||||
- [Inject annotation headers](#inject-annotation-headers)
|
||||
- [Skip annotation headers](#skip-annotation-headers)
|
||||
|
|
@ -54,11 +57,13 @@ the `influx write` command.
|
|||
- [Advanced examples](#advanced-examples)
|
||||
|
||||
##### Example write command
|
||||
|
||||
```sh
|
||||
influx write -b example-bucket -f path/to/example.csv
|
||||
```
|
||||
|
||||
##### example.csv
|
||||
|
||||
```
|
||||
#datatype measurement,tag,double,dateTime:RFC3339
|
||||
m,host,used_percent,time
|
||||
|
|
@ -71,6 +76,7 @@ mem,host2,73.77,2020-01-01T00:00:20Z
|
|||
```
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
mem,host=host1 used_percent=64.23 1577836800000000000
|
||||
mem,host=host2 used_percent=72.01 1577836800000000000
|
||||
|
|
@ -86,6 +92,7 @@ command to print the resulting line protocol to stdout rather than write to Infl
|
|||
{{% /note %}}
|
||||
|
||||
{{% note %}}
|
||||
|
||||
##### "too many open files" errors
|
||||
|
||||
When attempting to write large amounts of CSV data into InfluxDB, you might see an error like the following:
|
||||
|
|
@ -99,9 +106,11 @@ To fix this error, run the following command to increase the number of open file
|
|||
```
|
||||
ulimit -n 10000
|
||||
```
|
||||
|
||||
{{% /note %}}
|
||||
|
||||
## CSV Annotations
|
||||
|
||||
Use **CSV annotations** to specify which element of line protocol each CSV column
|
||||
represents and how to format the data. CSV annotations are rows at the beginning
|
||||
of a CSV file that describe column properties.
|
||||
|
|
@ -120,10 +129,12 @@ To write data to InfluxDB, data must include the following:
|
|||
Use CSV annotations to specify which of these elements each column represents.
|
||||
|
||||
## Write raw query results back to InfluxDB
|
||||
|
||||
Flux returns query results in [annotated CSV](/influxdb/v2.0/reference/syntax/annotated-csv/).
|
||||
These results include all annotations necessary to write the data back to InfluxDB.
|
||||
|
||||
## Inject annotation headers
|
||||
|
||||
If the CSV data you want to write to InfluxDB does not contain the annotations
|
||||
required to properly convert the data to line protocol, use the `--header` flag
|
||||
to inject annotation rows into the CSV data.
|
||||
|
|
@ -137,7 +148,9 @@ influx write -b example-bucket \
|
|||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### example.csv
|
||||
|
||||
```
|
||||
date,sighted,loc
|
||||
2020-01-01,12,Boise
|
||||
|
|
@ -147,9 +160,12 @@ date,sighted,loc
|
|||
2020-01-01,9,Detroit
|
||||
2020-06-01,135,Detroit
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
birds,loc=Boise sighted=12i 1577836800000000000
|
||||
birds,loc=Boise sighted=78i 1590969600000000000
|
||||
|
|
@ -158,10 +174,12 @@ birds,loc=Seattle sighted=112i 1590969600000000000
|
|||
birds,loc=Detroit sighted=9i 1577836800000000000
|
||||
birds,loc=Detroit sighted=135i 1590969600000000000
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
||||
#### Use files to inject headers
|
||||
|
||||
The `influx write` command supports importing multiple files in a single command.
|
||||
Include annotations and header rows in their own file and import them with the write command.
|
||||
Files are read in the order in which they're provided.
|
||||
|
|
@ -174,14 +192,19 @@ influx write -b example-bucket \
|
|||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### headers.csv
|
||||
|
||||
```
|
||||
#constant measurement,birds
|
||||
#datatype dateTime:2006-01-02,long,tag
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### example.csv
|
||||
|
||||
```
|
||||
date,sighted,loc
|
||||
2020-01-01,12,Boise
|
||||
|
|
@ -191,10 +214,12 @@ date,sighted,loc
|
|||
2020-01-01,9,Detroit
|
||||
2020-06-01,135,Detroit
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
birds,loc=Boise sighted=12i 1577836800000000000
|
||||
birds,loc=Boise sighted=78i 1590969600000000000
|
||||
|
|
@ -205,6 +230,7 @@ birds,loc=Detroit sighted=135i 1590969600000000000
|
|||
```
|
||||
|
||||
## Skip annotation headers
|
||||
|
||||
Some CSV data may include header rows that conflict with or lack the annotations
|
||||
necessary to write CSV data to InfluxDB.
|
||||
Use the `--skipHeader` flag to specify the **number of rows to skip** at the
|
||||
|
|
@ -220,6 +246,7 @@ You can then [inject new header rows](#inject-annotation-headers) to rename colu
|
|||
and provide the necessary annotations.
|
||||
|
||||
## Process input as CSV
|
||||
|
||||
The `influx write` command automatically processes files with the `.csv` extension as CSV files.
|
||||
If your CSV file uses a different extension, use the `--format` flat to explicitly
|
||||
declare the format of the input file.
|
||||
|
|
@ -236,6 +263,7 @@ include the `.csv` extension or you declare the `csv`.
|
|||
{{% /note %}}
|
||||
|
||||
## Specify CSV character encoding
|
||||
|
||||
The `influx write` command assumes CSV files contain UTF-8 encoded characters.
|
||||
If your CSV data uses different character encoding, specify the encoding
|
||||
with the `--encoding`.
|
||||
|
|
@ -247,6 +275,7 @@ influx write -b example-bucket \
|
|||
```
|
||||
|
||||
## Skip rows with errors
|
||||
|
||||
If a row in your CSV data is missing an
|
||||
[element required to write to InfluxDB](/influxdb/v2.0/reference/syntax/line-protocol/#elements-of-line-protocol)
|
||||
or data is incorrectly formatted, when processing the row, the `influx write` command
|
||||
|
|
@ -283,12 +312,15 @@ cpu,1.1
|
|||
---
|
||||
|
||||
### Define constants
|
||||
|
||||
Use the Extended annotated CSV [`#constant` annotation](/influxdb/v2.0/reference/syntax/annotated-csv/extended/#constant)
|
||||
to add a column and value to each row in the CSV data.
|
||||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### CSV with constants
|
||||
|
||||
```
|
||||
#constant measurement,example
|
||||
#constant tag,source,csv
|
||||
|
|
@ -299,27 +331,34 @@ count,time
|
|||
9,2020-01-03T00:00:00Z
|
||||
18,2020-01-04T00:00:00Z
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
example,source=csv count=1 1577836800000000000
|
||||
example,source=csv count=4 1577923200000000000
|
||||
example,source=csv count=9 1578009600000000000
|
||||
example,source=csv count=18 1578096000000000000
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
||||
---
|
||||
|
||||
### Annotation shorthand
|
||||
|
||||
Extended annotated CSV supports [annotation shorthand](/influxdb/v2.0/reference/syntax/annotated-csv/extended/#annotation-shorthand),
|
||||
which lets you define the **column label**, **datatype**, and **default value** in the column header.
|
||||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### CSV with annotation shorthand
|
||||
|
||||
```
|
||||
m|measurement,count|long|0,time|dateTime:RFC3339
|
||||
example,1,2020-01-01T00:00:00Z
|
||||
|
|
@ -327,19 +366,24 @@ example,4,2020-01-02T00:00:00Z
|
|||
example,,2020-01-03T00:00:00Z
|
||||
example,18,2020-01-04T00:00:00Z
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
example count=1 1577836800000000000
|
||||
example count=4 1577923200000000000
|
||||
example count=0 1578009600000000000
|
||||
example count=18 1578096000000000000
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
||||
#### Replace column header with annotation shorthand
|
||||
|
||||
It's possible to replace the column header row in a CSV file with annotation
|
||||
shorthand without modifying the CSV file.
|
||||
This lets you define column data types and default values while writing to InfluxDB.
|
||||
|
|
@ -362,7 +406,9 @@ influx write -b example-bucket \
|
|||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Unmodified example.csv
|
||||
|
||||
```
|
||||
m,count,time
|
||||
example,1,2020-01-01T00:00:00Z
|
||||
|
|
@ -370,27 +416,34 @@ example,4,2020-01-02T00:00:00Z
|
|||
example,,2020-01-03T00:00:00Z
|
||||
example,18,2020-01-04T00:00:00Z
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
example count=1i 1577836800000000000
|
||||
example count=4i 1577923200000000000
|
||||
example count=0i 1578009600000000000
|
||||
example count=18i 1578096000000000000
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
||||
---
|
||||
|
||||
### Ignore columns
|
||||
|
||||
Use the Extended annotated CSV [`#datatype ignored` annotation](/influxdb/v2.0/reference/syntax/annotated-csv/extended/#ignored)
|
||||
to ignore columns when writing CSV data to InfluxDB.
|
||||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### CSV data with ignored column
|
||||
|
||||
```
|
||||
#datatype measurement,long,time,ignored
|
||||
m,count,time,foo
|
||||
|
|
@ -399,21 +452,26 @@ example,4,2020-01-02T00:00:00Z,bar
|
|||
example,9,2020-01-03T00:00:00Z,baz
|
||||
example,18,2020-01-04T00:00:00Z,baz
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
m count=1i 1577836800000000000
|
||||
m count=4i 1577923200000000000
|
||||
m count=9i 1578009600000000000
|
||||
m count=18i 1578096000000000000
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
||||
---
|
||||
|
||||
### Use alternate numeric formats
|
||||
|
||||
If your CSV data contains numeric values that use a non-default fraction separator (`.`)
|
||||
or contain group separators, [define your numeric format](/influxdb/v2.0/reference/syntax/annotated-csv/extended/#double)
|
||||
in the `double`, `long`, and `unsignedLong` datatype annotations.
|
||||
|
|
@ -421,7 +479,7 @@ in the `double`, `long`, and `unsignedLong` datatype annotations.
|
|||
{{% note %}}
|
||||
If your **numeric format separators** include a comma (`,`), wrap the column annotation in double
|
||||
quotes (`""`) to prevent the comma from being parsed as a column separator or delimiter.
|
||||
You can also [define a custom column separator](##################).
|
||||
You can also [define a custom column separator](/influxdb/v2.0/reference/syntax/annotated-csv/extended/#define-custom-column-separator).
|
||||
{{% /note %}}
|
||||
|
||||
{{< tabs-wrapper >}}
|
||||
|
|
@ -433,7 +491,9 @@ You can also [define a custom column separator](##################).
|
|||
{{% tab-content %}}
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### CSV with non-default float values
|
||||
|
||||
```
|
||||
#datatype measurement,"double:.,",dateTime:RFC3339
|
||||
m,lbs,time
|
||||
|
|
@ -442,15 +502,19 @@ example,"1,352.5",2020-01-02T00:00:00Z
|
|||
example,"1,862.8",2020-01-03T00:00:00Z
|
||||
example,"2,014.9",2020-01-04T00:00:00Z
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
example lbs=1280.7 1577836800000000000
|
||||
example lbs=1352.5 1577923200000000000
|
||||
example lbs=1862.8 1578009600000000000
|
||||
example lbs=2014.9 1578096000000000000
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
{{% /tab-content %}}
|
||||
|
|
@ -458,7 +522,9 @@ example lbs=2014.9 1578096000000000000
|
|||
{{% tab-content %}}
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### CSV with non-default integer values
|
||||
|
||||
```
|
||||
#datatype measurement,"long:.,",dateTime:RFC3339
|
||||
m,lbs,time
|
||||
|
|
@ -467,15 +533,19 @@ example,"1,352.0",2020-01-02T00:00:00Z
|
|||
example,"1,862.0",2020-01-03T00:00:00Z
|
||||
example,"2,014.9",2020-01-04T00:00:00Z
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
example lbs=1280i 1577836800000000000
|
||||
example lbs=1352i 1577923200000000000
|
||||
example lbs=1862i 1578009600000000000
|
||||
example lbs=2014i 1578096000000000000
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
{{% /tab-content %}}
|
||||
|
|
@ -483,7 +553,9 @@ example lbs=2014i 1578096000000000000
|
|||
{{% tab-content %}}
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### CSV with non-default uinteger values
|
||||
|
||||
```
|
||||
#datatype measurement,"unsignedLong:.,",dateTime:RFC3339
|
||||
m,lbs,time
|
||||
|
|
@ -492,15 +564,19 @@ example,"1,352.0",2020-01-02T00:00:00Z
|
|||
example,"1,862.0",2020-01-03T00:00:00Z
|
||||
example,"2,014.9",2020-01-04T00:00:00Z
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
example lbs=1280u 1577836800000000000
|
||||
example lbs=1352u 1577923200000000000
|
||||
example lbs=1862u 1578009600000000000
|
||||
example lbs=2014u 1578096000000000000
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
{{% /tab-content %}}
|
||||
|
|
@ -509,6 +585,7 @@ example lbs=2014u 1578096000000000000
|
|||
---
|
||||
|
||||
### Use alternate boolean format
|
||||
|
||||
Line protocol supports only [specific boolean values](/influxdb/v2.0/reference/syntax/line-protocol/#boolean).
|
||||
If your CSV data contains boolean values that line protocol does not support,
|
||||
[define your boolean format](/influxdb/v2.0/reference/syntax/annotated-csv/extended/#boolean)
|
||||
|
|
@ -516,7 +593,9 @@ in the `boolean` datatype annotation.
|
|||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### CSV with non-default boolean values
|
||||
|
||||
```
|
||||
sep=;
|
||||
#datatype measurement,"boolean:y,Y,1:n,N,0",dateTime:RFC3339
|
||||
|
|
@ -526,21 +605,26 @@ example,n,2020-01-02T00:00:00Z
|
|||
example,1,2020-01-03T00:00:00Z
|
||||
example,N,2020-01-04T00:00:00Z
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
example verified=true 1577836800000000000
|
||||
example verified=false 1577923200000000000
|
||||
example verified=true 1578009600000000000
|
||||
example verified=false 1578096000000000000
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
||||
---
|
||||
|
||||
### Use different timestamp formats
|
||||
|
||||
The `influx write` command automatically detects **RFC3339** and **number** formatted
|
||||
timestamps when converting CSV to line protocol.
|
||||
If using a different timestamp format, [define your timestamp format](/influxdb/v2.0/reference/syntax/annotated-csv/extended/#datetime)
|
||||
|
|
@ -548,7 +632,9 @@ in the `dateTime` datatype annotation.
|
|||
|
||||
{{< flex >}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### CSV with non-default timestamps
|
||||
|
||||
```
|
||||
#datatype measurement,dateTime:2006-01-02,field
|
||||
m,time,lbs
|
||||
|
|
@ -557,14 +643,18 @@ example,2020-01-02,1352.5
|
|||
example,2020-01-03,1862.8
|
||||
example,2020-01-04,2014.9
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{% flex-content %}}
|
||||
|
||||
##### Resulting line protocol
|
||||
|
||||
```
|
||||
example lbs=1280.7 1577836800000000000
|
||||
example lbs=1352.5 1577923200000000000
|
||||
example lbs=1862.8 1578009600000000000
|
||||
example lbs=2014.9 1578096000000000000
|
||||
```
|
||||
|
||||
{{% /flex-content %}}
|
||||
{{< /flex >}}
|
||||
|
|
|
|||
Loading…
Reference in New Issue