added aws athena support to sql pkg, resolves #1078
parent
ae81534fc6
commit
b21df957bf
|
@ -5,7 +5,7 @@ list_title: Query SQL data
|
|||
description: >
|
||||
The Flux `sql` package provides functions for working with SQL data sources.
|
||||
Use `sql.from()` to query SQL databases like PostgreSQL, MySQL, Snowflake,
|
||||
SQLite, and Microsoft SQL Server.
|
||||
SQLite, Microsoft SQL Server, Amazon Athena.
|
||||
v2.0/tags: [query, flux, sql]
|
||||
menu:
|
||||
v2_0:
|
||||
|
@ -32,8 +32,9 @@ The [Flux](/v2.0/reference/flux) `sql` package provides functions for working wi
|
|||
[`sql.from()`](/v2.0/reference/flux/stdlib/sql/from/) lets you query SQL data sources
|
||||
like [PostgreSQL](https://www.postgresql.org/), [MySQL](https://www.mysql.com/),
|
||||
[Snowflake](https://www.snowflake.com/), [SQLite](https://www.sqlite.org/index.html),
|
||||
and [Microsoft SQL Server](https://www.microsoft.com/en-us/sql-server/default.aspx),
|
||||
and use the results with InfluxDB dashboards, tasks, and other operations.
|
||||
[Microsoft SQL Server](https://www.microsoft.com/en-us/sql-server/default.aspx),
|
||||
and [Amazon Athena](https://aws.amazon.com/athena/) and use the results with
|
||||
InfluxDB dashboards, tasks, and other operations.
|
||||
|
||||
- [Query a SQL data source](#query-a-sql-data-source)
|
||||
- [Join SQL data with data in InfluxDB](#join-sql-data-with-data-in-influxdb)
|
||||
|
|
|
@ -3,7 +3,7 @@ title: Flux SQL package
|
|||
list_title: SQL package
|
||||
description: >
|
||||
The Flux SQL package provides tools for working with data in SQL databases such
|
||||
as MySQL, PostgreSQL, Snowflake, SQLite, and Microsoft SQL Server.
|
||||
as MySQL, PostgreSQL, Snowflake, SQLite, Microsoft SQL Server, and Amazon Athena.
|
||||
Import the `sql` package.
|
||||
aliases:
|
||||
- /v2.0/reference/flux/functions/sql/
|
||||
|
@ -17,8 +17,15 @@ related:
|
|||
- /v2.0/query-data/flux/sql/
|
||||
---
|
||||
|
||||
SQL Flux functions provide tools for working with data in SQL databases such as
|
||||
MySQL, PostgreSQL, Snowflake, SQLite, and Microsoft SQL Server.
|
||||
SQL Flux functions provide tools for working with data in SQL databases such as:
|
||||
|
||||
- Amazon Athena
|
||||
- Microsoft SQL Server
|
||||
- MySQL
|
||||
- PostgreSQL
|
||||
- Snowflake
|
||||
- SQLite
|
||||
|
||||
Import the `sql` package:
|
||||
|
||||
```js
|
||||
|
|
|
@ -35,6 +35,7 @@ _**Data type:** String_
|
|||
|
||||
The following drivers are available:
|
||||
|
||||
- awsathena
|
||||
- mysql
|
||||
- postgres
|
||||
- snowflake
|
||||
|
@ -49,12 +50,16 @@ _**Data type:** String_
|
|||
|
||||
##### Driver dataSourceName examples
|
||||
```sh
|
||||
# Postgres Driver DSN
|
||||
postgres://pqgotest:password@localhost/pqgotest?sslmode=verify-full
|
||||
# Amazon Athena Driver DSN
|
||||
s3://myorgqueryresults/?accessID=AKIAJLO3F...®ion=us-west-1&secretAccessKey=NnQ7MUMp9PYZsmD47c%2BSsXGOFsd%2F...
|
||||
s3://myorgqueryresults/?accessID=AKIAJLO3F...&db=dbname&missingAsDefault=false&missingAsEmptyString=false®ion=us-west-1&secretAccessKey=NnQ7MUMp9PYZsmD47c%2BSsXGOFsd%2F...&WGRemoteCreation=false
|
||||
|
||||
# MySQL Driver DSN
|
||||
username:password@tcp(localhost:3306)/dbname?param=value
|
||||
|
||||
# Postgres Driver DSN
|
||||
postgres://pqgotest:password@localhost/pqgotest?sslmode=verify-full
|
||||
|
||||
# Snowflake Driver DSNs
|
||||
username[:password]@accountname/dbname/schemaname?param1=value1¶mN=valueN
|
||||
username[:password]@accountname/dbname?param1=value1¶mN=valueN
|
||||
|
@ -77,6 +82,13 @@ _**Data type:** String_
|
|||
|
||||
## Examples
|
||||
|
||||
- [MySQL](#query-a-mysql-database)
|
||||
- [Postgres](#query-a-postgres-database)
|
||||
- [Snowflake](#query-a-snowflake-database)
|
||||
- [SQLite](#query-an-sqlite-database)
|
||||
- [Amazon Athena](#query-an-amazon-athena-database)
|
||||
- [SQL Server](#query-a-sql-server-database)
|
||||
|
||||
{{% note %}}
|
||||
The examples below use [InfluxDB secrets](/v2.0/security/secrets/) to populate
|
||||
sensitive connection credentials.
|
||||
|
@ -147,6 +159,37 @@ sql.from(
|
|||
)
|
||||
```
|
||||
|
||||
### Query an Amazon Athena database
|
||||
```js
|
||||
import "sql"
|
||||
import "influxdata/influxdb/secrets"
|
||||
|
||||
region = us-west-1
|
||||
accessID = secrets.get(key: "ATHENA_ACCESS_ID")
|
||||
secretKey = secrets.get(key: "ATHENA_SECRET_KEY")
|
||||
|
||||
sql.from(
|
||||
driverName: "awsathena",
|
||||
dataSourceName: "s3://myorgqueryresults/?accessID=${accessID}®ion=${region}&secretAccessKey=${secretKey}",
|
||||
query:"SELECT * FROM example_table"
|
||||
)
|
||||
```
|
||||
|
||||
##### Athena connection string
|
||||
To query an Amazon Athena database, use the following query parameters in your Athena
|
||||
S3 connection string (DSN):
|
||||
|
||||
<span class="req">\* Required</span>
|
||||
|
||||
- **region** - AWS region <span class="req">\*</span>
|
||||
- **accessID** - AWS IAM access ID <span class="req">\*</span>
|
||||
- **secretAccessKey** - AWS IAM secret key <span class="req">\*</span>
|
||||
- **db** - database name
|
||||
- **WGRemoteCreation** - controls workgroup and tag creation
|
||||
- **missingAsDefault** - replace missing data with default values
|
||||
- **missingAsEmptyString** - replace missing data with empty strings
|
||||
|
||||
|
||||
### Query a SQL Server database
|
||||
```js
|
||||
import "sql"
|
||||
|
|
|
@ -86,6 +86,12 @@ If writing to a **SQLite** database, set `batchSize` to `999` or less.
|
|||
|
||||
## Examples
|
||||
|
||||
- [MySQL](#write-data-to-a-mysql-database)
|
||||
- [Postgres](#write-data-to-a-postgres-database)
|
||||
- [Snowflake](#write-data-to-a-snowflake-database)
|
||||
- [SQLite](#write-data-to-an-sqlite-database)
|
||||
- [SQL Server](#write-data-to-a-sql-server-database)
|
||||
|
||||
{{% note %}}
|
||||
The examples below use [InfluxDB secrets](/v2.0/security/secrets/) to populate
|
||||
sensitive connection credentials.
|
||||
|
@ -216,3 +222,8 @@ _For information about managed identities, see [Microsoft managed identities](ht
|
|||
```
|
||||
azure auth=MSI
|
||||
```
|
||||
|
||||
{{% warn %}}
|
||||
### sql.to does not support Amazon Athena
|
||||
The `sql.to` function does not support writing data to [Amazon Athena](https://aws.amazon.com/athena/).
|
||||
{{% /warn %}}
|
||||
|
|
Loading…
Reference in New Issue