Revert "Flux Google BigQuery support"
Scott Anderson
Scott Anderson
parent
4aaff13235
commit
b45b271805
|
|
@ -34,8 +34,8 @@ The [Flux](/v2.0/reference/flux) `sql` package provides functions for working wi
|
|||
like [PostgreSQL](https://www.postgresql.org/), [MySQL](https://www.mysql.com/),
|
||||
[Snowflake](https://www.snowflake.com/), [SQLite](https://www.sqlite.org/index.html),
|
||||
[Microsoft SQL Server](https://www.microsoft.com/en-us/sql-server/default.aspx),
|
||||
[Amazon Athena](https://aws.amazon.com/athena/) and [Google BigQuery](https://cloud.google.com/bigquery)
|
||||
and use the results with InfluxDB dashboards, tasks, and other operations.
|
||||
and [Amazon Athena](https://aws.amazon.com/athena/) and use the results with
|
||||
InfluxDB dashboards, tasks, and other operations.
|
||||
|
||||
- [Query a SQL data source](#query-a-sql-data-source)
|
||||
- [Join SQL data with data in InfluxDB](#join-sql-data-with-data-in-influxdb)
|
||||
|
|
@ -62,8 +62,6 @@ To query a SQL data source:
|
|||
[Snowflake](#)
|
||||
[SQLite](#)
|
||||
[SQL Server](#)
|
||||
[Athena](#)
|
||||
[BigQuery](#)
|
||||
{{% /code-tabs %}}
|
||||
|
||||
{{% code-tab-content %}}
|
||||
|
|
@ -132,34 +130,6 @@ sql.from(
|
|||
_For information about authenticating with SQL Server using ADO-style parameters,
|
||||
see [SQL Server ADO authentication](/v2.0/reference/flux/stdlib/sql/from/#sql-server-ado-authentication)._
|
||||
{{% /code-tab-content %}}
|
||||
{{% code-tab-content %}}
|
||||
```js
|
||||
import "sql"
|
||||
|
||||
sql.from(
|
||||
driverName: "awsathena",
|
||||
dataSourceName: "s3://myorgqueryresults/?accessID=12ab34cd56ef®ion=region-name&secretAccessKey=y0urSup3rs3crEtT0k3n",
|
||||
query: "GO SELECT * FROM Example.Table"
|
||||
)
|
||||
```
|
||||
|
||||
_For information about parameters to include in the Athena DSN,
|
||||
see [Athena connection string](/v2.0/reference/flux/stdlib/sql/from/#athena-connection-string)._
|
||||
{{% /code-tab-content %}}
|
||||
{{% code-tab-content %}}
|
||||
```js
|
||||
import "sql"
|
||||
|
||||
sql.from(
|
||||
driverName: "bigquery",
|
||||
dataSourceName: "bigquery://projectid/?apiKey=mySuP3r5ecR3tAP1K3y",
|
||||
query: "SELECT * FROM exampleTable"
|
||||
)
|
||||
```
|
||||
|
||||
_For information about authenticating with BigQuery, see
|
||||
[BigQuery authentication parameters](/v2.0/reference/flux/stdlib/sql/from/#bigquery-authentication-parameters)._
|
||||
{{% /code-tab-content %}}
|
||||
{{< /code-tabs-wrapper >}}
|
||||
|
||||
_See the [`sql.from()` documentation](/v2.0/reference/flux/stdlib/sql/from/) for
|
||||
|
|
|
|||
|
|
@ -3,8 +3,7 @@ title: Flux SQL package
|
|||
list_title: SQL package
|
||||
description: >
|
||||
The Flux SQL package provides tools for working with data in SQL databases such
|
||||
as MySQL, PostgreSQL, Snowflake, SQLite, Microsoft SQL Server, Amazon Athena,
|
||||
and Google BigQuery.
|
||||
as MySQL, PostgreSQL, Snowflake, SQLite, Microsoft SQL Server, and Amazon Athena.
|
||||
Import the `sql` package.
|
||||
aliases:
|
||||
- /v2.0/reference/flux/functions/sql/
|
||||
|
|
@ -21,7 +20,6 @@ related:
|
|||
SQL Flux functions provide tools for working with data in SQL databases such as:
|
||||
|
||||
- Amazon Athena
|
||||
- Google BigQuery
|
||||
- Microsoft SQL Server
|
||||
- MySQL
|
||||
- PostgreSQL
|
||||
|
|
|
|||
|
|
@ -36,7 +36,6 @@ _**Data type:** String_
|
|||
The following drivers are available:
|
||||
|
||||
- awsathena
|
||||
- bigquery
|
||||
- mysql
|
||||
- postgres
|
||||
- snowflake
|
||||
|
|
@ -74,10 +73,6 @@ sqlserver://username:password@localhost:1234?database=examplebdb
|
|||
server=localhost;user id=username;database=examplebdb;
|
||||
server=localhost;user id=username;database=examplebdb;azure auth=ENV
|
||||
server=localhost;user id=username;database=examplebdbr;azure tenant id=77e7d537;azure client id=58879ce8;azure client secret=0123456789
|
||||
|
||||
# Google BigQuery DSNs
|
||||
bigquery://projectid/?param1=value¶m2=value
|
||||
bigquery://projectid/location?param1=value¶m2=value
|
||||
```
|
||||
|
||||
### query
|
||||
|
|
@ -93,7 +88,6 @@ _**Data type:** String_
|
|||
- [SQLite](#query-an-sqlite-database)
|
||||
- [Amazon Athena](#query-an-amazon-athena-database)
|
||||
- [SQL Server](#query-a-sql-server-database)
|
||||
- [Google BigQuery](#query-a-bigquery-database)
|
||||
|
||||
{{% note %}}
|
||||
The examples below use [InfluxDB secrets](/v2.0/security/secrets/) to populate
|
||||
|
|
@ -256,42 +250,3 @@ _For information about managed identities, see [Microsoft managed identities](ht
|
|||
```
|
||||
azure auth=MSI
|
||||
```
|
||||
|
||||
### Query a BigQuery database
|
||||
```js
|
||||
import "sql"
|
||||
import "influxdata/influxdb/secrets"
|
||||
|
||||
projectID = secrets.get(key: "BIGQUERY_PROJECT_ID")
|
||||
apiKey = secrets.get(key: "BIGQUERY_APIKEY")
|
||||
|
||||
sql.from(
|
||||
driverName: "bigquery",
|
||||
dataSourceName: "bigquery://${projectID}/?apiKey=${apiKey}",
|
||||
query:"SELECT * FROM exampleTable"
|
||||
)
|
||||
```
|
||||
|
||||
#### Common BigQuery URL parameters
|
||||
- **dataset** - BigQuery dataset ID. When set, you can use unqualified table names in queries.
|
||||
|
||||
#### BigQuery authentication parameters
|
||||
The Flux BigQuery implementation uses the Google Cloud Go SDK.
|
||||
Provide your authentication credentials using one of the following methods:
|
||||
|
||||
- The `GOOGLE_APPLICATION_CREDENTIALS` environment variable that identifies the
|
||||
location of your credential JSON file.
|
||||
- Provide your BigQuery API key using the **apiKey** URL parameter in your BigQuery DSN.
|
||||
|
||||
###### Example apiKey URL parameter
|
||||
```
|
||||
bigquery://projectid/?apiKey=AIzaSyB6XK8IO5AzKZXoioQOVNTFYzbDBjY5hy4
|
||||
```
|
||||
|
||||
- Provide your base-64 encoded service account, refresh token, or JSON credentials
|
||||
using the **credentials** URL parameter in your BigQuery DSN.
|
||||
|
||||
###### Example credentials URL parameter
|
||||
```
|
||||
bigquery://projectid/?credentials=eyJ0eXBlIjoiYXV0...
|
||||
```
|
||||
|
|
|
|||
|
|
@ -34,18 +34,12 @@ _**Data type:** String_
|
|||
|
||||
The following drivers are available:
|
||||
|
||||
- bigquery
|
||||
- mysql
|
||||
- postgres
|
||||
- snowflake
|
||||
- sqlite3 – _Does not work with InfluxDB OSS or InfluxDB Cloud. More information [below](#write-data-to-an-sqlite-database)._
|
||||
- sqlserver, mssql
|
||||
|
||||
{{% warn %}}
|
||||
### sql.to does not support Amazon Athena
|
||||
The `sql.to` function does not support writing data to [Amazon Athena](https://aws.amazon.com/athena/).
|
||||
{{% /warn %}}
|
||||
|
||||
### dataSourceName
|
||||
The data source name (DSN) or connection string used to connect to the SQL database.
|
||||
The string's form and structure depend on the [driver](#drivername) used.
|
||||
|
|
@ -73,10 +67,6 @@ sqlserver://username:password@localhost:1234?database=examplebdb
|
|||
server=localhost;user id=username;database=examplebdb;
|
||||
server=localhost;user id=username;database=examplebdb;azure auth=ENV
|
||||
server=localhost;user id=username;database=examplebdbr;azure tenant id=77e7d537;azure client id=58879ce8;azure client secret=0123456789
|
||||
|
||||
# Google BigQuery DSNs
|
||||
bigquery://projectid/?param1=value¶m2=value
|
||||
bigquery://projectid/location?param1=value¶m2=value
|
||||
```
|
||||
|
||||
### table
|
||||
|
|
@ -101,7 +91,6 @@ If writing to a **SQLite** database, set `batchSize` to `999` or less.
|
|||
- [Snowflake](#write-data-to-a-snowflake-database)
|
||||
- [SQLite](#write-data-to-an-sqlite-database)
|
||||
- [SQL Server](#write-data-to-a-sql-server-database)
|
||||
- [Google BigQuery](#write-data-to-a-sql-server-database)
|
||||
|
||||
{{% note %}}
|
||||
The examples below use [InfluxDB secrets](/v2.0/security/secrets/) to populate
|
||||
|
|
@ -234,41 +223,7 @@ _For information about managed identities, see [Microsoft managed identities](ht
|
|||
azure auth=MSI
|
||||
```
|
||||
|
||||
### Write to a BigQuery database
|
||||
```js
|
||||
import "sql"
|
||||
import "influxdata/influxdb/secrets"
|
||||
|
||||
projectID = secrets.get(key: "BIGQUERY_PROJECT_ID")
|
||||
apiKey = secrets.get(key: "BIGQUERY_APIKEY")
|
||||
|
||||
sql.to(
|
||||
driverName: "bigquery",
|
||||
dataSourceName: "bigquery://${projectID}/?apiKey=${apiKey}",
|
||||
table:"exampleTable"
|
||||
)
|
||||
```
|
||||
|
||||
#### Common BigQuery URL parameters
|
||||
- **dataset** - BigQuery dataset ID. When set, you can use unqualified table names in queries.
|
||||
|
||||
#### BigQuery authentication parameters
|
||||
The Flux BigQuery implementation uses the Google Cloud Go SDK.
|
||||
Provide your authentication credentials using one of the following methods:
|
||||
|
||||
- The `GOOGLE_APPLICATION_CREDENTIALS` environment variable that identifies the
|
||||
location of your credential JSON file.
|
||||
- Provide your BigQuery API key using the **apiKey** URL parameter in your BigQuery DSN.
|
||||
|
||||
###### Example apiKey URL parameter
|
||||
```
|
||||
bigquery://projectid/?apiKey=AIzaSyB6XK8IO5AzKZXoioQOVNTFYzbDBjY5hy4
|
||||
```
|
||||
|
||||
- Provide your base-64 encoded service account, refresh token, or JSON credentials
|
||||
using the **credentials** URL parameter in your BigQuery DSN.
|
||||
|
||||
###### Example credentials URL parameter
|
||||
```
|
||||
bigquery://projectid/?credentials=eyJ0eXBlIjoiYXV0...
|
||||
```
|
||||
{{% warn %}}
|
||||
### sql.to does not support Amazon Athena
|
||||
The `sql.to` function does not support writing data to [Amazon Athena](https://aws.amazon.com/athena/).
|
||||
{{% /warn %}}
|
||||
|
|
|
|||
|
|
@ -23,8 +23,7 @@ InfluxDB until the next InfluxDB v2.0 release._
|
|||
|
||||
### Features
|
||||
- Add functions to convert semantic monotype to AST type.
|
||||
- Add [Google BigQuery](https://cloud.google.com/bigquery) support to
|
||||
[SQL package](/v2.0/reference/flux/stdlib/sql/).
|
||||
- Add BigQuery support.
|
||||
- Rust flatbuffer serialization for `MonoType` and `TypeExpression`.
|
||||
- Extend with Geo package with GIS functions and
|
||||
[unit support](/v2.0/reference/flux/stdlib/experimental/geo/#distance-units).
|
||||
|
|
|
|||
Loading…
Reference in New Issue