diff --git a/content/influxdb/cloud-dedicated/get-started/query.md b/content/influxdb/cloud-dedicated/get-started/query.md index f774a2a0e..eb08cef2c 100644 --- a/content/influxdb/cloud-dedicated/get-started/query.md +++ b/content/influxdb/cloud-dedicated/get-started/query.md @@ -46,7 +46,7 @@ The examples in this section of the tutorial query the {{< req type="key" text="Covered in this tutorial" color="magenta" >}} - [`influx3` data CLI](?t=influx3+CLI#execute-an-sql-query){{< req "\* " >}} - [InfluxDB v3 client libraries](/influxdb/cloud-dedicated/reference/client-libraries/v3/) -- [Flight SQL clients](?t=Go#execute-an-sql-query){{< req "\* " >}} +- [Flight clients](?t=Go#execute-an-sql-query){{< req "\* " >}} - [Superset](/influxdb/cloud-dedicated/query-data/sql/execute-queries/superset/) - [Grafana](/influxdb/cloud-dedicated/query-data/sql/execute-queries/grafana/) - [InfluxQL with InfluxDB v1 HTTP API](/influxdb/cloud-dedicated/primers/api/v1/#query-using-the-v1-api) diff --git a/content/influxdb/cloud-dedicated/process-data/downsample.md b/content/influxdb/cloud-dedicated/process-data/downsample.md index 5b15dec0b..5f76953b6 100644 --- a/content/influxdb/cloud-dedicated/process-data/downsample.md +++ b/content/influxdb/cloud-dedicated/process-data/downsample.md @@ -147,9 +147,11 @@ SELECT AVG(hum) AS hum, AVG(co) AS co FROM home +--In WHERE, time refers to .time WHERE time >= now() - INTERVAL '24 hours' +--1 refers to the DATE_BIN column GROUP BY 1, room -ORDER BY room, time +ORDER BY time ``` {{% /tab-content %}} @@ -208,9 +210,11 @@ SELECT AVG(hum) AS hum, AVG(co) AS co FROM home +--In WHERE, time refers to .time WHERE time >= now() - INTERVAL '24 hours' +--1 refers to the DATE_BIN column GROUP BY 1, room -ORDER BY time +ORDER BY 1 ''' table = influxdb_raw.query(query=query, language="sql") @@ -314,9 +318,11 @@ SELECT AVG(hum) AS hum, AVG(co) AS co FROM home +--In WHERE, time refers to .time WHERE time >= now() - INTERVAL '24 hours' +--1 refers to the DATE_BIN column GROUP BY 1, room -ORDER BY time +ORDER BY 1 ''' table = influxdb_raw.query(query=query, language="sql") diff --git a/content/influxdb/cloud-dedicated/query-data/influxql/execute-queries/python.md b/content/influxdb/cloud-dedicated/query-data/influxql/execute-queries/python.md index 5dd6f2e5b..dcf605ee9 100644 --- a/content/influxdb/cloud-dedicated/query-data/influxql/execute-queries/python.md +++ b/content/influxdb/cloud-dedicated/query-data/influxql/execute-queries/python.md @@ -37,7 +37,7 @@ list_code_example: | --- Use the `influxdb3-python` client library to query data stored in InfluxDB with InfluxQL. -The `influxdb3-client` uses Flight SQL to query data from InfluxDB and return +The `influxdb3-client` uses InfluxDB v3's Flight RPC protocol to query data from InfluxDB and return results in Apache Arrow format. - [Get started using Python to query InfluxDB](#get-started-using-python-to-query-influxdb) diff --git a/content/influxdb/cloud-dedicated/query-data/sql/aggregate-select.md b/content/influxdb/cloud-dedicated/query-data/sql/aggregate-select.md index 4c51b0b16..9ce83c063 100644 --- a/content/influxdb/cloud-dedicated/query-data/sql/aggregate-select.md +++ b/content/influxdb/cloud-dedicated/query-data/sql/aggregate-select.md @@ -33,9 +33,7 @@ list_code_example: | sum(field2), tag1 FROM home - GROUP BY - DATE_BIN(INTERVAL '1 hour', time, '2022-01-01T00:00:00Z'::TIMESTAMP), - tag1 + GROUP BY 1, tag1 ``` --- @@ -207,8 +205,9 @@ groups: nearest to `home.time`: ```sql - SELECT DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) AS time - from home + SELECT + DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) AS time + FROM home ... ``` @@ -216,20 +215,32 @@ groups: {{% influxdb/custom-timestamps-span %}}`2023-03-09T13:00:50.000Z`{{% /influxdb/custom-timestamps-span %}}, the output `time` column contains {{% influxdb/custom-timestamps-span %}}`2023-03-09T12:00:00.000Z`{{% /influxdb/custom-timestamps-span %}}. + - Use [aggregate](/influxdb/cloud-dedicated/reference/sql/functions/aggregate/) or [selector](/influxdb/cloud-dedicated/reference/sql/functions/selector/) functions on specified columns. - In your `GROUP BY` clause: - - Use the [`DATE_BIN` function](/influxdb/cloud-dedicated/reference/sql/functions/time-and-date/#date_bin) with the same parameters used in the `SELECT` clause. + - Specify the `DATE_BIN(...)` column ordinal reference (`1`). - Specify other columns (for example, `room`) that are specified in the `SELECT` clause and aren't used in a selector function. ```sql - SELECT DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) AS time + SELECT + DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) AS time ... - GROUP BY DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP), room + GROUP BY 1, room ... ``` + To reference the `DATE_BIN(...)` result column by _name_ in the `GROUP BY` clause, assign an alias other than "time" in the `SELECT` clause--for example: + + ```sql + SELECT + DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) AS _time + FROM home + ... + GROUP BY _time, room + ``` + - Include an `ORDER BY` clause with columns to sort by. The following example retrieves unique combinations of time intervals and rooms with their minimum, maximum, and average temperatures. @@ -242,8 +253,8 @@ SELECT selector_min(temp, time)['value'] AS 'min temp', avg(temp) AS 'average temp' FROM home -GROUP BY DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP), room -ORDER BY room, time +GROUP BY 1, room +ORDER BY room, 1 ``` {{< expand-wrapper >}} @@ -269,6 +280,25 @@ ORDER BY room, time {{% /expand %}} {{< /expand-wrapper >}} +{{% note %}} +#### GROUP BY time + +In the `GROUP BY` clause, the name "time" always refers to the `time` column in the source table. +If you want to reference a calculated time column by name, use an alias different from "time"--for example: + +```sql +SELECT + DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) + AS _time, + room, + selector_max(temp, time)['value'] AS 'max temp', + selector_min(temp, time)['value'] AS 'min temp', + avg(temp) AS 'average temp' +FROM home +GROUP BY _time, room +ORDER BY room, _time +``` +{{% /note %}} ### Query rows based on aggregate values diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/flight-sql/_index.md b/content/influxdb/cloud-dedicated/reference/client-libraries/flight-sql/_index.md deleted file mode 100644 index 88b72efc3..000000000 --- a/content/influxdb/cloud-dedicated/reference/client-libraries/flight-sql/_index.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Flight SQL clients -description: > - Flight SQL clients are language-specific drivers that can interact with SQL databases using the Arrow in-memory format and the Flight RPC framework. - View the list of available clients. -weight: 30 -menu: - influxdb_cloud_dedicated: - name: Flight SQL clients - parent: Client libraries -influxdb/cloud-dedicated/tags: [Golang, Python, client libraries, Flight SQL] -aliases: - - /influxdb/cloud-dedicated/reference/client-libraries/flight-sql/go-flightsql/ - - /influxdb/cloud-dedicated/reference/client-libraries/flight-sql/python-flightsql-dbapi/ ---- - -Flight and Flight SQL clients are language-specific drivers that interact with SQL databases using the Arrow in-memory format and the Flight RPC framework. -Clients provide a [`FlightClient`](https://arrow.apache.org/docs/python/generated/pyarrow.flight.FlightClient.html#pyarrow.flight.FlightClient) that can query and retrieve data from InfluxDB v3 using gRPC and Flight. - -Flight clients are maintained by Apache Arrow projects or third-parties. -For specifics about a Flight client, see the client's GitHub repository. - -{{% note %}} -[InfluxDB v3 client libraries](/influxdb/cloud-dedicated/reference/client-libraries/v3/) use Flight clients -and offer additional convenience for writing, querying, and retrieving data stored in {{% cloud-name %}}. -{{% /note %}} - -## C# (csharp) - -- [Apache Arrow C# FlightClient](https://github.com/apache/arrow/tree/main/csharp/examples/FlightClientExample) - -## Go -- [Apache Arrow Go Flight SQL client](https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight/flightsql#Client) - -## Java -- [Apache Arrow Java FlightClient](https://arrow.apache.org/docs/java/reference/org/apache/arrow/flight/FlightClient.html) - -## Python -- [Apache Arrow PyArrow FlightClient](https://arrow.apache.org/docs/python/generated/pyarrow.flight.FlightClient.html#pyarrow.flight.FlightClient) -- [InfluxData flightsql-dbapi](https://github.com/influxdata/flightsql-dbapi) - - - - diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/flight-sql/go-flightsql.md b/content/influxdb/cloud-dedicated/reference/client-libraries/flight-sql/go-flightsql.md deleted file mode 100644 index 1be00fbf8..000000000 --- a/content/influxdb/cloud-dedicated/reference/client-libraries/flight-sql/go-flightsql.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -title: Go Flight SQL client -description: The Go Flight SQL client integrates with golang scripts and applications to query data stored in an InfluxDB Cloud Dedicated database. -external_url: https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight/flightsql -menu: - influxdb_cloud_dedicated: - name: Go Flight SQL client - parent: Flight SQL clients - params: - url: https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight/flightsql - identifier: go-flightsql -influxdb/cloud-dedicated/tags: [Golang, gRPC, SQL, Flight SQL, client libraries] -weight: 201 ---- - -The [Go Flight SQL client](https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight/flightsql) integrates with golang scripts and applications to query data stored in an InfluxDB Cloud Dedicated database. -See the [example](/influxdb/cloud-dedicated/get-started/query/?t=Go#execute-an-sql-query). - -For more information, see the [Go client README on GitHub](https://github.com/influxdata/influxdb-client-go). diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/flight-sql/python-flightsql-dbapi.md b/content/influxdb/cloud-dedicated/reference/client-libraries/flight-sql/python-flightsql-dbapi.md deleted file mode 100644 index f9ee019dd..000000000 --- a/content/influxdb/cloud-dedicated/reference/client-libraries/flight-sql/python-flightsql-dbapi.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -title: Python Flight SQL DBAPI client -description: The Python `flightsql-dbapi` library integrates with Python scripts and applications to query data stored in an InfluxDB Cloud Dedicated database. -external_url: https://github.com/influxdata/flightsql-dbapi -menu: - influxdb_cloud_dedicated: - name: Python Flight SQL client - parent: Flight SQL clients - params: - url: https://github.com/influxdata/flightsql-dbapi -influxdb/cloud-dedicated/tags: [Python, gRPC, SQL, Flight SQL, client libraries] -weight: 201 ---- - -The [Python Flight SQL DBAPI library](https://github.com/influxdata/flightsql-dbapi) integrates with Python scripts and applications to query data stored in an InfluxDB Cloud Dedicated database. -The documentation for this client library is available on GitHub. - -Python Flight SQL DBAPI library \ No newline at end of file diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/flight/_index.md b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/_index.md new file mode 100644 index 000000000..fb8612723 --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/_index.md @@ -0,0 +1,38 @@ +--- +title: Apache Arrow Flight RPC clients +description: > + Flight clients are language-specific drivers that can interact with Flight servers using the Arrow in-memory format and the Flight RPC framework. + View the list of available clients. +weight: 30 +menu: + influxdb_cloud_dedicated: + name: Arrow Flight clients + parent: Client libraries +influxdb/cloud-dedicated/tags: [client libraries, Flight RPC, Flight SQL] +aliases: + - /influxdb/cloud-dedicated/reference/client-libraries/flight-sql/ + - /influxdb/cloud-dedicated/reference/client-libraries/flight-sql/go-flightsql/ + - /influxdb/cloud-dedicated/reference/client-libraries/flight-sql/python-flightsql-dbapi/ +--- + +Flight RPC and Flight SQL clients are language-specific drivers that interact with databases using the Arrow in-memory format and the Flight RPC protocol. +Apache Arrow Flight RPC and Flight SQL protocols define APIs for servers and clients. + +{{% note %}} +#### InfluxDB v3 client libraries + +We recommend using [InfluxDB v3 client libraries](/influxdb/cloud-dedicated/reference/client-libraries/v3/) for integrating InfluxDB v3 with your application code. +Client libraries wrap Apache Arrow Flight clients +and provide convenient methods for writing, querying, and processing data stored in {{% cloud-name %}}. +{{% /note %}} + +**Flight RPC clients** can use SQL or InfluxQL to query data stored in an {{% cloud-name %}} database. +Using InfluxDB v3's IOx-specific Flight RPC protocol, clients send a single `DoGet()` request to authenticate, query, and retrieve data. + +**Flight SQL clients** use the [Flight SQL protocol](https://arrow.apache.org/docs/format/FlightSql.html) for querying an SQL database server. +They can use SQL to query data stored in an {{% cloud-name %}} database, but they can't use InfuxQL. + +Clients are maintained by Apache Arrow projects or third-parties. +For specifics about a Flight client, see the client's GitHub repository. + +{{< children >}} diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/flight/csharp-flight.md b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/csharp-flight.md new file mode 100644 index 000000000..572f35b08 --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/csharp-flight.md @@ -0,0 +1,20 @@ +--- +title: C# .NET Flight client +description: The C# .NET Flight client integrates with C# .NET scripts and applications to query data stored in InfluxDB. +external_url: https://github.com/apache/arrow/tree/main/csharp/examples/FlightClientExample +menu: + influxdb_cloud_dedicated: + name: C# .NET + parent: Arrow Flight clients + params: + url: https://github.com/apache/arrow/tree/main/csharp/examples/FlightClientExample + identifier: csharp-flight-client +influxdb/cloud-dedicated/tags: [C#, gRPC, SQL, Flight SQL, client libraries] +aliases: + - /influxdb/cloud-dedicated/reference/client-libraries/flight-sql/csharp-flightsql/ +weight: 201 +--- + +[Apache Arrow for C# .NET](https://github.com/apache/arrow/blob/main/csharp/README.md) integrates with C# .NET scripts and applications to query data stored in InfluxDB. + +For more information, see the [C# client example on GitHub](https://github.com/apache/arrow/tree/main/csharp/examples/FlightClientExample). diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/flight/go-flight.md b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/go-flight.md new file mode 100644 index 000000000..f20e0ac98 --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/go-flight.md @@ -0,0 +1,20 @@ +--- +title: Go Flight client +description: The Go Flight client integrates with Go scripts and applications to query data stored in InfluxDB. +menu: + influxdb_cloud_dedicated: + name: Go + parent: Arrow Flight clients + params: + url: https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight#Client + identifier: go-flight-client +influxdb/cloud-dedicated/tags: [Go, gRPC, SQL, Flight SQL, client libraries] +aliases: + - /influxdb/cloud-dedicated/reference/client-libraries/flight-sql/go-flightsql/ +weight: 201 +--- + +[Apache Arrow for Go](https://pkg.go.dev/github.com/apache/arrow/go/v12) integrates with golang scripts and applications to query data stored in InfluxDB. +See the [Flight SQL example](/influxdb/cloud-dedicated/get-started/query/?t=Go#execute-an-sql-query). + +For more information, see the [Go Arrow Flight Client documentation](https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight#Client). diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/flight/java-flightsql.md b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/java-flightsql.md new file mode 100644 index 000000000..ed348c9b5 --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/java-flightsql.md @@ -0,0 +1,20 @@ +--- +title: Java Flight SQL package +description: The Java Flight SQL client integrates with Java applications to query and retrieve data from Flight database servers using RPC and SQL. +external_url: https://arrow.apache.org/docs/java/reference/org/apache/arrow/flight/sql/package-summary.html +menu: + influxdb_cloud_dedicated: + name: Java Flight SQL + parent: Arrow Flight clients + identifier: java-flightsql-client + params: + url: https://arrow.apache.org/docs/java/reference/org/apache/arrow/flight/sql/package-summary.html +influxdb/cloud-dedicated/tags: [Java, gRPC, SQL, Flight SQL, client libraries] +weight: 201 +aliases: + - /influxdb/cloud-dedicated/reference/client-libraries/flight-sql/java-flightsql/ +--- + +[Apache Arrow Flight SQL for Java](https://arrow.apache.org/docs/java/reference/org/apache/arrow/flight/sql/package-summary.html) integrates with Java applications to query and retrieve data from Flight database servers using RPC and SQL. + +Java Flight SQL package diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/flight/python-flight.md b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/python-flight.md new file mode 100644 index 000000000..d8fe22f73 --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/python-flight.md @@ -0,0 +1,146 @@ +--- +title: Python Flight client +description: The Python Flight client integrates with Python scripts and applications to query data stored in InfluxDB. +menu: + influxdb_cloud_dedicated: + name: Python + parent: Arrow Flight clients + identifier: python-flight-client +influxdb/cloud-dedicated/tags: [Python, gRPC, SQL, Flight SQL, client libraries] +aliases: + - /influxdb/cloud-dedicated/reference/client-libraries/flight-sql/python-flightsql/ +weight: 201 +list_code_example: | + ```py + from pyarrow.flight import FlightClient, Ticket, FlightCallOptions + import json + import pandas + import tabulate + + # Downsampling query groups data into 2-hour bins + sql=""" + SELECT DATE_BIN(INTERVAL '2 hours', + time, + '1970-01-01T00:00:00Z'::TIMESTAMP) AS time, + room, + selector_max(temp, time)['value'] AS 'max temp', + selector_min(temp, time)['value'] AS 'min temp', + avg(temp) AS 'average temp' + FROM home + GROUP BY + 1, + room + ORDER BY room, 1""" + + flight_ticket = Ticket(json.dumps({ + "namespace_name": "DATABASE_NAME", + "sql_query": sql, + "query_type": "sql" + })) + + token = (b"authorization", bytes(f"Bearer DATABASE_TOKEN".encode('utf-8'))) + options = FlightCallOptions(headers=[token]) + client = FlightClient(f"grpc+tls://cluster-id.influxdb.io:443") + + reader = client.do_get(flight_ticket, options) + arrow_table = reader.read_all() + ``` +--- + +[Apache Arrow Python bindings](https://arrow.apache.org/docs/python/index.html) integrate with Python scripts and applications to query data stored in InfluxDB. + +The following examples show how to use the `pyarrow.flight` and `pandas` Python modules to query and format data stored in an {{% cloud-name %}} database: + +{{% code-tabs-wrapper %}} +{{% code-tabs %}} +[SQL](#sql-python) +[InfluxQL](#influxql-python) +{{% /code-tabs %}} + +{{% code-tab-content %}} + +{{% code-placeholders "DATABASE_NAME|DATABASE_TOKEN" %}} +```py +# Using pyarrow>=12.0.0 FlightClient +from pyarrow.flight import FlightClient, Ticket, FlightCallOptions +import json +import pandas +import tabulate + +# Downsampling query groups data into 2-hour bins +sql=""" + SELECT DATE_BIN(INTERVAL '2 hours', time) AS time, + room, + selector_max(temp, time)['value'] AS 'max temp', + selector_min(temp, time)['value'] AS 'min temp', + avg(temp) AS 'average temp' + FROM home + GROUP BY + 1, + room + ORDER BY room, 1""" + +flight_ticket = Ticket(json.dumps({ + "namespace_name": "DATABASE_NAME", + "sql_query": sql, + "query_type": "sql" +})) + +token = (b"authorization", bytes(f"Bearer DATABASE_TOKEN".encode('utf-8'))) +options = FlightCallOptions(headers=[token]) +client = FlightClient(f"grpc+tls://cluster-id.influxdb.io:443") + +reader = client.do_get(flight_ticket, options) +arrow_table = reader.read_all() +# Use pyarrow and pandas to view and analyze data +data_frame = arrow_table.to_pandas() +print(data_frame.to_markdown()) +``` +{{% /code-placeholders %}} + +{{% /code-tab-content %}} +{{% code-tab-content %}} + +{{% code-placeholders "DATABASE_NAME|DATABASE_TOKEN" %}} +```py +# Using pyarrow>=12.0.0 FlightClient +from pyarrow.flight import FlightClient, Ticket, FlightCallOptions +import json +import pandas +import tabulate + +# Downsampling query groups data into 2-hour bins +influxql=""" + SELECT FIRST(temp) + FROM home + WHERE room = 'kitchen' + AND time >= now() - 100d + AND time <= now() - 10d + GROUP BY time(2h)""" + +flight_ticket = Ticket(json.dumps({ + "namespace_name": "DATABASE_NAME", + "sql_query": influxql, + "query_type": "influxql" +})) + +token = (b"authorization", bytes(f"Bearer DATABASE_TOKEN".encode('utf-8'))) +options = FlightCallOptions(headers=[token]) +client = FlightClient(f"grpc+tls://cluster-id.influxdb.io:443") + +reader = client.do_get(flight_ticket, options) +arrow_table = reader.read_all() +# Use pyarrow and pandas to view and analyze data +data_frame = arrow_table.to_pandas() +print(data_frame.to_markdown()) +``` +{{% /code-placeholders %}} + +{{% /code-tab-content %}} + +Replace the following: + +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: your {{% cloud-name %}} database +- {{% code-placeholder-key %}}`DATABASE_TOKEN`{{% /code-placeholder-key %}}: a [database token](/influxdb/cloud-dedicated/admin/tokens/) with sufficient permissions to the database + +{{% /code-tabs-wrapper %}} diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/flight/python-flightsql-dbapi.md b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/python-flightsql-dbapi.md new file mode 100644 index 000000000..518b9a3f0 --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/flight/python-flightsql-dbapi.md @@ -0,0 +1,170 @@ +--- +title: Python Flight SQL DBAPI client +description: The Python `flightsql-dbapi` library uses SQL and the Flight SQL protocol to query data stored in an InfluxDB Cloud Dedicated database. +menu: + influxdb_cloud_dedicated: + name: Python Flight SQL + parent: Arrow Flight clients + identifier: python-flightsql-client +influxdb/cloud-dedicated/tags: [Python, SQL, Flight SQL] +weight: 201 +aliases: + - /influxdb/cloud-dedicated/reference/client-libraries/flight-sql/python-flightsql/ +--- + +The [Python `flightsql-dbapi` Flight SQL DBAPI library](https://github.com/influxdata/flightsql-dbapi) integrates with Python applications using SQL to query data stored in an {{% cloud-name %}} database. The `flightsql-dbapi` library uses the [Flight SQL protocol](https://arrow.apache.org/docs/format/FlightSql.html) to query and retrieve data. + +{{% note %}} +#### InfluxDB v3 client libraries + +We recommend using the [`influxdb3-python` Python client library](/influxdb/cloud-dedicated/reference/client-libraries/v3/python/) for integrating InfluxDB v3 with your Python application code. + +[InfluxDB v3 client libraries](/influxdb/cloud-dedicated/reference/client-libraries/v3/) client libraries wrap Apache Arrow Flight clients +and provide convenient methods for writing, querying, and processing data stored in {{% cloud-name %}}. +Client libraries can query using [SQL](/influxdb/cloud-dedicated/query-data/sql/execute-queries/python/) or [InfluxQL](/influxdb/cloud-dedicated/query-data/influxql/execute-queries/python/). +{{% /note %}} + +## Installation + +The [`flightsql-dbapi`](https://github.com/influxdata/flightsql-dbapi) Flight SQL library for Python provides a +[DB API 2](https://peps.python.org/pep-0249/) interface and +[SQLAlchemy](https://www.sqlalchemy.org/) dialect for +[Flight SQL](https://arrow.apache.org/docs/format/FlightSql.html). +Installing `flightsql-dbapi` also installs the [`pyarrow`](https://arrow.apache.org/docs/python/index.html) library that you'll use for working with Arrow data. + +In your terminal, use `pip` to install `flightsql-dbapi`: + +```sh +pip install flightsql-dbapi +``` + +## Importing the module + +The `flightsql-dbapi` package provides the `flightsql` module. From the module, import the `FlightSQLClient` class method: + +```py +from flightsql import FlightSQLClient +``` + +- `flightsql.FlightSQLClient` class: an interface for [initializing +a client](#initialization) and interacting with a Flight SQL server. + +## API reference + +- [Class FlightSQLClient](#class-flightsqlclient) + - [Syntax](#syntax) +- [Initialize a client](#initialize-a-client) + - [Instance methods](#instance-methods) + - [FlightSQLClient.execute](#flightsqlclientexecute) + - [Syntax {#execute-query-syntax}](#syntax-execute-query-syntax) + - [Example {#execute-query-example}](#example-execute-query-example) + - [FlightSQLClient.do_get](#flightsqlclientdo_get) + - [Syntax {#retrieve-data-syntax}](#syntax-retrieve-data-syntax) + - [Example {#retrieve-data-example}](#example-retrieve-data-example) + +## Class FlightSQLClient + +Provides an interface for [initializing +a client](#initialize-a-client) and interacting with a Flight SQL server. + +### Syntax + +```py +__init__(self, host=None, token=None, metadata=None, features=None) +``` + +Initializes and returns a `FlightSQLClient` instance for interating with the server. + +## Initialize a client + +The following example shows how to use Python with `flightsql-dbapi` +and the _DB API 2_ interface to instantiate a Flight SQL client configured for an InfluxDB database. + +{{% code-placeholders "DATABASE_NAME|DATABASE_TOKEN" %}} +```py +from flightsql import FlightSQLClient + +# Instantiate a FlightSQLClient configured for a database +client = FlightSQLClient(host='cluster-id.influxdb.io', + token='DATABASE_TOKEN', + metadata={'database': 'DATABASE_NAME'}, + features={'metadata-reflection': 'true'}) +``` +{{% /code-placeholders %}} + +Replace the following: + +- {{% code-placeholder-key %}}`DATABASE_TOKEN`{{% /code-placeholder-key %}}: an {{% cloud-name %}} [database token](/influxdb/cloud-dedicated/admin/tokens/) with read permissions on the databases you want to query +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of your {{% cloud-name %}} [database](/influxdb/cloud-dedicated/admin/databases/) + +### Instance methods + +### FlightSQLClient.execute + +Sends a Flight SQL RPC request to execute the specified SQL Query. + +#### Syntax {#execute-query-syntax} + +```py +execute(query: str, call_options: Optional[FlightSQLCallOptions] = None) +``` + +#### Example {#execute-query-example} + +```py +# Execute the query +info = client.execute("SELECT * FROM home") +``` + +The response contains a `flight.FlightInfo` object that contains metadata and an `endpoints: [...]` list. Each endpoint contains the following: + +- A list of addresses where you can retrieve query result data. +- A `ticket` value that identifies the data to [retrieve](#retrieve-data-example). + +### FlightSQLClient.do_get + +Passes a Flight ticket (obtained from a `FlightSQLClient.execute` response) and retrieves Arrow data identified by the ticket. +Returns a `pyarrow.flight.FlightStreamReader` for streaming the data. + +#### Syntax {#retrieve-data-syntax} + +```py + do_get(ticket, call_options: Optional[FlightSQLCallOptions] = None) +``` + +#### Example {#retrieve-data-example} + +The following sample shows how to use Python with `flightsql-dbapi` and `pyarrow` to query InfluxDB and retrieve data. + +```py +from flightsql import FlightSQLClient + +# Instantiate a FlightSQLClient configured for a database +client = FlightSQLClient(host='cluster-id.influxdb.io', + token='DATABASE_TOKEN', + metadata={'database': 'DATABASE_NAME'}, + features={'metadata-reflection': 'true'}) + +# Execute the query to retrieve FlightInfo +info = client.execute("SELECT * FROM home") + +# Extract the token for retrieving data +ticket = info.endpoints[0].ticket + +# Use the ticket to request the Arrow data stream. +# Return a FlightStreamReader for streaming the results. +reader = client.do_get(ticket) + +# Read all data to a pyarrow.Table +table = reader.read_all() + +print(table) +``` + +`do_get(ticket)` returns a [`pyarrow.flight.FlightStreamReader`](https://arrow.apache.org/docs/python/generated/pyarrow.flight.FlightStreamReader.html) for streaming Arrow [record batches](https://arrow.apache.org/docs/python/data.html#record-batches). + +To read data from the stream, call one of the following `FlightStreamReader` methods: + +- `read_all()`: Read all record batches as a [`pyarrow.Table`](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html). +- `read_chunk()`: Read the next RecordBatch and metadata. +- `read_pandas()`: Read all record batches and convert them to a [`pandas.DataFrame`](https://pandas.pydata.org/docs/reference/frame.html). diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/v3/_index.md b/content/influxdb/cloud-dedicated/reference/client-libraries/v3/_index.md index 666f3bef6..fda64275f 100644 --- a/content/influxdb/cloud-dedicated/reference/client-libraries/v3/_index.md +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/v3/_index.md @@ -1,7 +1,7 @@ --- title: InfluxDB v3 API client libraries description: > - InfluxDB v3 client libraries use InfluxDB HTTP APIs to write data and use [Flight SQL clients](/influxdb/cloud-dedicated/reference/client-libraries/flight-sql) to query data stored in an InfluxDB Cloud Dedicated database. + InfluxDB v3 client libraries use InfluxDB HTTP APIs to write data and use [Flight clients](/influxdb/cloud-dedicated/reference/client-libraries/flight-sql) to execute SQL and InfluxQL queries. View the list of available client libraries. weight: 30 menu: @@ -20,7 +20,7 @@ InfluxDB client libraries provide configurable batch writing of data to InfluxDB They can be used to construct line protocol data and transform data from other formats to line protocol. -InfluxDB v3 client libraries can query InfluxDB v3 using the Arrow Flight protocol with gRPC. +InfluxDB v3 client libraries can query InfluxDB v3 using InfluxDB v3's IOx-specific Arrow Flight protocol with gRPC. Client libraries use Flight clients to execute SQL and InfluxQL queries, request database information, and retrieve data stored in {{% cloud-name %}}. diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/v3/csharp.md b/content/influxdb/cloud-dedicated/reference/client-libraries/v3/csharp.md new file mode 100644 index 000000000..e260e2ae0 --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/v3/csharp.md @@ -0,0 +1,21 @@ +--- +title: C# .NET client library for InfluxDB v3 +list_title: C# .NET +description: > + The InfluxDB v3 `influxdb3-csharp` C# .NET client library integrates with C# .NET scripts and applications to write and query data stored in an InfluxDB Cloud Dedicated database. +external_url: https://github.com/InfluxCommunity/influxdb3-csharp +menu: + influxdb_cloud_dedicated: + name: C# .NET + parent: v3 client libraries + identifier: influxdb3-csharp +influxdb/cloud-dedicated/tags: [C#, gRPC, SQL, Flight SQL, client libraries] +weight: 201 +--- + +The InfluxDB v3 [`influxdb3-csharp` C# .NET client library](https://github.com/InfluxCommunity/influxdb3-csharp) integrates with C# .NET scripts and applications +to write and query data stored in an {{% cloud-name %}} database. + +The documentation for this client library is available on GitHub. + +InfluxDB v3 C# .NET client library diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/v3/java.md b/content/influxdb/cloud-dedicated/reference/client-libraries/v3/java.md new file mode 100644 index 000000000..a5afdb21e --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/v3/java.md @@ -0,0 +1,21 @@ +--- +title: Java client library for InfluxDB v3 +list_title: Java +description: > + The InfluxDB v3 `influxdb3-java` Java client library integrates with Java scripts and applications to write and query data stored in an InfluxDB Cloud Dedicated database. +external_url: https://github.com/InfluxCommunity/influxdb3-java +menu: + influxdb_cloud_dedicated: + name: Java + parent: v3 client libraries + identifier: influxdb3-java +influxdb/cloud-dedicated/tags: [Java, gRPC, SQL, Flight SQL, client libraries] +weight: 201 +--- + +The InfluxDB v3 [`influxdb3-java` Java client library](https://github.com/InfluxCommunity/influxdb3-java) integrates with Java scripts and applications +to write and query data stored in an {{% cloud-name %}} database. + +The documentation for this client library is available on GitHub. + +InfluxDB v3 Java client library diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/v3/javascript.md b/content/influxdb/cloud-dedicated/reference/client-libraries/v3/javascript.md new file mode 100644 index 000000000..4aab5080d --- /dev/null +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/v3/javascript.md @@ -0,0 +1,24 @@ +--- +title: JavaScript client library for InfluxDB v3 +list_title: JavaScript +description: > + The InfluxDB v3 `influxdb3-js` JavaScript client library integrates with JavaScript scripts and applications to write and query data stored in an InfluxDB Cloud Dedicated database. +external_url: https://github.com/InfluxCommunity/influxdb3-js +menu: + influxdb_cloud_dedicated: + name: JavaScript + parent: v3 client libraries + identifier: influxdb3-js +influxdb/cloud-dedicated/tags: [JavaScript, gRPC, SQL, Flight SQL, client libraries] +weight: 201 +aliases: + - /influxdb/cloud-dedicated/reference/api/client-libraries/go/ + - /influxdb/cloud-dedicated/tools/client-libraries/go/ +--- + +The InfluxDB v3 [`influxdb3-js` JavaScript client library](https://github.com/InfluxCommunity/influxdb3-js) integrates with JavaScript scripts and applications +to write and query data stored in an {{% cloud-name %}} database. + +The documentation for this client library is available on GitHub. + +InfluxDB v3 JavaScript client library diff --git a/content/influxdb/cloud-dedicated/reference/client-libraries/v3/python.md b/content/influxdb/cloud-dedicated/reference/client-libraries/v3/python.md index 1807b1cec..d87ee52f3 100644 --- a/content/influxdb/cloud-dedicated/reference/client-libraries/v3/python.md +++ b/content/influxdb/cloud-dedicated/reference/client-libraries/v3/python.md @@ -163,10 +163,8 @@ client = InfluxDBClient3(token="DATABASE_TOKEN", Replace the following: -- {{% code-placeholder-key %}}`DATABASE_TOKEN`{{% /code-placeholder-key %}}: - Your InfluxDB token with READ permissions on the databases you want to query. -- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: - The name of your InfluxDB database. +- {{% code-placeholder-key %}}`DATABASE_TOKEN`{{% /code-placeholder-key %}}: an {{% cloud-name %}} [database token](/influxdb/cloud-dedicated/admin/tokens/) with read permissions on the databases you want to query +- {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: the name of your {{% cloud-name %}} [database](/influxdb/cloud-dedicated/admin/databases/) ##### Initialize a client for batch writing diff --git a/content/influxdb/cloud-dedicated/reference/internals/arrow-flightsql.md b/content/influxdb/cloud-dedicated/reference/internals/arrow-flightsql.md index cef488d42..e8d3790d0 100644 --- a/content/influxdb/cloud-dedicated/reference/internals/arrow-flightsql.md +++ b/content/influxdb/cloud-dedicated/reference/internals/arrow-flightsql.md @@ -9,6 +9,7 @@ menu: weight: 101 related: - /influxdb/cloud-dedicated/reference/sql/ + - /influxdb/cloud-dedicated/reference/client-libraries/flight/ --- The InfluxDB SQL implementation uses [Arrow Flight SQL](https://arrow.apache.org/docs/format/FlightSql.html) diff --git a/content/influxdb/cloud-dedicated/reference/sql/_index.md b/content/influxdb/cloud-dedicated/reference/sql/_index.md index 7999f66e5..e20f713ec 100644 --- a/content/influxdb/cloud-dedicated/reference/sql/_index.md +++ b/content/influxdb/cloud-dedicated/reference/sql/_index.md @@ -562,11 +562,13 @@ WHERE time >= timestamp '2019-09-10T00:00:00Z' AND time <= timestamp '2019-09-19 #### Examples ```sql -SELECT DATE_BIN(INTERVAL '1 hour', time, '2019-09-18T00:00:00Z'::timestamp), +SELECT DATE_BIN(INTERVAL '1 hour', time, '2019-09-18T00:00:00Z'::timestamp) AS "_time", SUM(water_level) FROM "h2o_feet" -GROUP BY DATE_BIN(INTERVAL '1 hour', time, '2019-09-18T00:00:00Z'::timestamp) +GROUP BY "_time" +``` +```sql SELECT DATE_TRUNC('month',time) AS "date", SUM(water_level) FROM "h2o_feet" diff --git a/content/influxdb/cloud-dedicated/reference/sql/functions/time-and-date.md b/content/influxdb/cloud-dedicated/reference/sql/functions/time-and-date.md index b8ad3ad6a..6b9eb877b 100644 --- a/content/influxdb/cloud-dedicated/reference/sql/functions/time-and-date.md +++ b/content/influxdb/cloud-dedicated/reference/sql/functions/time-and-date.md @@ -161,7 +161,7 @@ FROM "h2o_feet" WHERE time >= timestamp '2019-09-10T00:00:00Z' AND time <= timestamp '2019-09-20T00:00:00Z' -GROUP BY date_bin(INTERVAL '1 day', time, TIMESTAMP '1970-01-01 00:00:00Z') +GROUP BY 1 ORDER BY time DESC ``` diff --git a/content/influxdb/cloud-dedicated/reference/sql/group-by.md b/content/influxdb/cloud-dedicated/reference/sql/group-by.md index ad33c7295..1088eb33f 100644 --- a/content/influxdb/cloud-dedicated/reference/sql/group-by.md +++ b/content/influxdb/cloud-dedicated/reference/sql/group-by.md @@ -61,18 +61,18 @@ Group results in 15 minute time intervals by tag: ```sql SELECT "location", - DATE_BIN(INTERVAL '15 minutes', time, TIMESTAMP '2022-01-01 00:00:00Z') AS time, + DATE_BIN(INTERVAL '15 minutes', time, TIMESTAMP '2022-01-01 00:00:00Z') AS _time, COUNT("water_level") AS count FROM "h2o_feet" WHERE time >= timestamp '2019-09-17T00:00:00Z' AND time <= timestamp '2019-09-17T01:00:00Z' GROUP BY - DATE_BIN(INTERVAL '15 minutes', time, TIMESTAMP '2022-01-01 00:00:00Z'), + _time, location ORDER BY location, - time + _time ``` {{< expand-wrapper >}}} @@ -81,18 +81,18 @@ ORDER BY The query uses the `COUNT()` function to count the number of `water_level` points per 15 minute interval. Results are then ordered by location and time. -| location | time | count | -| :----------- | :------------------- | ----: | -| coyote_creek | 2019-09-16T23:45:00Z | 1 | -| coyote_creek | 2019-09-17T00:00:00Z | 2 | -| coyote_creek | 2019-09-17T00:15:00Z | 3 | -| coyote_creek | 2019-09-17T00:30:00Z | 2 | -| coyote_creek | 2019-09-17T00:45:00Z | 3 | -| santa_monica | 2019-09-16T23:45:00Z | 1 | -| santa_monica | 2019-09-17T00:00:00Z | 2 | -| santa_monica | 2019-09-17T00:15:00Z | 3 | -| santa_monica | 2019-09-17T00:30:00Z | 2 | -| santa_monica | 2019-09-17T00:45:00Z | 3 | +| location | _time | count | +| :----------- | :-------------------- | ----: | +| coyote_creek | 2019-09-16T23:45:00Z | 1 | +| coyote_creek | 2019-09-17T00:00:00Z | 2 | +| coyote_creek | 2019-09-17T00:15:00Z | 3 | +| coyote_creek | 2019-09-17T00:30:00Z | 2 | +| coyote_creek | 2019-09-17T00:45:00Z | 3 | +| santa_monica | 2019-09-16T23:45:00Z | 1 | +| santa_monica | 2019-09-17T00:00:00Z | 2 | +| santa_monica | 2019-09-17T00:15:00Z | 3 | +| santa_monica | 2019-09-17T00:30:00Z | 2 | +| santa_monica | 2019-09-17T00:45:00Z | 3 | {{% /expand %}} {{< /expand-wrapper >}} diff --git a/content/influxdb/cloud-serverless/get-started/query.md b/content/influxdb/cloud-serverless/get-started/query.md index 0a77575a2..04c6044b0 100644 --- a/content/influxdb/cloud-serverless/get-started/query.md +++ b/content/influxdb/cloud-serverless/get-started/query.md @@ -47,7 +47,7 @@ The examples in this section of the tutorial query the [**get-started** bucket]( - [InfluxDB user interface (UI)](?t=InfluxDB+UI#execute-an-sql-query){{< req "\* " >}} - [`influx3` data CLI](?t=influx3+CLI#execute-an-sql-query){{< req "\* " >}} - [InfluxDB v3 client libraries](/influxdb/cloud-serverless/reference/client-libraries/v3/) -- [Flight SQL clients](?t=Go#execute-an-sql-query){{< req "\* " >}} +- [Flight clients](?t=Go#execute-an-sql-query){{< req "\* " >}} - [Superset](/influxdb/cloud-serverless/query-data/sql/execute-queries/superset/) - [Grafana](/influxdb/cloud-serverless/query-data/sql/execute-queries/grafana/) - [Chronograf](/{{< latest "chronograf" >}}/) diff --git a/content/influxdb/cloud-serverless/process-data/downsample.md b/content/influxdb/cloud-serverless/process-data/downsample.md index 339ad820e..9cafac50f 100644 --- a/content/influxdb/cloud-serverless/process-data/downsample.md +++ b/content/influxdb/cloud-serverless/process-data/downsample.md @@ -149,9 +149,11 @@ SELECT AVG(hum) AS hum, AVG(co) AS co FROM home +--In WHERE, time refers to .time WHERE time >= now() - INTERVAL '24 hours' +--1 refers to the DATE_BIN column GROUP BY 1, room -ORDER BY room, time +ORDER BY time ``` {{% /tab-content %}} @@ -210,9 +212,11 @@ SELECT AVG(hum) AS hum, AVG(co) AS co FROM home +--In WHERE, time refers to .time WHERE time >= now() - INTERVAL '24 hours' +--1 refers to the DATE_BIN column GROUP BY 1, room -ORDER BY time +ORDER BY 1 ''' table = influxdb_raw.query(query=query, language="sql") @@ -316,9 +320,11 @@ SELECT AVG(hum) AS hum, AVG(co) AS co FROM home +--In WHERE, time refers to .time WHERE time >= now() - INTERVAL '24 hours' +--1 refers to the DATE_BIN column GROUP BY 1, room -ORDER BY time +ORDER BY 1 ''' table = influxdb_raw.query(query=query, language="sql") diff --git a/content/influxdb/cloud-serverless/process-data/visualize/_index.md b/content/influxdb/cloud-serverless/process-data/visualize/_index.md index a4fb34637..92777e3c4 100644 --- a/content/influxdb/cloud-serverless/process-data/visualize/_index.md +++ b/content/influxdb/cloud-serverless/process-data/visualize/_index.md @@ -8,7 +8,7 @@ menu: parent: Process & visualize data weight: 103 related: - - influxdb/cloud-dedicated/query-data/ + - influxdb/cloud-serverless/query-data/ --- Use visualization tools like Grafana, Superset, and others to visualize time diff --git a/content/influxdb/cloud-serverless/query-data/influxql/execute-queries/python.md b/content/influxdb/cloud-serverless/query-data/influxql/execute-queries/python.md index 6ad17a217..188692871 100644 --- a/content/influxdb/cloud-serverless/query-data/influxql/execute-queries/python.md +++ b/content/influxdb/cloud-serverless/query-data/influxql/execute-queries/python.md @@ -22,7 +22,7 @@ list_code_example: | client = InfluxDBClient3( host='cloud2.influxdata.com', org='ORG_NAME', - token='DATABASE_TOKEN', + token='API_TOKEN', database='DATABASE_NAME' ) @@ -38,8 +38,8 @@ list_code_example: | --- Use the `influxdb3-python` client library to query data stored in InfluxDB with InfluxQL. -The `influxdb3-client` uses Flight SQL to query data from InfluxDB and return -results in Apache Arrow format. +The `influxdb3-client` uses InfluxDB v3's Flight RPC protocol to query data from InfluxDB and return +results in Apache Arrow Table format. - [Get started using Python to query InfluxDB](#get-started-using-python-to-query-influxdb) - [Create a Python virtual environment](#create-a-python-virtual-environment) @@ -228,7 +228,7 @@ and to instantiate a client configured for an InfluxDB database. In your editor, copy and paste the following sample code to a new file--for example, `query-example.py`: -{{% code-placeholders "(DATABASE|ORG)_(NAME|TOKEN)" %}} +{{% code-placeholders "(DATABASE|ORG|API)_(NAME|TOKEN)" %}} ```py # query-example.py @@ -238,7 +238,7 @@ from influxdb_client_3 import InfluxDBClient3 client = InfluxDBClient3( host='cloud2.influxdata.com', org='ORG_NAME', - token='DATABASE_TOKEN', + token='API_TOKEN', database='DATABASE_NAME' ) ``` @@ -246,8 +246,8 @@ client = InfluxDBClient3( Replace the following configuration values: -- {{% code-placeholder-key %}}`DATABASE_TOKEN`{{% /code-placeholder-key %}}: - Your InfluxDB token with read permissions on the databases you want to query. +- {{% code-placeholder-key %}}`API_TOKEN`{{% /code-placeholder-key %}}: + Your InfluxDB [token](/influxdb/cloud-serverless/admin/tokens/) with read permissions on the databases you want to query. - {{% code-placeholder-key %}}`DATABASE_NAME`{{% /code-placeholder-key %}}: The name of your InfluxDB database. @@ -267,7 +267,7 @@ query(query: str, language: str) #### Example {#execute-query-example} -{{% code-placeholders "(DATABASE|ORG)_(NAME|TOKEN)" %}} +{{% code-placeholders "(DATABASE|ORG|API)_(NAME|TOKEN)" %}} ```py # query-example.py @@ -276,7 +276,7 @@ from influxdb_client_3 import InfluxDBClient3 client = InfluxDBClient3( host='cloud2.influxdata.com', org='ORG_NAME', - token='DATABASE_TOKEN', + token='API_TOKEN', database='DATABASE_NAME' ) diff --git a/content/influxdb/cloud-serverless/query-data/sql/aggregate-select.md b/content/influxdb/cloud-serverless/query-data/sql/aggregate-select.md index e3930740e..f63fd487f 100644 --- a/content/influxdb/cloud-serverless/query-data/sql/aggregate-select.md +++ b/content/influxdb/cloud-serverless/query-data/sql/aggregate-select.md @@ -33,13 +33,11 @@ list_code_example: | sum(field2), tag1 FROM home - GROUP BY - DATE_BIN(INTERVAL '1 hour', time, '2022-01-01T00:00:00Z'::TIMESTAMP), - tag1 + GROUP BY 1, tag1 ``` --- -A SQL query that aggregates data includes the following clauses: +An SQL query that aggregates data includes the following clauses: {{< req type="key" >}} @@ -207,8 +205,9 @@ groups: nearest to `home.time`: ```sql - SELECT DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) AS time - from home + SELECT + DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) AS time + FROM home ... ``` @@ -220,16 +219,27 @@ groups: - In your `GROUP BY` clause: - - Use the [`DATE_BIN` function](/influxdb/cloud-serverless/reference/sql/functions/time-and-date/#date_bin) with the same parameters used in the `SELECT` clause. + - Specify the `DATE_BIN(...)` column ordinal reference (`1`). - Specify other columns (for example, `room`) that are specified in the `SELECT` clause and aren't used in a selector function. ```sql - SELECT DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) AS time + SELECT + DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) AS time ... - GROUP BY DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP), room + GROUP BY 1, room ... ``` + To reference the `DATE_BIN(...)` result column by _name_ in the `GROUP BY` clause, assign an alias other than "time" in the `SELECT` clause--for example: + + ```sql + SELECT + DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) AS _time + FROM home + ... + GROUP BY _time, room + ``` + - Include an `ORDER BY` clause with columns to sort by. The following example retrieves unique combinations of time intervals and rooms with their minimum, maximum, and average temperatures. @@ -242,8 +252,8 @@ SELECT selector_min(temp, time)['value'] AS 'min temp', avg(temp) AS 'average temp' FROM home -GROUP BY DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP), room -ORDER BY room, time +GROUP BY 1, room +ORDER BY room, 1 ``` {{< expand-wrapper >}} @@ -269,6 +279,25 @@ ORDER BY room, time {{% /expand %}} {{< /expand-wrapper >}} +{{% note %}} +#### GROUP BY time + +In the `GROUP BY` clause, the name "time" always refers to the `time` column in the source table. +If you want to reference a calculated time column by name, use an alias different from "time"--for example: + +```sql +SELECT + DATE_BIN(INTERVAL '2 hours', time, '1970-01-01T00:00:00Z'::TIMESTAMP) + AS _time, + room, + selector_max(temp, time)['value'] AS 'max temp', + selector_min(temp, time)['value'] AS 'min temp', + avg(temp) AS 'average temp' +FROM home +GROUP BY _time, room +ORDER BY room, _time +``` +{{% /note %}} ### Query rows based on aggregate values diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/_index.md b/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/_index.md deleted file mode 100644 index 38b75a48b..000000000 --- a/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/_index.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -title: Flight SQL clients -description: > - Flight SQL clients are language-specific drivers that can interact with SQL databases using the Arrow in-memory format and the Flight RPC framework. - View the list of available clients. -weight: 30 -menu: - influxdb_cloud_serverless: - name: Flight SQL clients - parent: Client libraries -influxdb/cloud-serverless/tags: [client libraries, Flight SQL] ---- - -Flight SQL clients are language-specific drivers that can interact with SQL databases using the Arrow in-memory format and the Flight RPC framework. -Clients provide a [`FlightClient`](https://arrow.apache.org/docs/python/generated/pyarrow.flight.FlightClient.html#pyarrow.flight.FlightClient) that can query and retrieve data from InfluxDB v3 using gRPC and Flight SQL. -Some [InfluxDB v3 client libraries](/influxdb/cloud-serverless/reference/client-libraries/v3) include Flight SQL clients. - -For specifics about a client library, see the library's GitHub repository. - -{{< children depth="999" description="true" >}} diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/go-flightsql.md b/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/go-flightsql.md deleted file mode 100644 index cbe4a22b3..000000000 --- a/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/go-flightsql.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -title: Go Flight SQL client -description: The Go Flight SQL client integrates with golang scripts and applications to query data stored in an InfluxDB Cloud Serverless bucket. -external_url: https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight/flightsql -menu: - influxdb_cloud_serverless: - name: Go - parent: Flight SQL clients - identifier: go-flightsql-client - params: - url: https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight/flightsql -influxdb/cloud-serverless/tags: [Golang, gRPC, SQL, Flight SQL, client libraries] -weight: 201 ---- - -The [Go Flight SQL client](https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight/flightsql) integrates with golang scripts and applications to query data stored in an InfluxDB Cloud Serverless bucket. - -For more information, see the [Go client README on GitHub](https://github.com/influxdata/influxdb-client-go). diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/python-flightsql-dbapi.md b/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/python-flightsql-dbapi.md deleted file mode 100644 index 47424ac35..000000000 --- a/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/python-flightsql-dbapi.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -title: Python FlightSQL-DBAPI client -description: The Python `flightsql-dbapi` library integrates with Python scripts and applications to query data stored in an InfluxDB Cloud Serverless bucket. -external_url: https://github.com/influxdata/flightsql-dbapi -menu: - influxdb_cloud_serverless: - name: Python - parent: Flight SQL clients - identifier: python-flightsql-client - params: - url: https://github.com/influxdata/flightsql-dbapi -influxdb/cloud-serverless/tags: [Python, gRPC, SQL, Flight SQL, client libraries] -weight: 201 ---- - -The [Python Flight SQL DBAPI library](https://github.com/influxdata/flightsql-dbapi) integrates with Python scripts and applications to query data stored in an InfluxDB Cloud Serverless bucket. - -The documentation for this client library is available on GitHub. - -Python Flight SQL DBAPI library \ No newline at end of file diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/flight/_index.md b/content/influxdb/cloud-serverless/reference/client-libraries/flight/_index.md new file mode 100644 index 000000000..ce6d6e9fc --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/client-libraries/flight/_index.md @@ -0,0 +1,38 @@ +--- +title: Apache Arrow Flight RPC clients +description: > + Flight clients are language-specific drivers that can interact with Flight servers using the Arrow in-memory format and the Flight RPC framework. + View the list of available clients. +weight: 30 +menu: + influxdb_cloud_serverless: + name: Arrow Flight clients + parent: Client libraries +influxdb/cloud-serverless/tags: [client libraries, Flight RPC, Flight SQL] +aliases: + - /influxdb/cloud-serverless/reference/client-libraries/flight-sql/ + - /influxdb/cloud-serverless/reference/client-libraries/flight-sql/go-flightsql/ + - /influxdb/cloud-serverless/reference/client-libraries/flight-sql/python-flightsql-dbapi/ +--- + +Flight RPC and Flight SQL clients are language-specific drivers that interact with databases using the Arrow in-memory format and the Flight RPC protocol. +Apache Arrow Flight RPC and Flight SQL protocols define APIs for servers and clients. + +{{% note %}} +#### InfluxDB v3 client libraries + +We recommend using [InfluxDB v3 client libraries](/influxdb/cloud-serverless/reference/client-libraries/v3/) for integrating InfluxDB v3 with your application code. +Client libraries wrap Apache Arrow Flight clients +and provide convenient methods for writing, querying, and processing data stored in {{% cloud-name %}}. +{{% /note %}} + +**Flight RPC clients** can use SQL or InfluxQL to query data stored in an {{% cloud-name %}} bucket. +Using InfluxDB v3's IOx-specific Flight RPC protocol, clients send a single `DoGet()` request to authenticate, query, and retrieve data. + +**Flight SQL clients** use the [Flight SQL protocol](https://arrow.apache.org/docs/format/FlightSql.html) for querying an SQL database server. +They can use SQL to query data stored in an {{% cloud-name %}} bucket, but they can't use InfuxQL. + +Clients are maintained by Apache Arrow projects or third-parties. +For specifics about a Flight client, see the client's GitHub repository. + +{{< children >}} diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/flight/csharp-flight.md b/content/influxdb/cloud-serverless/reference/client-libraries/flight/csharp-flight.md new file mode 100644 index 000000000..4d7df103a --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/client-libraries/flight/csharp-flight.md @@ -0,0 +1,20 @@ +--- +title: C# .NET Flight client +description: The C# .NET Flight client integrates with C# .NET scripts and applications to query data stored in InfluxDB. +external_url: https://github.com/apache/arrow/tree/main/csharp/examples/FlightClientExample +menu: + influxdb_cloud_serverless: + name: C# .NET + parent: Arrow Flight clients + params: + url: https://github.com/apache/arrow/tree/main/csharp/examples/FlightClientExample + identifier: csharp-flight-client +influxdb/cloud-serverless/tags: [C#, gRPC, SQL, Flight SQL, client libraries] +aliases: + - /influxdb/cloud-serverless/reference/client-libraries/flight-sql/csharp-flightsql/ +weight: 201 +--- + +[Apache Arrow for C# .NET](https://github.com/apache/arrow/blob/main/csharp/README.md) integrates with C# .NET scripts and applications to query data stored in InfluxDB. + +For more information, see the [C# client example on GitHub](https://github.com/apache/arrow/tree/main/csharp/examples/FlightClientExample). diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/flight/go-flight.md b/content/influxdb/cloud-serverless/reference/client-libraries/flight/go-flight.md new file mode 100644 index 000000000..2e15859f7 --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/client-libraries/flight/go-flight.md @@ -0,0 +1,20 @@ +--- +title: Go Flight client +description: The Go Flight client integrates with Go scripts and applications to query data stored in InfluxDB. +menu: + influxdb_cloud_serverless: + name: Go + parent: Arrow Flight clients + params: + url: https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight#Client + identifier: go-flight-client +influxdb/cloud-serverless/tags: [Golang, gRPC, SQL, Flight SQL, client libraries] +aliases: + - /influxdb/cloud-serverless/reference/client-libraries/flight-sql/go-flightsql/ +weight: 201 +--- + +[Apache Arrow for Go](https://pkg.go.dev/github.com/apache/arrow/go/v12) integrates with golang scripts and applications to query data stored in InfluxDB. +See the [Flight SQL example](/influxdb/cloud-serverless/get-started/query/?t=Go#execute-an-sql-query). + +For more information, see the [Go Arrow Flight Client documentation](https://pkg.go.dev/github.com/apache/arrow/go/v12/arrow/flight#Client). diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/java-flightsql.md b/content/influxdb/cloud-serverless/reference/client-libraries/flight/java-flightsql.md similarity index 52% rename from content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/java-flightsql.md rename to content/influxdb/cloud-serverless/reference/client-libraries/flight/java-flightsql.md index 4e1b72ca2..aa55eaf95 100644 --- a/content/influxdb/cloud-serverless/reference/client-libraries/flight-sql/java-flightsql.md +++ b/content/influxdb/cloud-serverless/reference/client-libraries/flight/java-flightsql.md @@ -1,18 +1,20 @@ --- title: Java Flight SQL package -description: The `org.apache.arrow.flight.sql` package integrates with Java applications to query and retrieve data from Flight database servers using RPC and SQL. +description: The Java Flight SQL clients integrates with Java applications to query and retrieve data from Flight database servers using RPC and SQL. external_url: https://arrow.apache.org/docs/java/reference/org/apache/arrow/flight/sql/package-summary.html menu: influxdb_cloud_serverless: - name: Java - parent: Flight SQL clients + name: Java Flight SQL + parent: Arrow Flight clients identifier: java-flightsql-client params: url: https://arrow.apache.org/docs/java/reference/org/apache/arrow/flight/sql/package-summary.html influxdb/cloud-serverless/tags: [Java, gRPC, SQL, Flight SQL, client libraries] weight: 201 +aliases: + - /influxdb/cloud-serverless/reference/client-libraries/flight-sql/java-flightsql/ --- -The [`org.apache.arrow.flight.sql`](https://arrow.apache.org/docs/java/reference/org/apache/arrow/flight/sql/package-summary.html) package integrates with Java applications to query and retrieve data from Flight database servers using RPC and SQL. +[Apache Arrow Flight SQL for Java](https://arrow.apache.org/docs/java/reference/org/apache/arrow/flight/sql/package-summary.html) integrates with Java applications to query and retrieve data from Flight database servers using RPC and SQL. Java Flight SQL package diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/flight/python-flight.md b/content/influxdb/cloud-serverless/reference/client-libraries/flight/python-flight.md new file mode 100644 index 000000000..756246188 --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/client-libraries/flight/python-flight.md @@ -0,0 +1,147 @@ +--- +title: Python Flight client +description: The Python Flight client integrates with Python scripts and applications to query data stored in InfluxDB. +menu: + influxdb_cloud_serverless: + name: Python + parent: Arrow Flight clients + identifier: python-flight-client +influxdb/cloud-serverless/tags: [Python, gRPC, SQL, Flight SQL, client libraries] +aliases: + - /influxdb/cloud-serverless/reference/client-libraries/flight-sql/python-flightsql/ +weight: 201 +list_code_example: | + ```py + from pyarrow.flight import FlightClient, Ticket, FlightCallOptions + import json + import pandas + import tabulate + + # Downsampling query groups data into 2-hour bins + sql=""" + SELECT DATE_BIN(INTERVAL '2 hours', + time, + '1970-01-01T00:00:00Z'::TIMESTAMP) AS time, + room, + selector_max(temp, time)['value'] AS 'max temp', + selector_min(temp, time)['value'] AS 'min temp', + avg(temp) AS 'average temp' + FROM home + GROUP BY + 1, + room + ORDER BY room, 1""" + + flight_ticket = Ticket(json.dumps({ + "namespace_name": "BUCKET_NAME", + "sql_query": sql, + "query_type": "sql" + })) + + token = (b"authorization", bytes(f"Bearer API_TOKEN".encode('utf-8'))) + options = FlightCallOptions(headers=[token]) + client = FlightClient(f"grpc+tls://cloud2.influxdata.com:443") + + reader = client.do_get(flight_ticket, options) + arrow_table = reader.read_all() + ``` +--- + +[Apache Arrow Python bindings](https://arrow.apache.org/docs/python/index.html) integrate with Python scripts and applications to query data stored in InfluxDB. + +The following examples show how to use the `pyarrow.flight` and `pandas` Python modules to query and format data stored in an {{% cloud-name %}} bucket: + +{{% code-tabs-wrapper %}} +{{% code-tabs %}} +[SQL](#sql-python) +[InfluxQL](#influxql-python) +{{% /code-tabs %}} +{{% code-tab-content %}} + +{{% code-placeholders "BUCKET_NAME|API_TOKEN" %}} +```py +# Using pyarrow>=12.0.0 FlightClient +from pyarrow.flight import FlightClient, Ticket, FlightCallOptions +import json +import pandas +import tabulate + +# Downsampling query groups data into 2-hour bins +sql=""" + SELECT DATE_BIN(INTERVAL '2 hours', + time, + '1970-01-01T00:00:00Z'::TIMESTAMP) AS time, + room, + selector_max(temp, time)['value'] AS 'max temp', + selector_min(temp, time)['value'] AS 'min temp', + avg(temp) AS 'average temp' + FROM home + GROUP BY + 1, + room + ORDER BY room, 1""" + +flight_ticket = Ticket(json.dumps({ + "namespace_name": "BUCKET_NAME", + "sql_query": sql, + "query_type": "sql" +})) + +token = (b"authorization", bytes(f"Bearer API_TOKEN".encode('utf-8'))) +options = FlightCallOptions(headers=[token]) +client = FlightClient(f"grpc+tls://cloud2.influxdata.com:443") + +reader = client.do_get(flight_ticket, options) +arrow_table = reader.read_all() +# Use pyarrow and pandas to view and analyze data +data_frame = arrow_table.to_pandas() +print(data_frame.to_markdown()) +``` +{{% /code-placeholders %}} + +{{% /code-tab-content %}} +{{% code-tab-content %}} + +{{% code-placeholders "BUCKET_NAME|API_TOKEN" %}} +```py +# Using pyarrow>=12.0.0 FlightClient +from pyarrow.flight import FlightClient, Ticket, FlightCallOptions +import json +import pandas +import tabulate + +# Downsampling query groups data into 2-hour bins +influxql=""" + SELECT FIRST(temp) + FROM home + WHERE room = 'kitchen' + AND time >= now() - 100d + AND time <= now() - 10d + GROUP BY time(2h)""" + +flight_ticket = Ticket(json.dumps({ + "namespace_name": "BUCKET_NAME", + "sql_query": influxql, + "query_type": "influxql" +})) + +token = (b"authorization", bytes(f"Bearer API_TOKEN".encode('utf-8'))) +options = FlightCallOptions(headers=[token]) +client = FlightClient(f"grpc+tls://cloud2.influxdata.com:443") + +reader = client.do_get(flight_ticket, options) +arrow_table = reader.read_all() +# Use pyarrow and pandas to view and analyze data +data_frame = arrow_table.to_pandas() +print(data_frame.to_markdown()) +``` +{{% /code-placeholders %}} + +{{% /code-tab-content %}} + +Replace the following: + +- {{% code-placeholder-key %}}`BUCKET_NAME`{{% /code-placeholder-key %}}: your {{% cloud-name %}} bucket +- {{% code-placeholder-key %}}`API_TOKEN`{{% /code-placeholder-key %}}: a [token](/influxdb/cloud-serverless/admin/tokens/) with sufficient permissions to the bucket + +{{% /code-tabs-wrapper %}} diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/flight/python-flightsql-dbapi.md b/content/influxdb/cloud-serverless/reference/client-libraries/flight/python-flightsql-dbapi.md new file mode 100644 index 000000000..2425334b7 --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/client-libraries/flight/python-flightsql-dbapi.md @@ -0,0 +1,168 @@ +--- +title: Python Flight SQL DBAPI client +description: The Python `flightsql-dbapi` library uses SQL and the Flight SQL protocol to query data stored in an InfluxDB Cloud Serverless bucket. +menu: + influxdb_cloud_serverless: + name: Python Flight SQL + parent: Arrow Flight clients + identifier: python-flightsql-client +influxdb/cloud-serverless/tags: [Python, SQL, Flight SQL] +weight: 201 +--- + +The [Python `flightsql-dbapi` Flight SQL DBAPI library](https://github.com/influxdata/flightsql-dbapi) integrates with Python applications using SQL to query data stored in an {{% cloud-name %}} bucket. The `flightsql-dbapi` library uses the [Flight SQL protocol](https://arrow.apache.org/docs/format/FlightSql.html) to query and retrieve data. + +{{% note %}} +#### InfluxDB v3 client libraries + +We recommend using the [`influxdb3-python` Python client library](/influxdb/cloud-serverless/reference/client-libraries/v3/python/) for integrating InfluxDB v3 with your Python application code. + +[InfluxDB v3 client libraries](/influxdb/cloud-serverless/reference/client-libraries/v3/) client libraries wrap Apache Arrow Flight clients +and provide convenient methods for writing, querying, and processing data stored in {{% cloud-name %}}. +Client libraries can query using [SQL](/influxdb/cloud-serverless/query-data/sql/execute-queries/python/) or [InfluxQL](/influxdb/cloud-serverless/query-data/influxql/execute-queries/python/). +{{% /note %}} + +## Installation + +The [`flightsql-dbapi`](https://github.com/influxdata/flightsql-dbapi) Flight SQL library for Python provides a +[DB API 2](https://peps.python.org/pep-0249/) interface and +[SQLAlchemy](https://www.sqlalchemy.org/) dialect for +[Flight SQL](https://arrow.apache.org/docs/format/FlightSql.html). +Installing `flightsql-dbapi` also installs the [`pyarrow`](https://arrow.apache.org/docs/python/index.html) library that you'll use for working with Arrow data. + +In your terminal, use `pip` to install `flightsql-dbapi`: + +```sh +pip install flightsql-dbapi +``` + +## Importing the module + +The `flightsql-dbapi` package provides the `flightsql` module. From the module, import the `FlightSQLClient` class method: + +```py +from flightsql import FlightSQLClient +``` + +- `flightsql.FlightSQLClient` class: an interface for [initializing +a client](#initialization) and interacting with a Flight SQL server. + +## API reference + +- [Class FlightSQLClient](#class-flightsqlclient) + - [Syntax](#syntax) +- [Initialize a client](#initialize-a-client) + - [Instance methods](#instance-methods) + - [FlightSQLClient.execute](#flightsqlclientexecute) + - [Syntax {#execute-query-syntax}](#syntax-execute-query-syntax) + - [Example {#execute-query-example}](#example-execute-query-example) + - [FlightSQLClient.do_get](#flightsqlclientdo_get) + - [Syntax {#retrieve-data-syntax}](#syntax-retrieve-data-syntax) + - [Example {#retrieve-data-example}](#example-retrieve-data-example) + +## Class FlightSQLClient + +Provides an interface for [initializing +a client](#initialize-a-client) and interacting with a Flight SQL server. + +### Syntax + +```py +__init__(self, host=None, token=None, metadata=None, features=None) +``` + +Initializes and returns a `FlightSQLClient` instance for interating with the server. + +## Initialize a client + +The following example shows how to use Python with `flightsql-dbapi` +and the _DB API 2_ interface to instantiate a Flight SQL client configured for an InfluxDB database. + +{{% code-placeholders "BUCKET_NAME|API_TOKEN" %}} +```py +from flightsql import FlightSQLClient + +# Instantiate a FlightSQLClient configured for a database +client = FlightSQLClient(host='cloud2.influxdata.com', + token='API_TOKEN', + metadata={'database': 'BUCKET_NAME'}, + features={'metadata-reflection': 'true'}) +``` +{{% /code-placeholders %}} + +Replace the following: + +- {{% code-placeholder-key %}}`API_TOKEN`{{% /code-placeholder-key %}}: an {{% cloud-name %}} [API token](/influxdb/cloud-serverless/admin/tokens/) with read permissions on the buckets you want to query +- {{% code-placeholder-key %}}`BUCKET_NAME`{{% /code-placeholder-key %}}: the name of your {{% cloud-name %}} [bucket](/influxdb/cloud-serverless/admin/buckets/) + +### Instance methods + +### FlightSQLClient.execute + +Sends a Flight SQL RPC request to execute the specified SQL Query. + +#### Syntax {#execute-query-syntax} + +```py +execute(query: str, call_options: Optional[FlightSQLCallOptions] = None) +``` + +#### Example {#execute-query-example} + +```py +# Execute the query +info = client.execute("SELECT * FROM home") +``` + +The response contains a `flight.FlightInfo` object that contains metadata and an `endpoints: [...]` list. Each endpoint contains the following: + +- A list of addresses where you can retrieve query result data. +- A `ticket` value that identifies the data to [retrieve](#retrieve-data-example). + +### FlightSQLClient.do_get + +Passes a Flight ticket (obtained from a `FlightSQLClient.execute` response) and retrieves Arrow data identified by the ticket. +Returns a `pyarrow.flight.FlightStreamReader` for streaming the data. + +#### Syntax {#retrieve-data-syntax} + +```py + do_get(ticket, call_options: Optional[FlightSQLCallOptions] = None) +``` + +#### Example {#retrieve-data-example} + +The following sample shows how to use Python with `flightsql-dbapi` and `pyarrow` to query InfluxDB and retrieve data. + +```py +from flightsql import FlightSQLClient + +# Instantiate a FlightSQLClient configured for a database +client = FlightSQLClient(host='cloud2.influxdata.com', + token='API_TOKEN', + metadata={'database': 'BUCKET_NAME'}, + features={'metadata-reflection': 'true'}) + +# Execute the query to retrieve FlightInfo +info = client.execute("SELECT * FROM home") + +# Extract the token for retrieving data +ticket = info.endpoints[0].ticket + +# Use the ticket to request the Arrow data stream. +# Return a FlightStreamReader for streaming the results. +reader = client.do_get(ticket) + +# Read all data to a pyarrow.Table +table = reader.read_all() + +print(table) +``` + +`do_get(ticket)` returns a [`pyarrow.flight.FlightStreamReader`](https://arrow.apache.org/docs/python/generated/pyarrow.flight.FlightStreamReader.html) for streaming Arrow [record batches](https://arrow.apache.org/docs/python/data.html#record-batches). + +To read data from the stream, call one of the following `FlightStreamReader` methods: + +- `read_all()`: Read all record batches as a [`pyarrow.Table`](https://arrow.apache.org/docs/python/generated/pyarrow.Table.html). +- `read_chunk()`: Read the next RecordBatch and metadata. +- `read_pandas()`: Read all record batches and convert them to a [`pandas.DataFrame`](https://pandas.pydata.org/docs/reference/frame.html). diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/v3/_index.md b/content/influxdb/cloud-serverless/reference/client-libraries/v3/_index.md index ac8d2c930..c1528e2a1 100644 --- a/content/influxdb/cloud-serverless/reference/client-libraries/v3/_index.md +++ b/content/influxdb/cloud-serverless/reference/client-libraries/v3/_index.md @@ -1,7 +1,7 @@ --- title: InfluxDB v3 API client libraries description: > - InfluxDB v3 client libraries use InfluxDB HTTP APIs to write data and use [Flight SQL clients](/influxdb/cloud-serverless/reference/client-libraries/flight-sql) to query data stored in an InfluxDB Cloud Serverless bucket. + InfluxDB v3 client libraries use InfluxDB HTTP APIs to write data and use [Flight clients](/influxdb/cloud-serverless/reference/client-libraries/flight-sql) to execute SQL and InfluxQL queries. View the list of available client libraries. weight: 30 menu: @@ -20,9 +20,9 @@ InfluxDB client libraries provide configurable batch writing of data to InfluxDB They can be used to construct line protocol data and transform data from other formats to line protocol. -InfluxDB v3 client libraries can query InfluxDB v3 using the Arrow Flight protocol with gRPC. -Client libraries use Flight clients to execute SQL queries, request -bucket information, and retrieve data stored in {{% cloud-name %}}. +InfluxDB v3 client libraries can query InfluxDB v3 using InfluxDB v3's IOx-specific Arrow Flight protocol with gRPC. +Client libraries use Flight clients to execute SQL and InfluxQL queries, request +database information, and retrieve data stored in {{% cloud-name %}}. Additional functionality may vary among client libraries and some may not be feature-complete. diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/v3/csharp.md b/content/influxdb/cloud-serverless/reference/client-libraries/v3/csharp.md new file mode 100644 index 000000000..fe0dd9c6e --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/client-libraries/v3/csharp.md @@ -0,0 +1,21 @@ +--- +title: C# .NET client library for InfluxDB v3 +list_title: C# .NET +description: > + The InfluxDB v3 `influxdb3-csharp` C# .NET client library integrates with C# .NET scripts and applications to write and query data stored in an InfluxDB Cloud Serverless bucket. +external_url: https://github.com/InfluxCommunity/influxdb3-csharp +menu: + influxdb_cloud_serverless: + name: C# .NET + parent: v3 client libraries + identifier: influxdb3-csharp +influxdb/cloud-serverless/tags: [C#, gRPC, SQL, Flight SQL, client libraries] +weight: 201 +--- + +The InfluxDB v3 [`influxdb3-csharp` C# .NET client library](https://github.com/InfluxCommunity/influxdb3-csharp) integrates with C# .NET scripts and applications +to write and query data stored in an {{% cloud-name %}} bucket. + +The documentation for this client library is available on GitHub. + +InfluxDB v3 C# .NET client library diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/v3/go.md b/content/influxdb/cloud-serverless/reference/client-libraries/v3/go.md index df6840ee8..8d7717fee 100644 --- a/content/influxdb/cloud-serverless/reference/client-libraries/v3/go.md +++ b/content/influxdb/cloud-serverless/reference/client-libraries/v3/go.md @@ -16,7 +16,7 @@ aliases: --- The InfluxDB v3 [`influxdb3-go` Go client library](https://github.com/InfluxCommunity/influxdb3-go) integrates with Go scripts and applications -to write and query data stored in an {{% cloud-name %}} database. +to write and query data stored in an {{% cloud-name %}} bucket. The documentation for this client library is available on GitHub. diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/v3/java.md b/content/influxdb/cloud-serverless/reference/client-libraries/v3/java.md new file mode 100644 index 000000000..4f929b64d --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/client-libraries/v3/java.md @@ -0,0 +1,21 @@ +--- +title: Java client library for InfluxDB v3 +list_title: Java +description: > + The InfluxDB v3 `influxdb3-java` Java client library integrates with Java scripts and applications to write and query data stored in an InfluxDB Cloud Serverless bucket. +external_url: https://github.com/InfluxCommunity/influxdb3-java +menu: + influxdb_cloud_serverless: + name: Java + parent: v3 client libraries + identifier: influxdb3-java +influxdb/cloud-serverless/tags: [Java, gRPC, SQL, Flight SQL, client libraries] +weight: 201 +--- + +The InfluxDB v3 [`influxdb3-java` Java client library](https://github.com/InfluxCommunity/influxdb3-java) integrates with Java scripts and applications +to write and query data stored in an {{% cloud-name %}} bucket. + +The documentation for this client library is available on GitHub. + +InfluxDB v3 Java client library diff --git a/content/influxdb/cloud-serverless/reference/client-libraries/v3/javascript.md b/content/influxdb/cloud-serverless/reference/client-libraries/v3/javascript.md new file mode 100644 index 000000000..d65aa63ac --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/client-libraries/v3/javascript.md @@ -0,0 +1,24 @@ +--- +title: JavaScript client library for InfluxDB v3 +list_title: JavaScript +description: > + The InfluxDB v3 `influxdb3-js` JavaScript client library integrates with JavaScript scripts and applications to write and query data stored in an InfluxDB Cloud Serverless bucket. +external_url: https://github.com/InfluxCommunity/influxdb3-js +menu: + influxdb_cloud_serverless: + name: JavaScript + parent: v3 client libraries + identifier: influxdb3-js +influxdb/cloud-serverless/tags: [JavaScript, gRPC, SQL, Flight SQL, client libraries] +weight: 201 +aliases: + - /influxdb/cloud-serverless/reference/api/client-libraries/go/ + - /influxdb/cloud-serverless/tools/client-libraries/go/ +--- + +The InfluxDB v3 [`influxdb3-js` JavaScript client library](https://github.com/InfluxCommunity/influxdb3-js) integrates with JavaScript scripts and applications +to write and query data stored in an {{% cloud-name %}} bucket. + +The documentation for this client library is available on GitHub. + +InfluxDB v3 JavaScript client library diff --git a/content/influxdb/cloud-serverless/reference/glossary.md b/content/influxdb/cloud-serverless/reference/glossary.md index e04068f45..1affa6d3d 100644 --- a/content/influxdb/cloud-serverless/reference/glossary.md +++ b/content/influxdb/cloud-serverless/reference/glossary.md @@ -421,8 +421,8 @@ Related entries: diff --git a/content/influxdb/cloud-serverless/reference/internals/arrow-flightsql.md b/content/influxdb/cloud-serverless/reference/internals/arrow-flightsql.md new file mode 100644 index 000000000..d7da23ec7 --- /dev/null +++ b/content/influxdb/cloud-serverless/reference/internals/arrow-flightsql.md @@ -0,0 +1,69 @@ +--- +title: Arrow Flight SQL +description: > + The InfluxDB SQL implementation uses **Arrow Flight SQL** to query InfluxDB + and return results. +menu: + influxdb_cloud_serverless: + parent: InfluxDB Cloud internals +weight: 101 +related: + - /influxdb/cloud-serverless/reference/sql/ + - /influxdb/cloud-serverless/reference/client-libraries/flight/ +--- + +The InfluxDB SQL implementation uses [Arrow Flight SQL](https://arrow.apache.org/docs/format/FlightSql.html) +to query InfluxDB and return results. + +> Arrow Flight SQL is a protocol for interacting with SQL databases using the +> Arrow in-memory format and the [Flight RPC](https://arrow.apache.org/docs/format/Flight.html) +> framework. +> +> {{% cite %}}-- [Arrow Flight SQL documentation](https://arrow.apache.org/docs/format/FlightSql.html){{% /cite %}} + +Flight SQL uses the RPC methods defined in the in Flight RPC framework and provides +various commands that pair those methods with request and response messages. +The InfluxDB Flight SQL implementation supports the following Flight SQL commands: + +### Flight SQL metadata commands + +{{% caption %}} +_For command descriptions, see the +[Arrow Flight SQL RPC methods documentation](https://arrow.apache.org/docs/format/FlightSql.html#sql-metadata)_. +{{% /caption %}} + +| Message | Supported | +| :----------------------- | :----------------------: | +| CommandGetCatalogs | **{{% icon "check" %}}** | +| CommandGetDbSchemas | **{{% icon "check" %}}** | +| CommandGetTables | **{{% icon "check" %}}** | +| CommandGetTableTypes | **{{% icon "check" %}}** | +| CommandGetSqlInfo | **{{% icon "check" %}}** | +| CommandGetPrimaryKeys | **{{% icon "check" %}}** | +| CommandGetExportedKeys | **{{% icon "check" %}}** | +| CommandGetImportedKeys | **{{% icon "check" %}}** | +| CommandGetCrossReference | **{{% icon "check" %}}** | +| CommandGetXdbcTypeInfo | | + +### Flight SQL query execution commands + +{{% caption %}} +_For command descriptions, see the +[Arrow Flight SQL RPC methods documentation](https://arrow.apache.org/docs/format/FlightSql.html#query-execution)_. +{{% /caption %}} + +| Message | Supported | +| :--------------------------------------- | :----------------------: | +| ActionCreatePreparedStatementRequest | | +| ActionCreatePreparedSubstraitPlanRequest | | +| ActionClosePreparedStatementRequest | | +| ActionBeginTransactionRequest | | +| ActionBeginSavepointRequest | | +| ActionBeginSavepointResult | | +| ActionEndTransactionRequest | | +| ActionEndSavepointRequest | | +| CommandStatementQuery | **{{% icon "check" %}}** | +| CommandStatementSubstraitPlan | | +| CommandPreparedStatementQuery | **{{% icon "check" %}}** | +| CommandPreparedStatementUpdate | | +| ActionCancelQueryRequest | | diff --git a/content/influxdb/cloud-serverless/reference/sql/_index.md b/content/influxdb/cloud-serverless/reference/sql/_index.md index 8f907e68c..2093d5e0e 100644 --- a/content/influxdb/cloud-serverless/reference/sql/_index.md +++ b/content/influxdb/cloud-serverless/reference/sql/_index.md @@ -7,6 +7,8 @@ menu: name: SQL reference parent: Reference weight: 101 +related: + - /influxdb/cloud-serverless/reference/internals/arrow-flightsql/ --- {{% cloud-name %}} uses the [Apache Arrow DataFusion](https://arrow.apache.org/datafusion/) implementation of SQL. @@ -560,11 +562,13 @@ WHERE time >= timestamp '2019-09-10T00:00:00Z' AND time <= timestamp '2019-09-19 #### Examples ```sql -SELECT DATE_BIN(INTERVAL '1 hour', time, '2019-09-18T00:00:00Z'::timestamp), +SELECT DATE_BIN(INTERVAL '1 hour', time, '2019-09-18T00:00:00Z'::timestamp) AS "_time", SUM(water_level) FROM "h2o_feet" -GROUP BY DATE_BIN(INTERVAL '1 hour', time, '2019-09-18T00:00:00Z'::timestamp) +GROUP BY "_time" +``` +```sql SELECT DATE_TRUNC('month',time) AS "date", SUM(water_level) FROM "h2o_feet" diff --git a/content/influxdb/cloud-serverless/reference/sql/functions/time-and-date.md b/content/influxdb/cloud-serverless/reference/sql/functions/time-and-date.md index 12749b2bc..e050d9a0a 100644 --- a/content/influxdb/cloud-serverless/reference/sql/functions/time-and-date.md +++ b/content/influxdb/cloud-serverless/reference/sql/functions/time-and-date.md @@ -161,7 +161,7 @@ FROM "h2o_feet" WHERE time >= timestamp '2019-09-10T00:00:00Z' AND time <= timestamp '2019-09-20T00:00:00Z' -GROUP BY date_bin(INTERVAL '1 day', time, TIMESTAMP '1970-01-01 00:00:00Z') +GROUP BY 1 ORDER BY time DESC ``` diff --git a/content/influxdb/cloud-serverless/reference/sql/group-by.md b/content/influxdb/cloud-serverless/reference/sql/group-by.md index a198dc2d1..58433b121 100644 --- a/content/influxdb/cloud-serverless/reference/sql/group-by.md +++ b/content/influxdb/cloud-serverless/reference/sql/group-by.md @@ -61,18 +61,18 @@ Group results in 15 minute time intervals by tag: ```sql SELECT "location", - DATE_BIN(INTERVAL '15 minutes', time, TIMESTAMP '2022-01-01 00:00:00Z') AS time, + DATE_BIN(INTERVAL '15 minutes', time, TIMESTAMP '2022-01-01 00:00:00Z') AS _time, COUNT("water_level") AS count FROM "h2o_feet" WHERE time >= timestamp '2019-09-17T00:00:00Z' AND time <= timestamp '2019-09-17T01:00:00Z' GROUP BY - DATE_BIN(INTERVAL '15 minutes', time, TIMESTAMP '2022-01-01 00:00:00Z'), + _time, location ORDER BY location, - time + _time ``` {{< expand-wrapper >}}} @@ -81,18 +81,18 @@ ORDER BY The query uses the `COUNT()` function to count the number of `water_level` points per 15 minute interval. Results are then ordered by location and time. -| location | time | count | -| :----------- | :------------------- | ----: | -| coyote_creek | 2019-09-16T23:45:00Z | 1 | -| coyote_creek | 2019-09-17T00:00:00Z | 2 | -| coyote_creek | 2019-09-17T00:15:00Z | 3 | -| coyote_creek | 2019-09-17T00:30:00Z | 2 | -| coyote_creek | 2019-09-17T00:45:00Z | 3 | -| santa_monica | 2019-09-16T23:45:00Z | 1 | -| santa_monica | 2019-09-17T00:00:00Z | 2 | -| santa_monica | 2019-09-17T00:15:00Z | 3 | -| santa_monica | 2019-09-17T00:30:00Z | 2 | -| santa_monica | 2019-09-17T00:45:00Z | 3 | +| location | _time | count | +| :----------- | :-------------------- | ----: | +| coyote_creek | 2019-09-16T23:45:00Z | 1 | +| coyote_creek | 2019-09-17T00:00:00Z | 2 | +| coyote_creek | 2019-09-17T00:15:00Z | 3 | +| coyote_creek | 2019-09-17T00:30:00Z | 2 | +| coyote_creek | 2019-09-17T00:45:00Z | 3 | +| santa_monica | 2019-09-16T23:45:00Z | 1 | +| santa_monica | 2019-09-17T00:00:00Z | 2 | +| santa_monica | 2019-09-17T00:15:00Z | 3 | +| santa_monica | 2019-09-17T00:30:00Z | 2 | +| santa_monica | 2019-09-17T00:45:00Z | 3 | {{% /expand %}} {{< /expand-wrapper >}}