influxdb/influxdb_iox_client
Marco Neumann 3bd24b67ba
feat: extend flight client to accept multiple (changing) schemas (#4853)
* feat: extend flight client to accept multiple (changing) schemas

See #4849.

Originally I intended not to use Flight at all for the new
ingester<>querier protocol. However since flight also deals with
dictionary batches and multiple batches and the gRPC protocol that I
would write would look very similar, I will use Flight with a bit more
flexible message types.

The rough idea for the protocol is the following stream:

- for each partition:
  1. "none" message with partition metadata
  2. for each chunk (can have different schemas under certain
     circumstances):
     1. "schema" message (resets dictionary state)
     2. (optional) dictionary batch messages
     3. one or more "record batch" message

The nice thing about it is that the same arrow client works also for the
existing client<>querier protocol since there we just send:

1. "schema" message (no app metadata)
2. (optional) dictionary batch messages
3. zero, one or more "record batch" message (no app metadata)

* refactor: separate high- and low-level flight client

It is very unlikely that a user will use the high-level batch-producing
functionality and the low-level stuff within the same session. So let's
split this into to clients (high-level uses the low-level one
internally) to avoid confusion.

Also add documentation on our protocol handling.

* refactor: enumerate all variants in match statement to better catch errors in the future
2022-06-15 11:38:08 +00:00
..
src feat: extend flight client to accept multiple (changing) schemas (#4853) 2022-06-15 11:38:08 +00:00
Cargo.toml chore: Update datafusion + `arrow`/`parquet`/`arrow-flight` to `16.0.0 (#4851) 2022-06-14 16:31:40 +00:00
README.md docs: fix README for influxdb_iox_client (#4825) 2022-06-10 09:13:17 +00:00

README.md

InfluxDB IOx Client

This is the official Rust client library for connecting to InfluxDB IOx.

Currently only gRPC is supported.

Using the gRPC Write Client

To write to IOx, create a connection and a write client, and then send line protocol. Here is an example of creating an instance that connects to an IOx server running at http://127.0.0.1:8081 (the default bind address for the gRPC endpoint of IOx when running in all-in-one mode) and sending a line of line protocol:

#[tokio::main]
fn main() {
    use influxdb_iox_client::{
        write::Client,
        connection::Builder,
    };

    let mut connection = Builder::default()
        .build("http://127.0.0.1:8081")
        .await
        .unwrap();

    let mut client = Client::new(connection);

    // write a line of line procol data
    client
        .write_lp("bananas", "cpu,region=west user=23.2 100",0)
        .await
        .expect("failed to write to IOx");
    }
}