diff --git a/README.md b/README.md index e3f7760446..364dde292c 100644 --- a/README.md +++ b/README.md @@ -28,27 +28,54 @@ We're also hosting monthly tech talks and community office hours on the project * [Signup for upcoming IOx tech talks](https://www.influxdata.com/community-showcase/influxdb-tech-talks) * [Watch past IOx tech talks](https://www.youtube.com/playlist?list=PLYt2jfZorkDp-PKBS05kf2Yx2NrRyPAAz) -## Quick Start +## Get started -To compile and run InfluxDB IOx from source, you'll need a Rust compiler and `clang`. +1. [Install dependencies](#install-dependencies) +1. [Clone the repository](#clone-the-repository) +1. [Configure the server](#configure-the-server) +1. [Compile and start the server](#compile-and-start-the-server) + (You can also [build a Docker image](#build-a-docker-image-optional) to run InfluxDB IOx.) +1. [Write and read data](#write-and-read-data) +1. [Use the CLI](#use-the-cli) +1. [Use InfluxDB 2.0 API compatibility](#use-influxdb-20-api-compatibility) +1. [Run health checks](#run-health-checks) +1. [Manually call the gRPC API](#manually-call-the-grpc-api) -### Build a Docker Image +### Install dependencies -**BuildKit is required.** -[Enable BuildKit](https://docs.docker.com/develop/develop-images/build_enhancements/#to-enable-buildkit-builds): -- Use Docker version 18.09 or later -- Enable BuildKit by default by setting `{ "features": { "buildkit": true } }` in the Docker engine config - - ...or run `docker build .` with env var `DOCKER_BUILDKIT=1` +To compile and run InfluxDB IOx from source, you'll need the following: -To build the Docker image: -``` -DOCKER_BUILDKIT=1 docker build . +- [Rust](#rust) +- [Clang](#clang) + +#### Rust + +The easiest way to install Rust is to use [`rustup`](https://rustup.rs/), a Rust version manager. +Follow the instructions for your operating system on the `rustup` site. + +`rustup` will check the [`rust-toolchain`](./rust-toolchain.toml) file and automatically install and use the correct Rust version for you. + +#### Clang + +Building InfluxDB IOx requires `clang` (for the [`croaring`] dependency). +Check for `clang` by running `clang --version`. + +```shell +clang --version +Apple clang version 12.0.0 (clang-1200.0.32.27) +Target: x86_64-apple-darwin20.1.0 +Thread model: posix +InstalledDir: /Library/Developer/CommandLineTools/usr/bin ``` -### Cloning the Repository +If `clang` is not already present, it can typically be installed with the system package manager. -Using `git`, check out the code by cloning this repository. If you use the `git` command line, this -looks like: +[`croaring`]: https://github.com/saulius/croaring-rs + +### Clone the repository + +Clone this repository using `git`. +If you use the `git` command line, this looks like: ```shell git clone git@github.com:influxdata/influxdb_iox.git @@ -60,61 +87,25 @@ Then change into the directory containing the code: cd influxdb_iox ``` -The rest of the instructions assume you are in this directory. +The rest of these instructions assume you are in this directory. -### Installing Rust +### Configure the server -The easiest way to install Rust is by using [`rustup`], a Rust version manager. -Follow the instructions on the `rustup` site for your operating system. +InfluxDB IOx can be configured using either environment variables or a configutation file, +making it suitable for deployment in containerized environments. -[`rustup`]: https://rustup.rs/ +For a list configuration options, run `influxdb_iox --help`. +For configuration options for specific subcommands, run `influxdb_iox --help`. -By default, `rustup` will install the latest stable version of Rust. InfluxDB IOx is currently -using a nightly version of Rust to get performance benefits from the unstable `simd` feature. The -exact nightly version is specified in the `rust-toolchain` file. When you're in the directory -containing this repository's code, `rustup` will look in the `rust-toolchain` file and -automatically install and use the correct Rust version for you. Test this out with: - -```shell -rustc --version -``` - -and you should see a nightly version of Rust! - -### Installing `clang` - -An installation of `clang` is required to build the [`croaring`] dependency - if -it is not already present, it can typically be installed with the system -package manager. - -```shell -clang --version -Apple clang version 12.0.0 (clang-1200.0.32.27) -Target: x86_64-apple-darwin20.1.0 -Thread model: posix -InstalledDir: /Library/Developer/CommandLineTools/usr/bin -``` - -[`croaring`]: https://github.com/saulius/croaring-rs - -### Specifying Configuration - -IOx is designed for running in modern containerized environments. As such, it -takes its configuration as environment variables. - -You can see a list of the current configuration values by running `influxdb_iox ---help`, as well as the specific subcommand config options such as `influxdb_iox -run --help`. - -Should you desire specifying config via a file, you can do so using a -`.env` formatted file in the working directory. You can use the -provided [example](docs/env.example) as a template if you want: +To use a configuration file, use a `.env` file in the working directory. +See the provided [example configuration file](docs/env.example). +To use the example configuration file, run: ```shell cp docs/env.example .env ``` -### Compiling and Starting the Server +### Compile and start the server InfluxDB IOx is built using Cargo, Rust's package manager and build tool. @@ -124,40 +115,53 @@ To compile for development, run: cargo build ``` -which will create a binary in `target/debug` that you can run with: +This which will create a binary at `target/debug/influxdb_iox`. + +To start the InfluxDB IOx server, run: ```shell -./target/debug/influxdb_iox +./target/debug/influxdb_iox run ``` -You can compile and run with one command by using: +By default the server will start an HTTP server on port `8080` and a gRPC server on port `8082`. + +You can also compile and run with one command: ```shell -cargo run -- server +cargo run -- run ``` -When compiling for performance testing, build in release mode by using: +To compile for performance testing, build in release mode: ```shell cargo build --release -``` - -which will create the corresponding binary in `target/release`: - -```shell ./target/release/influxdb_iox run ``` -Similarly, you can do this in one step with: +You can also run in release mode with one step: ```shell -cargo run --release -- server +cargo run --release -- run ``` -The server will, by default, start an HTTP API server on port `8080` and a gRPC server on port -`8082`. +### Build a Docker image (optional) -### Writing and Reading Data +Building the Docker image requires: + +- Docker 18.09+ +- BuildKit + +To [enable BuildKit] by default, set `{ "features": { "buildkit": true } }` in the Docker engine configuration, +or run `docker build` with`DOCKER_BUILDKIT=1` + +To build the Docker image: + +``` +DOCKER_BUILDKIT=1 docker build . +``` + +[Enable BuildKit]: https://docs.docker.com/develop/develop-images/build_enhancements/#to-enable-buildkit-builds +### Write and read data Each IOx instance requires a writer ID. This can be set one of 4 ways: @@ -165,22 +169,20 @@ This can be set one of 4 ways: - set a flag `--writer-id 42` - use the API (not convered here) - use the CLI -```shell -influxdb_iox writer set 42 -``` + ```shell + influxdb_iox writer set 42 + ``` -To write data, you need to create a database. You can do so via the API or using the CLI. For example, to create a database called `company_sensors` with a 100MB mutable buffer, use this command: +To write data, you need to create a database. +You can do so via the API or using the CLI. +For example, to create a database called `company_sensors`, use this command: ```shell influxdb_iox database create company_sensors ``` -Data can be stored in InfluxDB IOx by sending it in [line protocol] -format to the `/api/v2/write` endpoint or using the CLI. For example, -here is a command that will send the data in the -`tests/fixtures/lineproto/metrics.lp` file in this repository, -assuming that you're running the server on the default port into -the `company_sensors` database, you can use: +Data can be stored in InfluxDB IOx by sending it in [line protocol] format to the `/api/v2/write` endpoint or using the CLI. +For example, here is a command that will send the data in the `tests/fixtures/lineproto/metrics.lp` file in this repository, assuming that you're running the server on the default port into the `company_sensors` database, you can use: ```shell influxdb_iox database write company_sensors tests/fixtures/lineproto/metrics.lp @@ -192,28 +194,23 @@ To query data stored in the `company_sensors` database: influxdb_iox database query company_sensors "SELECT * FROM cpu LIMIT 10" ``` -### Using the CLI +### Use the CLI -To ease deloyment, IOx is packaged as a combined binary which has -commands to start the IOx server as well as a CLI interface for -interacting with and configuring such servers. +InfluxDB IOx is packaged as a binary with commands to start the IOx server, +as well as a CLI interface for interacting with and configuring such servers. -The CLI itself is documented via extensive built in help which you can -access by runing `influxdb_iox --help` +The CLI itself is documented via built-in help which you can access by running `influxdb_iox --help` - -### InfluxDB 2.0 compatibility +### Use InfluxDB 2.0 API compatibility InfluxDB IOx allows seamless interoperability with InfluxDB 2.0. -InfluxDB 2.0 stores data in organization and buckets, but InfluxDB IOx -stores data in named databases. IOx maps `organization` and `bucket` -to a database named with the two parts separated by an underscore -(`_`): `organization_bucket`. +Where InfluxDB 2.0 stores data in organizations and buckets, +InfluxDB IOx stores data in _named databases_. +IOx maps `organization` and `bucket` pairs to databases named with the two parts separated by an underscore (`_`): +`organization_bucket`. -Here's an example using [`curl`] command to send the same data into -the `company_sensors` database using the InfluxDB 2.0 `/api/v2/write` -API: +Here's an example using [`curl`] to send data into the `company_sensors` database using the InfluxDB 2.0 `/api/v2/write` API: ```shell curl -v "http://127.0.0.1:8080/api/v2/write?org=company&bucket=sensors" --data-binary @tests/fixtures/lineproto/metrics.lp @@ -222,8 +219,7 @@ curl -v "http://127.0.0.1:8080/api/v2/write?org=company&bucket=sensors" --data-b [line protocol]: https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/ [`curl`]: https://curl.se/ - -### Health Checks +### Run health checks The HTTP API exposes a healthcheck endpoint at `/health` @@ -232,21 +228,20 @@ $ curl http://127.0.0.1:8080/health OK ``` -The gRPC API implements the [gRPC Health Checking Protocol](https://github.com/grpc/grpc/blob/master/doc/health-checking.md). This can be tested with [grpc-health-probe](https://github.com/grpc-ecosystem/grpc-health-probe) +The gRPC API implements the [gRPC Health Checking Protocol](https://github.com/grpc/grpc/blob/master/doc/health-checking.md). +This can be tested with [`grpc-health-probe`](https://github.com/grpc-ecosystem/grpc-health-probe): ```console $ grpc_health_probe -addr 127.0.0.1:8082 -service influxdata.platform.storage.Storage status: SERVING ``` -### Manually calling gRPC API +### Manually call the gRPC API -If you want to manually invoke one of the gRPC APIs, you can use any gRPC CLI client; -a good one is [grpcurl](https://github.com/fullstorydev/grpcurl). +To manually invoke one of the gRPC APIs, use a gRPC CLI client such as [grpcurl](https://github.com/fullstorydev/grpcurl). -Tonic (the gRPC server library we're using) currently doesn't have support for gRPC reflection, -hence you must pass all `.proto` files to your client. You can find a conventient `grpcurl` wrapper -that does that in the `scripts` directory: +Tonic (the gRPC server library we're using) currently doesn't have support for gRPC reflection, hence you must pass all `.proto` files to your client. +You can find a conventient `grpcurl` wrapper that does that in the `scripts` directory: ```console $ ./scripts/grpcurl -plaintext 127.0.0.1:8082 list