influxdb/ingester2
Dom Dwyer adc6fcfb04
feat(catalog): linearise sort key updates
Updating the sort key is not commutative and MUST be serialised. The
correctness of the current catalog interface relies on the caller
serialising updates globally, something it cannot reasonably assert in a
distributed system.

This change of the catalog interface pushes this responsibility to the
catalog itself where it can be effectively enforced, and allows a caller
to detect parallel updates to the sort key.
2022-12-20 12:31:00 +01:00
..
benches feat: optimize wal with batching (#6399) 2022-12-14 16:07:20 +00:00
src feat(catalog): linearise sort key updates 2022-12-20 12:31:00 +01:00
Cargo.toml chore(deps): Bump thiserror from 1.0.37 to 1.0.38 2022-12-19 10:33:32 +00:00
README.md fix(ingester2): Make ingester2 work with existing catalogs and document quickstart 2022-12-16 13:16:31 +01:00

README.md

ingester2

Quick run

Set-up empty catalog db:

mkdir -p /tmp/iox/{wal,obj}

createdb iox_shared
./target/debug/influxdb_iox catalog setup --catalog-dsn postgres:///iox_shared

# there has to exist one "topic", see https://github.com/influxdata/influxdb_iox/issues/6420
psql 'dbname=iox_shared options=-csearch_path=public,iox_catalog' -c "insert into topic (name) values ('iox-shared')"

Run ingester2:

INFLUXDB_IOX_RPC_MODE=2 ./target/debug/influxdb_iox run ingester2 --api-bind=127.0.0.1:8081 --grpc-bind=127.0.0.1:8042 --wal-directory /tmp/iox/wal  --catalog-dsn postgres:///iox_shared --object-store=file --data-dir=/tmp/iox/obj -v 

Run router2:

INFLUXDB_IOX_RPC_MODE=2 ./target/debug/influxdb_iox run router2 --api-bind=127.0.0.1:8080 --grpc-bind=127.0.0.1:8085 --ingester-addresses=127.0.0.1:8042 --catalog-dsn postgres:///iox_shared -v

Run querier:

INFLUXDB_IOX_RPC_MODE=2 ./target/debug/influxdb_iox run querier --ingester-addresses=http://127.0.0.1:8042 --api-bind 127.0.0.1:8083 --grpc-bind 127.0.0.1:8082 --catalog-dsn postgres:///iox_shared --object-store=file --data-dir=/tmp/iox/obj -v