influxdb/ingester2
Marco Neumann 20ec47b00b
feat: virtual chunk order col (#7240)
* feat: introduce `CHUNK_ORDER_COLUMN_NAME`

* feat: impl `ChunkOrder` everywhere

* feat: `ChunkOrder::get`

* feat: emit chunk order column for `RecordBatchesExec`

* feat: `chunk_order_field`

* feat: chunk order col for parquet chunks

* feat: optional chunk order col handling for dedup

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-03-17 09:39:21 +00:00
..
benches feat(metrics): ingester2 WAL replay 2023-02-02 14:52:09 +01:00
src feat: virtual chunk order col (#7240) 2023-03-17 09:39:21 +00:00
tests Merge branch 'main' into dom/query-instrumentation 2023-02-23 10:30:51 +00:00
Cargo.toml chore(deps): Bump futures from 0.3.26 to 0.3.27 (#7193) 2023-03-13 10:53:59 +00:00
README.md fix(ingester2): Make ingester2 work with existing catalogs and document quickstart 2022-12-16 13:16:31 +01:00

README.md

ingester2

Quick run

Set-up empty catalog db:

mkdir -p /tmp/iox/{wal,obj}

createdb iox_shared
./target/debug/influxdb_iox catalog setup --catalog-dsn postgres:///iox_shared

# there has to exist one "topic", see https://github.com/influxdata/influxdb_iox/issues/6420
psql 'dbname=iox_shared options=-csearch_path=public,iox_catalog' -c "insert into topic (name) values ('iox-shared')"

Run ingester2:

INFLUXDB_IOX_RPC_MODE=2 ./target/debug/influxdb_iox run ingester2 --api-bind=127.0.0.1:8081 --grpc-bind=127.0.0.1:8042 --wal-directory /tmp/iox/wal  --catalog-dsn postgres:///iox_shared --object-store=file --data-dir=/tmp/iox/obj -v 

Run router2:

INFLUXDB_IOX_RPC_MODE=2 ./target/debug/influxdb_iox run router2 --api-bind=127.0.0.1:8080 --grpc-bind=127.0.0.1:8085 --ingester-addresses=127.0.0.1:8042 --catalog-dsn postgres:///iox_shared -v

Run querier:

INFLUXDB_IOX_RPC_MODE=2 ./target/debug/influxdb_iox run querier --ingester-addresses=http://127.0.0.1:8042 --api-bind 127.0.0.1:8083 --grpc-bind 127.0.0.1:8082 --catalog-dsn postgres:///iox_shared --object-store=file --data-dir=/tmp/iox/obj -v