Commit Graph

360 Commits (1ddc64d68db906c6490f36d4aecde7ccd5bff945)

Author SHA1 Message Date
Andrew Lamb 73bed810da
chore: Update arrow, arrow-flight, parquet, tonic, prost, etc (#4357)
* chore: Update datafusion

* chore: Update arrow/arrow-flight/parquet to 12

* chore: update datafusion correctly

* chore: Update prost, tonic, and dependents

* fix: Fixup some api changes

* fix: Update test output in db

* fix: Update test output in parquet_file

* fix: remove old pbjson types

* fix: Add "--experimental_allow_proto3_optional" flag

* chore: Run cargo hakari tasks

* fix: compile error

* chore: Update heappy

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-20 11:12:17 +00:00
Andrew Lamb 0642ec0b82
docs: add note about write_info API being internal (#4356)
* docs: add note about write_info API being internal

* fix: update doc urls

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-20 09:25:14 +00:00
Andrew Lamb 5ea676d3f7
feat: add per kafka partition durability reporting to write info response (#4341)
* feat: add per kafka partition durability reporting to write info response

* fix: buf lint + test cleanup

* fix: clean up protobuf

* refactor: pull out conversion of KafkaPartitionStatus into a function

* fix: fmt

* fix: typo

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-19 16:46:20 +00:00
Paul Dix 103629b01d
feat: add client and CLI to get file from object store (#4343)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-19 16:16:27 +00:00
Paul Dix 197a3818d3
feat: add catalog client and remote command (#4329)
* feat: add catalog client and remote command

Adds the catalog gRPC service to influxdb_iox_client.
Adds a new remote command to execute commands against a remote IOx host.
Adds partition subcommand to remote to get the details of a partition by id.

* test: add end to end test for `remote partition` CLI (#4336)

* chore: cleanup partition CLI PR feedback

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2022-04-15 13:59:49 +00:00
Marco Neumann 380cd9bbff
refactor: use a single flight client implementation (#4273)
"end-user -> querier" and "querier -> ingester" should use a single
Flight client implementation. The difference is just the request and
response metadata.

This changes our default Flight client to use protobuf instead of JSON
for the ticket format.
2022-04-12 09:08:25 +00:00
Andrew Lamb a30a85e62c
feat: Add get_write_info service (#4227)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-04-07 19:24:58 +00:00
Andrew Lamb a1df864283
feat: Support 'SHOW NAMESPACES' in sql repl (#4164)
* feat: Support `SHOW NAMESPACES` in sql repl

* feat: add basic support to clients

* fix: add get_namespaces service test

* fix: proper error handling

* test: end to end test for namespace client

* refactor: Use QuerierDatabase rather than Catalog

* refactor: remove unused function
2022-03-31 12:57:33 +00:00
Andrew Lamb 5c69a3f43b
chore: Update deps: datafusion, arrow/arrow-flight/parquet to 11, zstd to 0.11 (#4119)
* chore: update datafusion

* chore(deps): Bump arrow from 10.0.0 to 11.0.0

Bumps [arrow](https://github.com/apache/arrow-rs) from 10.0.0 to 11.0.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/10.0.0...11.0.0)

---
updated-dependencies:
- dependency-name: arrow
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): Bump arrow-flight from 10.0.0 to 11.0.0

Bumps [arrow-flight](https://github.com/apache/arrow-rs) from 10.0.0 to 11.0.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/10.0.0...11.0.0)

---
updated-dependencies:
- dependency-name: arrow-flight
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: update parquet to 11.0.0

* fix: error on create schema, test for same

* fix: upgrade zstd

* chore: Run cargo hakari tasks

* fix: fix logical merge conflict

* fix: hakari

* fix: hakari

* fix: update newly introduced dep

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-24 15:27:36 +00:00
Luke Bond e109fa4987
feat: schema client and CLI (#4105)
* feat: schema client and CLI

* chore: clarification in comment in schema command

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-23 13:49:24 +00:00
Andrew Lamb 2c3d30ca32
chore: Update datafusion, arrow, flight and parquet (#4000)
* chore: Update datafusion, arrow, flight and parquet

* fix: api change

* fix: fmt

* fix: update test metadata size

* fix: Update sizes in parquet test

* fix: more metadata size update
2022-03-10 12:24:47 +00:00
Andrew Lamb d2c0acdd46
refactor: Remove serving readiness gate (#3986)
* refactor: Remove serving_readiness

* fix: remove more

* fix: remove test

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-09 12:17:44 +00:00
Andrew Lamb 677a272095
refactor: Clean up some future clippy warnings from nightly (#3892)
* refactor: clean up new clippy lints

* refactor: complete other cleanups

* fix: ignore overzealous clippy

* fix: re-remove old code
2022-03-03 19:14:27 +00:00
dependabot[bot] 5a79b3a68b
chore(deps): Bump arrow-flight from 9.0.2 to 9.1.0 (#3829)
Bumps [arrow-flight](https://github.com/apache/arrow-rs) from 9.0.2 to 9.1.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/9.0.2...9.1.0)

---
updated-dependencies:
- dependency-name: arrow-flight
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-23 11:03:22 +00:00
dependabot[bot] 3b7d31c88a
chore(deps): Bump arrow from 9.0.2 to 9.1.0 (#3826)
Bumps [arrow](https://github.com/apache/arrow-rs) from 9.0.2 to 9.1.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/9.0.2...9.1.0)

---
updated-dependencies:
- dependency-name: arrow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-23 09:25:46 +00:00
dependabot[bot] ad3868ed7c
chore(deps): Bump tokio from 1.16.1 to 1.17.0 (#3814)
* chore(deps): Bump tokio from 1.16.1 to 1.17.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.16.1 to 1.17.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.16.1...tokio-1.17.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build: update workspace-hack

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom Dwyer <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-22 16:27:43 +00:00
Andrew Lamb a30803e692
chore: Update datafusion, update `arrow`/`parquet`/`arrow-flight` to 9.0 (#3733)
* chore: Update datafusion

* chore: Update arrow

* fix: missing updates

* chore: Update cargo.lock

* fix: update for smaller parquet size

* fix: update test for smaller parquet files

* test: ensure parquet_file tests write multiple row groups

* fix: update callsite

* fix: Update for tests

* fix: harkari

* fix: use IoxObjectStore::existing

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-15 12:10:24 +00:00
dependabot[bot] 60a7f87645
chore(deps): Bump serde_json from 1.0.78 to 1.0.79 (#3739)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.78...v1.0.79)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-14 20:42:54 +00:00
Raphael Taylor-Davies 4e3f66ed07
feat: CLI and gRPC APIs for shutting down and restarting databases (#3720)
* feat: allow catalog wipe and rebuild whilst shutdown

* feat: CLI and gRPC APIs for shutting down and restarting databases

* feat: add ability to skip replay on restart

* fix: test_wipe_persisted_catalog_error_db_exists

* fix: wipe_preserved_catalog
2022-02-11 10:14:43 +00:00
Carol (Nichols || Goulding) 73828323ac
feat: Ingester Flight gRPC API (#3623)
* feat: Add a way to run ingester with an in-memory catalog from the CLI

If you set the --catalog-dsn string to "mem", rather than using that as
a Postgres connection URL, create an in-memory catalog.

Planning on using this in tests, so not documenting.

* fix: Set default topic to the same value as SHARED_KAFKA_TOPIC

Namely, both should use an underscore. I don't think there's a way to
directly share these values between a constant and an annotation.

* feat: Add a flight API (handshake only) to ingester

* fix: Create partitions if using file-based write buffer

* fix: Change the server fixture to handle ingester server type

For now, the ingester doesn't implement the deployment API. Not sure if
it should or not.

* feat: Start implementing ingester do_get, namely decoding the query

Skip serialization of the predicate for the moment.

* refactor: Rename ingest protos to ingester to match crate name

* refactor: Rename QueryResults to QueryData

* feat: Move ingester flight client to new querier crate

* fix: Off by one error, different starting indexes in sequencers

* fix: Create new CLI argument to pick the catalog type

* fix: Create a CLI option to set the number of topics to auto-create in the write buffer

* fix: Check the arrow flight service's health to tell that the ingester gRPC is up

* fix: Set postgres as the default catalog type

* fix: Return an error rather than panicking if CLI args aren't right
2022-02-09 19:07:44 +00:00
Raphael Taylor-Davies 4101d16f71
chore: feature flag consistency (#3574)
* chore: feature flag consistency

* chore: add aarch64-apple-darwin to hakari

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-28 16:38:59 +00:00
Paul Dix 16d584b2ff
feat: Add db_name/namespace to DmlWrite and DmlDelete (#3531)
* feat: Add db_name/namespace to DmlWrite and DmlDelete

This is required for the new ingester to be able to work with the write buffer. The protobuf that gets serialized over Kafka already includes the database name, it just wasn't getting carried through to the marshaled Dml operation.

* fix: database != namespace, propagation through write buffer

Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 14:12:20 +00:00
Andrew Lamb 5488c257d1
chore: Update datafusion, upgrade to arrow/parqet/arrow-flight 8.0.0 (#3517)
* chore: Update datafusion

* chore: update to arrow 8

* fix: update to use new DataFusion APIs

* fix: update case for sortedness

* fix: cargo hakari
2022-01-27 13:33:27 +00:00
Andrew Lamb dd23056efd
chore: update datafusion, arrow, prost, tonic, pbjson, etc (#3455)
* chore: update datafusion, arrow, prost, tonic, etc

* fix: update pprof as well

* chore: update hakari

* fix: update pbjson

* chore: update heappy

* fix: hakari

* fix: workaround https://github.com/influxdata/influxdb_iox/issues/3458

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-13 17:07:15 +00:00
Andrew Lamb 758b65dd29
feat: Add database initialization state and errors to CLI and remove list_databases_detailed gRPC (#3377)
* feat: Add database initialization state and errors to CLI:

* fix: do not use optional in protobuf

* fix: clippy

* fix: correct check I broke appeasing clippy
2021-12-15 12:18:41 +00:00
Nga Tran c0ba69f09e chore: marge main to branch and resolve conflict 2021-12-09 15:40:33 -05:00
Andrew Lamb 3cda6b6c0f
refactor: Remove collect_query and replication (#3348)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-09 19:58:19 +00:00
Nga Tran efbfbb1a0b feat: compact all object store chunks of a given partition 2021-12-08 16:06:03 -05:00
Andrew Lamb c6a3765d76
feat: Add force flag to RebuildCatalog (#3292)
* feat: Add force flag to RebuildCatalog

* fix: small cleanups

* docs: Update comments and add WARNING
2021-12-08 15:36:07 +00:00
Marco Neumann c17a6c10c1 feat: add `GetRouter` gRPC method 2021-12-08 13:45:55 +01:00
kodiakhq[bot] 75cd1d24f4
Merge branch 'main' into ntran/grpc_compact_os_chunks 2021-12-07 03:26:06 +00:00
Nga Tran 3834435620 chore: Merge branch 'ntran/grpc_compact_os_chunks' of https://github.com/influxdata/influxdb_iox into ntran/grpc_compact_os_chunks 2021-12-06 12:06:22 -05:00
Nga Tran d8ed8a3bf7 refactor: address review comments 2021-12-06 12:06:07 -05:00
Carol (Nichols || Goulding) 02c297e850
fix: Always specify the parking_lot feature of tokio to get potential perf boost 2021-12-06 09:37:15 -05:00
Carol (Nichols || Goulding) 7a241882c2
fix: Make version reqs on futures-related crates consistent 2021-12-06 09:37:14 -05:00
Nga Tran 65660dace5
Merge branch 'main' into ntran/grpc_compact_os_chunks 2021-12-03 18:16:42 -05:00
Nga Tran 5f3706e0ee feat: grpc call for copact object store chunks 2021-12-03 18:01:28 -05:00
Andrew Lamb 7dea2ef362
feat: Add force flag to ClaimDatabase (#3284)
* feat: Add force flag to ClaimDatabase

* fix: Update name of test

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* docs: Update influxdb_iox/tests/end_to_end_cases/management_cli.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-12-03 21:11:23 +00:00
Raphael Taylor-Davies f17b670fe9
refactor: cleanup gRPC client error handling (#3251) (#3277)
* refactor: cleanup gRPC client error handling (#3251)

* chore: review feedback

* chore: review feedback

Co-authored-by: Andrew Lamb <alamb@influxdata.com>

Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-02 20:56:58 +00:00
Edd Robinson 5becb7e359 refactor: tidy up comments 2021-12-02 12:59:52 +00:00
Raphael Taylor-Davies 1e515a1dec
feat: load RUB from object store (#3224) (#3250) 2021-11-30 14:39:52 +00:00
Raphael Taylor-Davies 88acf3788e
feat: rebuild catalog (#3225) (#3253)
* feat: rebuild catalog (#3225)

* chore: fix doc

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-30 12:17:27 +00:00
Raphael Taylor-Davies f165faf5bc
feat: support forcing persistence of partition irrespective of arrival time (#3249)
* feat: force persistence of partition irrespective of arrival time

* feat: expose to CLI

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 22:07:35 +00:00
dependabot[bot] 5d784ba866
chore(deps): bump serde_json from 1.0.71 to 1.0.72
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:28:07 +00:00
Marco Neumann aa636eaed9 refactor: remove methods from `ManagementService` that were moved to `RemoteService`
Tests are covered by `remote_api.rs`.

Ref #2980.
2021-11-24 10:01:32 +01:00
Marco Neumann 164db06235 fix: remove obsolete errors
This was a leftover after #3201 was merged.
2021-11-24 09:54:25 +01:00
Marco Neumann 5ecaa6d8a9 refactor: use serialized predicates instead of strings for gRPC deletes
IOx is the only consumer of this API so we might just use the serialized
form. Cloud2 uses the HTTP API which supports SQL-like predicates.

Fixes #3192.
2021-11-24 09:30:32 +01:00
Marco Neumann dc6b44818d refactor: remove methods from `ManagementService` that were moved to `DeploymentService`
Tests are covered by `deployment_api.rs`.

Ref #2980.
2021-11-23 18:04:41 +01:00
Marco Neumann 2ef3ab8544 refactor: remove methods from `ManagementService` that were moved to `DeleteService`
Tests are covered by `delete_api.rs`.

Ref #2980.
2021-11-23 16:40:00 +01:00
dependabot[bot] 5c6deb81db
chore(deps): bump serde_json from 1.0.70 to 1.0.71
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.70 to 1.0.71.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.70...v1.0.71)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-22 01:21:06 +00:00
Carol (Nichols || Goulding) f69d37e9a8
fix: Remove database delete/restore entirely 2021-11-17 12:03:11 -05:00
Carol (Nichols || Goulding) bc11244828
feat: Rename database disown/adopt to release/claim (#3111)
* fix: Rename 'disown' to 'release' database

Connects to #3110

* fix: Rename 'adopt' to 'claim' database

Fixes #3110.
2021-11-15 20:28:09 +00:00
Jake Goulding af28cfa2a6
feat: Add an adopt database API
Fixes #2679.
2021-11-15 09:26:06 -05:00
kodiakhq[bot] 236edc0e7e
Merge branch 'main' into crepererum/improve_router_client_errors 2021-11-15 11:36:57 +00:00
kodiakhq[bot] 9868430c12
Merge branch 'main' into crepererum/fix_server_id_on_router_startup 2021-11-15 10:29:21 +00:00
Marco Neumann 48d0388ea0 feat: improve `RouterClient` errors 2021-11-15 11:18:46 +01:00
dependabot[bot] 3485d70409
chore(deps): bump serde_json from 1.0.69 to 1.0.70
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.69 to 1.0.70.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.69...v1.0.70)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-15 08:48:44 +00:00
kodiakhq[bot] 376471591f
Merge branch 'main' into cn/disown 2021-11-12 19:31:03 +00:00
Carol (Nichols || Goulding) 8bdfb253ce
fix: Remove context from disown operation 2021-11-12 11:10:20 -05:00
Marco Neumann f3d4b27358 fix: also export `write_buffer` gRPC types from router client
This is required to construct a write-buffer-based router config.
2021-11-12 12:13:50 +01:00
Marco Neumann 8c1cf00164 feat: improve and test handling of updating a server ID a 2nd time 2021-11-12 11:31:52 +01:00
Carol (Nichols || Goulding) 84a5df44e3
fix: Let structopt handle parsing Uuid 2021-11-11 14:37:19 -05:00
kodiakhq[bot] 95c46e58bc
Merge branch 'main' into jpg/restore-uuid-bytes 2021-11-11 15:34:23 +00:00
Raphael Taylor-Davies 6f268f8260
refactor: extract DML types (#2731) (#3084)
* refactor: extract DML types (#2731)

* chore: fmt
2021-11-11 12:34:07 +00:00
Carol (Nichols || Goulding) 7e13cb77ac
feat: add transactions for context to database owner info on disown 2021-11-10 16:49:32 -05:00
Carol (Nichols || Goulding) 489fad0040
feat: Add disown database to the gRPC API 2021-11-10 16:49:31 -05:00
Carol (Nichols || Goulding) b1e81186d3
feat: Add disown database commands to the database CLI 2021-11-10 16:49:31 -05:00
Carol (Nichols || Goulding) d4d52de8b7
docs: Correct copypasta comment 2021-11-10 16:49:31 -05:00
dependabot[bot] 7214fa9d6a
chore(deps): bump serde_json from 1.0.68 to 1.0.69
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.68 to 1.0.69.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.68...v1.0.69)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-08 10:31:33 +00:00
Jake Goulding de672b7599 fix: Store the restore database UUID as bytes instead of as a string 2021-11-05 15:20:18 -04:00
Raphael Taylor-Davies 88868e7496
feat: remove legacy write service from influxdb_iox_client (#3043)
* feat: remove legacy write service from influxdb_iox_client

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-05 17:26:18 +00:00
Marco Neumann 4840dc11cb feat: add basic CRUD operations for router configs 2021-11-04 12:03:14 +01:00
Carol (Nichols || Goulding) 52d9df5caf
fix: Thread a Not Found error on restore database through all APIs 2021-11-03 15:02:21 -04:00
Carol (Nichols || Goulding) 62e7376394
fix: Restore database should only need UUID, not database name too 2021-11-03 15:01:26 -04:00
Marco Neumann a91e572b43 feat: implement basic router server type
This only implements the deployment service and nothing else yet.
2021-11-03 16:34:51 +01:00
Marco Neumann 0d0c0cb42b refactor: move write buffer configs to new home
Write buffer configs will partially be shared by database and router
nodes, so lets move them into a shared home.
2021-11-02 10:17:01 +01:00
Marco Neumann 011af2d6ba feat: add `DeploymentService`
Ref #2980.
2021-11-01 17:55:40 +01:00
kodiakhq[bot] 8c3446ac87
Merge branch 'main' into crepererum/issue2980b 2021-11-01 16:39:32 +00:00
kodiakhq[bot] c724bb2ee8
Merge branch 'main' into dependabot/cargo/tokio-1.13.0 2021-11-01 11:30:33 +00:00
dependabot[bot] c540b40f05
chore(deps): bump tokio from 1.12.0 to 1.13.0
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.12.0...tokio-1.13.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 11:21:59 +00:00
Marco Neumann bcd66c555a feat: add `RemoteService`
Ref #2980.
2021-11-01 11:38:46 +01:00
Carol (Nichols || Goulding) 20e12b911b
fix: Invalid UUID coming from an IOx server shouldn't happen; panic instead of a separate error enum 2021-10-29 09:40:23 -04:00
Marco Neumann 83e4514a43 feat: add `DeleteService`
Ref #2980.
2021-10-29 11:57:31 +02:00
Carol (Nichols || Goulding) ce55cf401c
feat: Return UUID when creating a database
Seems polite and is useful for some test setup
2021-10-28 13:20:29 -04:00
Carol (Nichols || Goulding) 4077d3a20a
fix: Take a UUID when restoring a database 2021-10-28 13:20:29 -04:00
Carol (Nichols || Goulding) 95ba84edca
fix: Return UUID when a database is deleted
So that it can be immediately used to restore elsewhere.
2021-10-28 13:20:29 -04:00
Carol (Nichols || Goulding) f160712f5e
fix: Remove the concept of generations
From iox_object_store and everywhere that fails to compile as a result.

Also make path handling in iox_object_store a bit more consistent.
2021-10-28 13:20:28 -04:00
Marco Neumann c8981e4f78 fix: fix optional dependencies for `influxdb_iox_client` 2021-10-25 10:58:20 +02:00
Marco Neumann bc7244c48e chore: use Rust edition 2021 2021-10-25 10:58:20 +02:00
Carol (Nichols || Goulding) 948cf92aaa fix: Remove the ability to list deleted databases
Once we're relying on the server config file to know about databases,
IOx shouldn't be listing files in object storage to try and find deleted
databases.

When the disown API is implemented for the floating database design, it
will return the UUID of the database just disowned, so that it can
immediately be used in an adopt API call.
2021-10-22 14:50:06 -04:00
Andrew Lamb a82dc6f5f0
chore: Update datafusion + arrow (#2903)
* chore: Update datafusion to latest, arrow to 6.0.0

* fix: Update tests

* fix: bubble internal error

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-19 17:14:08 +00:00
Raphael Taylor-Davies 074ae40382
feat: migrate entry to use bytes::Bytes (#2842)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-14 10:55:51 +00:00
dependabot[bot] 1327735537
chore(deps): bump thiserror from 1.0.29 to 1.0.30
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.29...1.0.30)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-11 08:01:21 +00:00
Marco Neumann 63d74be490 refactor: make `ChunkId` a UUID 2021-10-07 10:23:27 +02:00
Nga Tran ff8e037971 feat: handle delete response 2021-10-04 20:37:53 -04:00
dependabot[bot] 7a6f381e69
chore(deps): bump arrow-flight from 5.4.0 to 5.5.0
Bumps [arrow-flight](https://github.com/apache/arrow-rs) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0)

---
updated-dependencies:
- dependency-name: arrow-flight
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 13:27:26 +00:00
dependabot[bot] d1f5209869
chore(deps): bump arrow from 5.4.0 to 5.5.0
Bumps [arrow](https://github.com/apache/arrow-rs) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0)

---
updated-dependencies:
- dependency-name: arrow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 08:55:38 +00:00
Marco Neumann fb7299a169
fix: bubble up write errors (#2598)
Fixes #2538.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-21 11:08:37 -04:00
Carol (Nichols || Goulding) 51a40b31bf feat: Add a --detailed option to the database list CLI
That will list both active and deleted databases with their generations.

Closes #2462.
2021-09-17 15:27:23 -04:00
Carol (Nichols || Goulding) 44a89cdf75 refactor: Change DeletedDatabase to DetailedDatabase
So this info can be reused for active databases in detailed database
lists.
2021-09-17 15:27:22 -04:00
kodiakhq[bot] 23cc980d9e
Merge branch 'main' into cn/restore 2021-09-17 17:52:56 +00:00
Raphael Taylor-Davies 37b615f301
feat: migrate operations CLI to use pbjson (#2562)
* feat: migrate operations CLI to use pbjson

* fix: reserve removed field

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-16 19:15:24 +00:00
Carol (Nichols || Goulding) 7b6d8f9327 feat: Add an API for restoring a database that was marked deleted 2021-09-15 16:59:37 -04:00
Nga Tran b4f8fad400 refactor: address review comments and remove uno longer needed dependencies 2021-09-09 14:24:58 -04:00
Nga Tran 00df7b064c feat: finally have the delete predicate parsed 2021-09-08 17:30:10 -04:00
Nga Tran dbe4bcff22 chore: merge main to branch 2021-09-07 10:54:59 -04:00
Nga Tran 9ee1bdeeb9 refactor: address review comments 2021-09-07 10:24:38 -04:00
Dom dd88b5ed1d build: feature-gate arrow dep for mgmt client
Avoid pulling in & compiling arrow for consumers of the managemnt
client.

PR #2098 added arrow_util as a dependency of this crate which in turn
pulls in all of arrow, however it is only used in the flight client.

This commit feature-gates arrow_util behind "flight" with the rest of
the arrow deps.
2021-09-06 14:11:20 +01:00
dependabot[bot] b67610d9b9
chore(deps): bump tokio from 1.10.1 to 1.11.0
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.10.1 to 1.11.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.10.1...tokio-1.11.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 09:11:38 +00:00
Nga Tran 9de3b79a90 refactor: more cleanup 2021-09-06 01:45:47 -04:00
Nga Tran de0bd80c3d refactor: cleanup 2021-09-06 01:07:07 -04:00
Nga Tran 4801b2c238 feat: Have the ParseDelete message and its corresponding ProvidedParseDelete struct ready for building delete parser 2021-09-06 00:13:59 -04:00
Nga Tran d8b3208896 refactor: sstart building parser in the client side 2021-09-03 17:56:45 -04:00
Carol (Nichols || Goulding) ce6030a3cb feat: Wire list deleted databases through gRPC and CLI APIs 2021-09-02 15:48:07 -04:00
Nga Tran efcdc8e50f feat: implement delete parser 2021-09-01 17:31:49 -04:00
Nga Tran a4183de411 feat: more progress on the delete flow from grpc API to catalog chunks 2021-08-31 17:42:07 -04:00
Carol (Nichols || Goulding) fe8e551f2c feat: Add delete database to the management client 2021-08-30 10:42:46 -04:00
dependabot[bot] 3e154b3b02
chore(deps): bump serde_json from 1.0.66 to 1.0.67
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.66 to 1.0.67.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.66...v1.0.67)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-08-30 11:28:35 +00:00
dependabot[bot] 52fbaef2df
chore(deps): bump thiserror from 1.0.26 to 1.0.28
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.26 to 1.0.28.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.26...1.0.28)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-08-30 11:05:11 +00:00
Andrew Lamb 2f49e47a23
feat: return DatabaseRules for ListDatabases request (#2431) 2021-08-28 10:53:24 +00:00
Andrew Lamb 779b027271
feat: Store only the database rules sent by the client (do not store default values) (#2430)
* feat: add omit_default to protobuf definition

* feat: Persist only the client provided rules

* fix: Remove race conditions

* fix: merge confit

* refactor: do not use macro

* refactor: restore use of glob import

* fix: review comments

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-28 10:26:32 +00:00
Nga Tran 499af57299 chore: merge mian to branch and resolve conflicts 2021-08-27 17:51:07 -04:00
Marco Neumann 6625b3a768 feat: "persist partition" gRPC 2021-08-27 15:16:13 +02:00
Nga Tran bcd39e225c feat: Management API for delete 2021-08-26 17:31:21 -04:00
Raphael Taylor-Davies 1773bf5d37
feat: add storage client to influxdb_iox_client (#2404)
* feat: add storage client to influxdb_iox_client

* chore: fix type_url

* refactor: split storage into separate crate

* chore: fix doctest

* chore: review feedback

* chore: add generated_types cleanup ticket

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-26 10:30:59 +00:00
Dom 2da438de2b refactor: move re-export to crate root 2021-08-23 16:28:16 +01:00
Dom 87beefa96c refactor: re-export google proto types
The TemplatePart::Table() variant contains a google::protobuf::Empty
type which is defined in the generated_types crate.

Rather than force consumers to import two crates which have to be kept
in sync, this commit re-exports the google types within the
influxdb_iox_client crate as it's already a dependency, and makes using
types like protobuf's Duration or long-running-operation API easier for
the consumer.
2021-08-23 16:11:42 +01:00
dependabot[bot] f11ad0bf1d
chore(deps): bump serde from 1.0.127 to 1.0.128
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-08-23 01:16:38 +00:00
Andrew Lamb e62790cf4b
fix: include underlying error message in connection error (#2326)
* fix: include underlying error message in connection error

* fix: clippy

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-18 11:33:30 +00:00
Marco Neumann ceaa841e59 docs: improve `DropPartitionError::LifecycleError` description
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-08-17 09:44:35 +02:00
Marco Neumann fcf2bee443 feat: drop partition gRPC 2021-08-17 09:44:35 +02:00
Marco Neumann 42d5f9f3a1 feat: skip replay via CLI 2021-08-16 13:47:07 +02:00
Raphael Taylor-Davies 0a065b4968
feat: set header on gRPC requests (#2283)
* feat: set header on gRPC requests

* feat: always insert header value

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-13 14:13:50 +00:00
Dom 3de6b44e23
build: use new rustdoc lint name (#2261)
* fix: nocache feature code rot

The MBChunk::snapshot code when using the "nocache" option no longer
compiles - this commit updates it to match the not(nocache) code.

* build: use updated broken_intra_doc_links name

The broken_intra_doc_links lint was renamed
rustdoc::broken_intra_doc_links

https://doc.rust-lang.org/rustdoc/lints.html
2021-08-11 19:48:51 +00:00
Raphael Taylor-Davies 1f450ef371
feat: add Database abstraction (#2186) (#2203)
* feat: add Database abstraction

* chore: minor tweaks

* chore: remove redundant test fixture restart

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-08 17:14:23 +00:00
Andrew Lamb e92e94caad
chore: Update deps (including arrow 5.1.0, tonic -> 0.5, and prost 0.5) (#2172)
* chore: Update deps (including arrow 5.0.0 --> arrow 5.1.0)

* chore: update all the things

* refactor: Update serving readiness check due to change in Tonic API

* chore: update more deps

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-05 15:57:38 +00:00
Jacob Marble 98d4c9fca1
feat: switch protobuf write service to canonical definition (#2182)
* feat: switch protobuf write service to canonical definition

The protobuf definition used for the proto write endpoint was a WIP. Now
that a canonical definition exists at
https://github.com/influxdata/influxdb-pb-data-protocol/ we can switch
to that.

* chore: lint etc

* chore: fix rustdoc nit in proto definition comment
2021-08-04 00:16:49 +00:00
kodiakhq[bot] 0297aae17e
Merge branch 'main' into cn/1.54 2021-07-30 17:01:37 +00:00
Jacob Marble 8f05569007
test: add Flight/handshake test (#2156) 2021-07-30 15:43:28 +00:00
Carol (Nichols || Goulding) 9d15798288 fix: Address or allow Clippy warnings new with Rust 1.54 2021-07-30 09:59:59 -04:00
Andrew Lamb 01c79f1a1a
fix: Print all timestamps using RFC3339 format (#2098)
* fix: Use IOx pretty printer rather than arrow pretty printer

* chore: update tests in the query crate

* chore: update influxdb_iox tests

* chore: Update end to end tests

* chore: update query_tests

* chore: update mutable_buffer tests

* refactor: update parquet_file tests

* refactor: update db tests

* chore: update kafka integration test output

* fix: merge conflict
2021-07-22 19:04:52 +00:00
Andrew Lamb 4da8a16c18
chore: update to arrow 5.0 and master datafusion (#2049)
* chore: update to arrow 5.0 and master datafusion

* fix: Update test for change in object size
2021-07-19 12:49:51 +00:00
Jacob Marble b79d9eb0ab
chore: add end-to-end test for PB write service (#1894)
* chore: add end-to-end test for PB write service

* chore: lint

* chore: fix test
2021-07-14 14:20:37 +00:00
Andrew Lamb d35b74c226
fix: Fix doc build warnings (#1945)
* fix: Fix doc build warnings

* refactor: add deny bare_urls to crates

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-07-13 08:03:42 +00:00
Marco Neumann 3d008f4d27 feat: add API+CLI to unload chunks
Closes #1919.
2021-07-12 14:06:01 +02:00
Raphael Taylor-Davies 3ae8ac2467
chore: improve wait_for_chunk failure output (#1835) 2021-06-29 11:54:32 +00:00
Marco Neumann 47f8310a27 docs: clarify wipe behavior 2021-06-21 13:27:18 +02:00
Marco Neumann 8e69202270 feat: catalog wiping gRPC 2021-06-21 09:31:23 +02:00
Raphael Taylor-Davies 11b25b3aaf
refactor: swap order of partition and table in in-memory catalog (#1678)
* refactor: swap order of partition and table in in-memory catalog

* chore: review feedback

* chore: validate panic message

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-10 16:40:30 +00:00
Marco Neumann 1c5a779b83 chore: enforce `clippy::future_not_send` for `influxdb_iox_client` 2021-06-10 09:45:02 +02:00
kodiakhq[bot] b49abf9b02
Merge branch 'main' into crepererum/lazy_db_loading 2021-06-09 07:23:35 +00:00
Carol (Nichols || Goulding) f4a9a5ae56 fix: Remove write buffer 2021-06-04 14:40:17 -04:00
Marco Neumann f38a6378a6 fix: make `getServerStatus` gRPC-compliant 2021-06-04 12:00:49 +02:00
Marco Neumann e06d65bb2a refactor: migrate "DBs initialized" RPC to "server status" 2021-06-04 11:33:41 +02:00
Marco Neumann b30d7e2821 feat: move DB loading into background worker
Before this change we loaded databases eagerly when a serverID was
passed on startup BEFORE starting up the gRPC server. Since loading
(esp. at its current state without checkpoints and with too many small
parquet files) can take very long, K8s thinks IOx is unhealthy. With
this change we are now loading databases in the server background worker
once a serverID is available. Until then we block all DB-related
interactions including adding new databases (since without inspecting
the object store there is now way we can check if the DB already
exists).

Furthermore we now load database no matter if the serverID was passed on
startup (via CLI or environment variable) or was set later via gRPC
call. Before this change the latter case was somewhat forgotten.
2021-06-04 11:33:41 +02:00
Marko Mikulicic 9765c53bb4
feat: Implement service readiness API 2021-05-24 17:57:45 +02:00
Andrew Lamb 14ba25f86d
chore: Update datafusion and use released version of arrow crates (#1546)
* chore: Update datafusion and use released version of arrow crate

* fix: Update for change in API
2021-05-24 15:37:22 +00:00
Carol (Nichols || Goulding) febc1538ff
chore: Update Rust version (#1445)
* chore: Update Rust version

* refactor: Make struct constructor field orderings consistent

Sometimes I changed the struct definition, sometimes changed the struct
construction instance, depending on consistency with code around each
(other similar structs, function argument orders, etc)

More info: https://rust-lang.github.io/rust-clippy/master/index.html#inconsistent_struct_constructor

* refactor: Use flatten where appropriate

One instance is a false positive with a clippy bug.

More info:

- https://rust-lang.github.io/rust-clippy/master/index.html#filter_map_identity
- https://rust-lang.github.io/rust-clippy/master/index.html#manual_flatten

* refactor: Use Option map instead of match

More info: https://rust-lang.github.io/rust-clippy/master/index.html#manual_map

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-06 22:07:10 +00:00
Raphael Taylor-Davies 411cf134e9
refactor: explode arrow_deps (#1425)
* refactor: explode arrow_deps

* chore: workaround doctest bug
2021-05-05 16:59:12 +00:00
Marco Neumann eddc9319ff docs: deny broken intradoc links 2021-04-27 13:22:28 +02:00
Carol (Nichols || Goulding) 272cdb85ce fix: Use the ServerId type everywhere, for writing, querying, anything 2021-04-26 18:44:32 +00:00
Marco Neumann f12f1eeef7 refactor: make chunks per-table
This changes the hierarchy from

```
database -> partition -> chunk -> table
```

to

```
database -> partition -> table -> chunk
```

Only the high-level APIs are changed for now. The chunk states (like
MutableBuffer and ReadBuffer) still multiplex tables, although they will
always only get a single table assigned (or no table if no data was
presented yet).

Closes #1256.
2021-04-23 09:18:55 +02:00
Marko Mikulicic 83d6550316 feat: Implement write_entry_downstream 2021-04-21 20:50:46 +00:00
Carol (Nichols || Goulding) 04a360fa75 fix: Make names more idiomatic, even though they're acronyms 2021-04-19 08:45:47 -04:00
Marko Mikulicic e76980928b feat: Implement Update API 2021-04-08 22:25:36 +00:00
Nga Tran ce8200e8b0 chore: remove unused code 2021-04-08 10:02:28 -04:00
Nga Tran c32fe3decd chore: clean up 2021-04-07 21:43:45 -04:00
Nga Tran f501a74aea refactor: Address review comments 2021-04-07 21:28:03 -04:00
Nga Tran be6e1e48e4 feat: add writer_id and object_store in Db 2021-04-07 18:36:07 -04:00
Andrew Lamb 3034b7e76d docs: Improve influxdb_iox_client module descriptions 2021-04-05 09:33:52 -04:00
Nga Tran a630c119ab feat: make it easy to get OperationMetadata from Operation 2021-03-30 12:57:11 -04:00
Andrew Lamb 44d67db733
feat: Rework Db to use Catalog for chunk state (#1043)
* feat: Rework Db to use Catalog for chunk state

* docs: Update server/src/db.rs

* fix: fmt

* fix: fmt

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-24 17:57:11 +00:00
Andrew Lamb 3a53923684
feat: Management API + CLI command to close a chunk and move to read buffer (#1002)
* feat: Management API + CLI command to close a  chunk and move to read buffer

* refactor: Less copy-pasta

* fix: track only once, use `let _` instead of `.ok()`

* docs: Apply suggestions from code review

fix comments ( 🤦‍♀️  for copy/pasta)

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* docs: Update server/src/lib.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* refactor: Use DatabaseName rather than impl Into<String>

* fix: Fixup logical merge conflicts

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-03-17 15:25:27 +00:00
Andrew Lamb ef9c3f3d8f
feat: Management API + CLI command to list chunks in a partition (#995)
* feat: Management API + CLI command to list chunks in a partition

For ease of use

* refactor: remove unecessary Result
2021-03-16 20:10:55 +00:00
Raphael Taylor-Davies 3fe1b8c5b7
feat: add longrunning operations client (#981)
refactor: add separate format feature influxdb_iox_client

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-16 13:19:44 +00:00
Andrew Lamb 4781317647
feat: Management API + CLI command to create a new chunk (rollover partition) (#990)
* feat: Management API + CLI command to create a new chunk (rollover partition)

* fix: Update tests/end_to_end_cases/management_api.rs

fix typo

* fix: logical merge conflict

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-15 16:41:18 +00:00
Raphael Taylor-Davies 65f7a1ac5b
fix: use consistent crate versions (#989)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-15 15:42:19 +00:00
Andrew Lamb 1b36d6b0cd fix: Return Partition rather than strings 2021-03-12 20:33:39 +00:00
Andrew Lamb cd66814c3d feat: Add management API and CLI for listing partitions 2021-03-12 20:33:39 +00:00
Andrew Lamb 6ac7e2c1a7
feat: Add management API and CLI to list chunks (#968)
* feat: Add management API and CLI to list chunks

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: add comment to protobuf

* fix: fix comment

* fix: fmt, fixup merge errors

* fix: fascinating type dance with prost generated types

* fix: clippy

* fix: move command to influxdb_iox database chunk list

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-03-12 13:56:14 +00:00
Marko Mikulicic 1b40ca7ab5 feat: Implement CRUD for remote addresses 2021-03-10 13:39:59 +00:00
Andrew Lamb 9ebedf744f
feat: Add CLI to query the database using SQL (#943)
* feat: Add CLI to query the database using SQL

* fix: merge conflicts
2021-03-10 13:21:12 +00:00
Andrew Lamb 500c237f62
feat: Add Service + CLI command to write line protocol to IOx (#939)
* feat: Add Service + CLI command to write line protocol to IOx

* fix: clippy

* fix: protobuf lint

* fix: LinePrtocol typo
2021-03-08 17:35:19 +00:00
Raphael Taylor-Davies 8f3b9b7389
fix: build of influxdb_iox_client with flight disabled (#925) 2021-03-04 12:10:18 +00:00
Raphael Taylor-Davies 51981c92f5
feat: implement gRPC API and migrate influxdb_iox_client to use it (#853)
* feat: implement gRPC management API

* feat: migrate influxdb_iox_client to use gRPC API

* fix: review comments

* refactor: separate influxdb_iox_client error types

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-02 17:51:46 +00:00
Raphael Taylor-Davies ffc20fa821
feat: add basic gRPC health service (#862)
* feat: add basic gRPC health service

* feat: update README.md add /health to HTTP API

* feat: add health client to influxdb_iox_client

feat: end-to-end test health check service

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-02-25 13:24:12 +00:00
Dom 4e6841bc5e refactor: derive Clone for API client 2021-02-17 16:53:00 +00:00
Dom 1a67b73ace feat: get writer ID endpoint
Adds an API endpoint to retrieve the currently set server writer ID:

	GET /iox/api/v1/id

Returning:

	{"id":42}

Also adds a get_writer_id() method to the IOx API client.
2021-02-17 16:03:47 +00:00
Edd Robinson dfe6e50e26 refactor: add arc clone lint 2021-02-15 12:52:12 +00:00
Marko Mikulicic 9e39e91139 chore: Cleaning things in prep for rust 2021
Also remove a NUL byte in a test string literal; some editors drop them.
2021-02-12 16:48:17 +00:00
kodiakhq[bot] a242fc39aa
Merge branch 'main' into jg/flight-client 2021-02-11 14:29:54 +00:00
Paul Dix 4c43c311f0
feat: Add database rules configuration for mutable buffer (#779)
* feat: Add database rules configuration for mutable buffer

* refactor: change all database rules usage to use mutable_buffer_config rather than store_locally
2021-02-10 18:36:29 -05:00
Carol (Nichols || Goulding) 523e361118 test: Fix another doc test that wasn't quite working as intended 2021-02-10 16:30:39 -05:00
Carol (Nichols || Goulding) b0c7aa7ea1 test: Fix doc test that wasn't getting compiled and was broken 2021-02-10 16:17:54 -05:00
Marko Mikulicic e4baf12c06
feat: Implement database list API in the client.
Closes #767
2021-02-10 17:59:19 +01:00
Carol (Nichols || Goulding) afdb0c8274 fix: Return better errors instead of unwrapping 2021-02-10 10:30:58 -05:00
Carol (Nichols || Goulding) 261a05b55b docs: Add and improve more documentation for public items 2021-02-10 10:30:58 -05:00
Carol (Nichols || Goulding) 82dfb375f3 fix: Adjust imports and visibility to improve the public API 2021-02-10 10:30:56 -05:00
Carol (Nichols || Goulding) a8fedb2eb5 docs: Create a README that explains the two clients and the feature 2021-02-10 10:30:06 -05:00
Jake Goulding 699f4a577f feat: Add an optional Flight client to the IOx client library 2021-02-10 10:30:05 -05:00