b49abf9b02
Merge branch 'main' into crepererum/lazy_db_loading
2021-06-09 07:23:35 +00:00
f4a9a5ae56
fix: Remove write buffer
2021-06-04 14:40:17 -04:00
f38a6378a6
fix: make `getServerStatus` gRPC-compliant
2021-06-04 12:00:49 +02:00
e06d65bb2a
refactor: migrate "DBs initialized" RPC to "server status"
2021-06-04 11:33:41 +02:00
b30d7e2821
feat: move DB loading into background worker
...
Before this change we loaded databases eagerly when a serverID was
passed on startup BEFORE starting up the gRPC server. Since loading
(esp. at its current state without checkpoints and with too many small
parquet files) can take very long, K8s thinks IOx is unhealthy. With
this change we are now loading databases in the server background worker
once a serverID is available. Until then we block all DB-related
interactions including adding new databases (since without inspecting
the object store there is now way we can check if the DB already
exists).
Furthermore we now load database no matter if the serverID was passed on
startup (via CLI or environment variable) or was set later via gRPC
call. Before this change the latter case was somewhat forgotten.
2021-06-04 11:33:41 +02:00
9765c53bb4
feat: Implement service readiness API
2021-05-24 17:57:45 +02:00
14ba25f86d
chore: Update datafusion and use released version of arrow crates ( #1546 )
...
* chore: Update datafusion and use released version of arrow crate
* fix: Update for change in API
2021-05-24 15:37:22 +00:00
febc1538ff
chore: Update Rust version ( #1445 )
...
* chore: Update Rust version
* refactor: Make struct constructor field orderings consistent
Sometimes I changed the struct definition, sometimes changed the struct
construction instance, depending on consistency with code around each
(other similar structs, function argument orders, etc)
More info: https://rust-lang.github.io/rust-clippy/master/index.html#inconsistent_struct_constructor
* refactor: Use flatten where appropriate
One instance is a false positive with a clippy bug.
More info:
- https://rust-lang.github.io/rust-clippy/master/index.html#filter_map_identity
- https://rust-lang.github.io/rust-clippy/master/index.html#manual_flatten
* refactor: Use Option map instead of match
More info: https://rust-lang.github.io/rust-clippy/master/index.html#manual_map
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-06 22:07:10 +00:00
411cf134e9
refactor: explode arrow_deps ( #1425 )
...
* refactor: explode arrow_deps
* chore: workaround doctest bug
2021-05-05 16:59:12 +00:00
eddc9319ff
docs: deny broken intradoc links
2021-04-27 13:22:28 +02:00
272cdb85ce
fix: Use the ServerId type everywhere, for writing, querying, anything
2021-04-26 18:44:32 +00:00
f12f1eeef7
refactor: make chunks per-table
...
This changes the hierarchy from
```
database -> partition -> chunk -> table
```
to
```
database -> partition -> table -> chunk
```
Only the high-level APIs are changed for now. The chunk states (like
MutableBuffer and ReadBuffer) still multiplex tables, although they will
always only get a single table assigned (or no table if no data was
presented yet).
Closes #1256 .
2021-04-23 09:18:55 +02:00
83d6550316
feat: Implement write_entry_downstream
2021-04-21 20:50:46 +00:00
04a360fa75
fix: Make names more idiomatic, even though they're acronyms
2021-04-19 08:45:47 -04:00
e76980928b
feat: Implement Update API
2021-04-08 22:25:36 +00:00
ce8200e8b0
chore: remove unused code
2021-04-08 10:02:28 -04:00
c32fe3decd
chore: clean up
2021-04-07 21:43:45 -04:00
f501a74aea
refactor: Address review comments
2021-04-07 21:28:03 -04:00
be6e1e48e4
feat: add writer_id and object_store in Db
2021-04-07 18:36:07 -04:00
3034b7e76d
docs: Improve influxdb_iox_client module descriptions
2021-04-05 09:33:52 -04:00
a630c119ab
feat: make it easy to get OperationMetadata from Operation
2021-03-30 12:57:11 -04:00
44d67db733
feat: Rework Db to use Catalog for chunk state ( #1043 )
...
* feat: Rework Db to use Catalog for chunk state
* docs: Update server/src/db.rs
* fix: fmt
* fix: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-24 17:57:11 +00:00
3a53923684
feat: Management API + CLI command to close a chunk and move to read buffer ( #1002 )
...
* feat: Management API + CLI command to close a chunk and move to read buffer
* refactor: Less copy-pasta
* fix: track only once, use `let _` instead of `.ok()`
* docs: Apply suggestions from code review
fix comments ( 🤦♀️ for copy/pasta)
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* docs: Update server/src/lib.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* refactor: Use DatabaseName rather than impl Into<String>
* fix: Fixup logical merge conflicts
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-03-17 15:25:27 +00:00
ef9c3f3d8f
feat: Management API + CLI command to list chunks in a partition ( #995 )
...
* feat: Management API + CLI command to list chunks in a partition
For ease of use
* refactor: remove unecessary Result
2021-03-16 20:10:55 +00:00
3fe1b8c5b7
feat: add longrunning operations client ( #981 )
...
refactor: add separate format feature influxdb_iox_client
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-16 13:19:44 +00:00
4781317647
feat: Management API + CLI command to create a new chunk (rollover partition) ( #990 )
...
* feat: Management API + CLI command to create a new chunk (rollover partition)
* fix: Update tests/end_to_end_cases/management_api.rs
fix typo
* fix: logical merge conflict
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-15 16:41:18 +00:00
65f7a1ac5b
fix: use consistent crate versions ( #989 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-15 15:42:19 +00:00
1b36d6b0cd
fix: Return Partition rather than strings
2021-03-12 20:33:39 +00:00
cd66814c3d
feat: Add management API and CLI for listing partitions
2021-03-12 20:33:39 +00:00
6ac7e2c1a7
feat: Add management API and CLI to list chunks ( #968 )
...
* feat: Add management API and CLI to list chunks
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: add comment to protobuf
* fix: fix comment
* fix: fmt, fixup merge errors
* fix: fascinating type dance with prost generated types
* fix: clippy
* fix: move command to influxdb_iox database chunk list
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-03-12 13:56:14 +00:00
1b40ca7ab5
feat: Implement CRUD for remote addresses
2021-03-10 13:39:59 +00:00
9ebedf744f
feat: Add CLI to query the database using SQL ( #943 )
...
* feat: Add CLI to query the database using SQL
* fix: merge conflicts
2021-03-10 13:21:12 +00:00
500c237f62
feat: Add Service + CLI command to write line protocol to IOx ( #939 )
...
* feat: Add Service + CLI command to write line protocol to IOx
* fix: clippy
* fix: protobuf lint
* fix: LinePrtocol typo
2021-03-08 17:35:19 +00:00
8f3b9b7389
fix: build of influxdb_iox_client with flight disabled ( #925 )
2021-03-04 12:10:18 +00:00
51981c92f5
feat: implement gRPC API and migrate influxdb_iox_client to use it ( #853 )
...
* feat: implement gRPC management API
* feat: migrate influxdb_iox_client to use gRPC API
* fix: review comments
* refactor: separate influxdb_iox_client error types
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-02 17:51:46 +00:00
ffc20fa821
feat: add basic gRPC health service ( #862 )
...
* feat: add basic gRPC health service
* feat: update README.md add /health to HTTP API
* feat: add health client to influxdb_iox_client
feat: end-to-end test health check service
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-02-25 13:24:12 +00:00
4e6841bc5e
refactor: derive Clone for API client
2021-02-17 16:53:00 +00:00
1a67b73ace
feat: get writer ID endpoint
...
Adds an API endpoint to retrieve the currently set server writer ID:
GET /iox/api/v1/id
Returning:
{"id":42}
Also adds a get_writer_id() method to the IOx API client.
2021-02-17 16:03:47 +00:00
dfe6e50e26
refactor: add arc clone lint
2021-02-15 12:52:12 +00:00
9e39e91139
chore: Cleaning things in prep for rust 2021
...
Also remove a NUL byte in a test string literal; some editors drop them.
2021-02-12 16:48:17 +00:00
a242fc39aa
Merge branch 'main' into jg/flight-client
2021-02-11 14:29:54 +00:00
4c43c311f0
feat: Add database rules configuration for mutable buffer ( #779 )
...
* feat: Add database rules configuration for mutable buffer
* refactor: change all database rules usage to use mutable_buffer_config rather than store_locally
2021-02-10 18:36:29 -05:00
523e361118
test: Fix another doc test that wasn't quite working as intended
2021-02-10 16:30:39 -05:00
b0c7aa7ea1
test: Fix doc test that wasn't getting compiled and was broken
2021-02-10 16:17:54 -05:00
e4baf12c06
feat: Implement database list API in the client.
...
Closes #767
2021-02-10 17:59:19 +01:00
afdb0c8274
fix: Return better errors instead of unwrapping
2021-02-10 10:30:58 -05:00
261a05b55b
docs: Add and improve more documentation for public items
2021-02-10 10:30:58 -05:00
82dfb375f3
fix: Adjust imports and visibility to improve the public API
2021-02-10 10:30:56 -05:00
a8fedb2eb5
docs: Create a README that explains the two clients and the feature
2021-02-10 10:30:06 -05:00
699f4a577f
feat: Add an optional Flight client to the IOx client library
2021-02-10 10:30:05 -05:00
kodiakhq[bot]
Carol (Nichols || Goulding)
Marco Neumann
Marko Mikulicic
Andrew Lamb
Carol (Nichols || Goulding)
Raphael Taylor-Davies
Nga Tran
Andrew Lamb
Dom
Edd Robinson
Paul Dix
Jake Goulding