d8810074e8
refactor: better test action CLI
2021-12-16 09:29:25 +01:00
f4fde15810
refactor: nicer code
...
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-12-16 09:29:25 +01:00
ed775431b6
fix: treat early server worker exit as proper error
...
Instead of just logging the issue, also make sure the error gets
propagated all the way up to the exit code.
Fixes #3375 .
2021-12-16 09:29:25 +01:00
da0330fc5f
fix: print newline at end of `database list --detailed` ( #3382 )
2021-12-15 19:33:24 +00:00
758b65dd29
feat: Add database initialization state and errors to CLI and remove list_databases_detailed gRPC ( #3377 )
...
* feat: Add database initialization state and errors to CLI:
* fix: do not use optional in protobuf
* fix: clippy
* fix: correct check I broke appeasing clippy
2021-12-15 12:18:41 +00:00
7fe6897c59
refactor: add support for handling `TagValuesGroupedByMeasurementAndTagKeyRequest` ( #3373 )
...
* chore: update Storage service
* refactor: handle rpc
* refactor: log request fields
* refactor: add new RPC to client
* test: basic test of RPC
* refactor: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-14 18:42:12 +00:00
ec6d945cbe
feat: wire up general predicates to measurement_names
2021-12-13 14:01:09 +00:00
c0ba69f09e
chore: marge main to branch and resolve conflict
2021-12-09 15:40:33 -05:00
35370922f3
refactor: make a setup for 2 persisted chunks that can be used in for different places
2021-12-09 15:21:56 -05:00
3cda6b6c0f
refactor: Remove collect_query and replication ( #3348 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-09 19:58:19 +00:00
099e2d4056
chore: Apply suggestions from code review
...
Co-authored-by: Marco Neumann <marco@crepererum.net>
2021-12-09 13:52:42 -05:00
e46708354e
test: add management cli tests
2021-12-09 12:53:45 -05:00
365917c2aa
fix: Organize imports
2021-12-09 08:49:34 -05:00
471c3181bb
feat: Make a top-level influxdb_iox feature for kafka functionality
...
Thread the feature through router and server to the write buffer crate.
Move an end-to-end test that uses Kafka behind the feature flag.
2021-12-09 08:49:34 -05:00
38ce54c1be
Merge branch 'main' into crepererum/issue3336
2021-12-09 09:44:55 +00:00
efbfbb1a0b
feat: compact all object store chunks of a given partition
2021-12-08 16:06:03 -05:00
876a9af35a
fix: limit number of S3 connections
...
Otherwise the whole thing blows up when starting a server that has many
DBs registerd, because we potentially create 1 connection per DB (e.g.
to read out the preserved catalog).
Fixes #3336 .
2021-12-08 19:06:02 +01:00
a8bc5b3b6d
fix: allow setting the same server ID twice
...
This is important for idempotence and simplifies clients and helper
scripts a lot.
2021-12-08 19:03:30 +01:00
35edee6b4f
fix: flaky test migrate_table_files_from_one_server_to_another ( #3339 )
2021-12-08 12:08:25 -05:00
218042784f
feat: add `system.queries `system table ( #3328 )
...
* feat: Add query log and system table
* docs: Add docstrings for test normalization
2021-12-08 16:26:24 +00:00
c6a3765d76
feat: Add force flag to RebuildCatalog ( #3292 )
...
* feat: Add force flag to RebuildCatalog
* fix: small cleanups
* docs: Update comments and add WARNING
2021-12-08 15:36:07 +00:00
3e97e49c35
refactor: "update router" -> "create or update router"
2021-12-08 13:45:55 +01:00
e42c6974cf
fix: remove unused code
2021-12-08 13:45:55 +01:00
af71e2422e
fix: typos
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-12-08 13:45:55 +01:00
b7d6865f87
feat: add router CLI
...
Closes #3331 .
2021-12-08 13:45:55 +01:00
c17a6c10c1
feat: add `GetRouter` gRPC method
2021-12-08 13:45:55 +01:00
577df009d8
fix: test flakyness while waiting for gRPC connection
...
Fixes #3312 .
2021-12-07 17:05:06 +01:00
75cd1d24f4
Merge branch 'main' into ntran/grpc_compact_os_chunks
2021-12-07 03:26:06 +00:00
7073691f04
test: grpc test for compaction os chunks
2021-12-06 22:25:33 -05:00
5316037ffe
fix: fix flay migrate_database_files_from_one_server_to_another and prepare for reuse + fix ( #3314 )
...
* fix: fix flay migrate_database_files_from_one_server_to_another and prepare for reuse + fix
* docs: clarify comments
2021-12-06 18:28:08 +00:00
3834435620
chore: Merge branch 'ntran/grpc_compact_os_chunks' of https://github.com/influxdata/influxdb_iox into ntran/grpc_compact_os_chunks
2021-12-06 12:06:22 -05:00
d8ed8a3bf7
refactor: address review comments
2021-12-06 12:06:07 -05:00
0b7c7764c6
fix: Use a consistent version range for tokio-stream across all crates
2021-12-06 09:37:16 -05:00
16d8ae5e04
fix: Match tokio features to what's actually in use in each crate
...
Some crates listed features they don't use; other crates ware relying on
feature flags enabled by something else. I tested these changes by
disabling the workspace hack crate and testing each crate.
2021-12-06 09:37:16 -05:00
02c297e850
fix: Always specify the parking_lot feature of tokio to get potential perf boost
2021-12-06 09:37:15 -05:00
e8d9e35321
fix: influxdb_iox tests need the json feature of reqwest
...
I think this didn't fail before because influxdb_iox used to use
influxdb2_client, which turned on the json feature. Then the workspace
hack crate also turned on the json feature, so when influxdb_iox stopped
using influxdb2_client, it *still* worked. But excluding the object
store crates from hakari analysis that also turned on the json feature
meant nothing turned it on anymore, so now influxdb_iox needs to turn it
on itself.
2021-12-06 09:37:15 -05:00
30c4da7ca7
fix: Be consistent with regex version range specification
2021-12-06 09:37:15 -05:00
8ad2b5e9fd
fix: In end-to-end tests, use influxdb_iox_client instead of influxdb2_client
2021-12-06 09:37:14 -05:00
3900324d33
fix: Organize imports
2021-12-06 09:37:13 -05:00
39862e3886
fix: Disable default chrono features
...
Connects to #3117 . Won't affect the workspace-hack crate until changes
to some other crates can be upstreamed.
2021-12-06 09:36:49 -05:00
994938b2f0
fix: Alphabetize/organize deps
2021-12-06 09:36:30 -05:00
174a07963a
fix: Remove a dev-dependency already listed under regular dependencies
2021-12-06 09:31:44 -05:00
1099f5c417
refactor: break out database migration end to end tests ( #3308 )
2021-12-06 12:21:10 +00:00
0b87c1e170
chore(deps): bump clap from 2.33.3 to 2.34.0
...
Bumps [clap](https://github.com/clap-rs/clap ) from 2.33.3 to 2.34.0.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/commits )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-12-06 09:07:47 +00:00
65660dace5
Merge branch 'main' into ntran/grpc_compact_os_chunks
2021-12-03 18:16:42 -05:00
5f3706e0ee
feat: grpc call for copact object store chunks
2021-12-03 18:01:28 -05:00
7dea2ef362
feat: Add force flag to ClaimDatabase ( #3284 )
...
* feat: Add force flag to ClaimDatabase
* fix: Update name of test
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* docs: Update influxdb_iox/tests/end_to_end_cases/management_cli.rs
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-12-03 21:11:23 +00:00
f17b670fe9
refactor: cleanup gRPC client error handling ( #3251 ) ( #3277 )
...
* refactor: cleanup gRPC client error handling (#3251 )
* chore: review feedback
* chore: review feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-02 20:56:58 +00:00
2857b6a990
Merge branch 'main' into er/feat/load_chunk_cli
2021-12-02 20:20:56 +00:00
b4ea9887ba
refactor: error name
2021-12-02 20:14:02 +00:00
7d4693ed8c
fix: Box contents of an enum to make the variants close in size
...
Identified by clippy
https://rust-lang.github.io/rust-clippy/master/index.html#large_enum_variant
2021-12-02 11:52:02 -05:00
1a899b939e
fix: Remove redundant closures identified by clippy
...
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_closure
2021-12-02 11:52:02 -05:00
5d0fd1c603
fix: Allow dead code on fields that are now detected as never read
2021-12-02 11:52:01 -05:00
15f9918eda
test: add end to end test
2021-12-02 12:59:52 +00:00
4153311ff5
feat: implement loading chunk CLI
2021-12-02 12:59:52 +00:00
9e45ce4121
feat: cleanup gRPC server errors ( #3251 ) ( #3265 )
...
* feat: add precondition violation enumeration (#3251 )
* chore: review feedback
2021-12-01 13:33:17 +00:00
f1936f0b08
refactor: add resource type enumeration ( #3251 ) ( #3260 )
...
* refactor: add resource type enumeration (#3251 )
* chore: more error details
2021-12-01 10:26:12 +00:00
05064eba62
fix: don't error on delete to non-existent table ( #3259 ) ( #3263 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-01 00:40:05 +00:00
7a02830bf7
fix: GIT version in `--version`
...
The GIT version was always "UNKNOWN", a regression introduced by #2965 .
Also adds a regression test.
2021-11-30 16:06:27 +01:00
c161d1176a
refactor: cleanup Db write path error handling ( #3258 )
2021-11-30 15:05:02 +00:00
1e515a1dec
feat: load RUB from object store ( #3224 ) ( #3250 )
2021-11-30 14:39:52 +00:00
88acf3788e
feat: rebuild catalog ( #3225 ) ( #3253 )
...
* feat: rebuild catalog (#3225 )
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-30 12:17:27 +00:00
f165faf5bc
feat: support forcing persistence of partition irrespective of arrival time ( #3249 )
...
* feat: force persistence of partition irrespective of arrival time
* feat: expose to CLI
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 22:07:35 +00:00
837e029ed8
refactor: cleanup database recovery ( #3225 ) ( #3245 )
...
* refactor: cleanup database recovery (#3225 )
* chore: fix errors
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 18:49:46 +00:00
4e043ecb55
refactor: remove old routing / sharding config
...
This is superseded by the new router subsystem.
2021-11-29 12:33:48 +01:00
5d784ba866
chore(deps): bump serde_json from 1.0.71 to 1.0.72
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:28:07 +00:00
b46d07b8a5
chore(deps): bump predicates from 2.0.3 to 2.1.0
...
Bumps [predicates](https://github.com/assert-rs/predicates-rs ) from 2.0.3 to 2.1.0.
- [Release notes](https://github.com/assert-rs/predicates-rs/releases )
- [Changelog](https://github.com/assert-rs/predicates-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/assert-rs/predicates-rs/compare/v2.0.3...v2.1.0 )
---
updated-dependencies:
- dependency-name: predicates
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 08:40:45 +00:00
ab4f8cdbf2
test: add end to end tests for Storage/Offsets gRPC call ( #3232 )
2021-11-27 12:01:33 +00:00
8fe2c64f5a
fix: Implement a dummy Storage.Offsets method
...
Closes https://github.com/influxdata/conductor/issues/766
2021-11-26 16:39:07 +01:00
7f2e4f4342
refactor: remove write buffer direction
...
The direction was required when a database could read or write from/to a
write buffer. Now it is clear from the usage context of a write buffer
context which of the two applications is meant (databases read, routers
write) so the direction flag is no longer required.
2021-11-26 12:38:40 +01:00
ac20fecf6f
refactor: remove write buffer producer logic from `Db`
...
As a side effect, writing to database no longer requires `async`.
Closes #2243 .
2021-11-26 12:02:27 +01:00
589b7ad3af
fix: typo
...
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-11-26 11:40:41 +01:00
e5bfa08910
feat: disallow unsequenced deletes when reading from write buffer
...
Similar to how writes to a databases that reads from a write buffer are
forbidden deletes should be rejected as well.
2021-11-26 11:40:08 +01:00
22936abb23
refactor: remove connection manager and resolver from `Server`
2021-11-26 11:00:19 +01:00
0e06026fbd
refactor: remove write routing logic from `Server`
...
API users should use `Database` instead, there's no need any longer to mirror
this API in `Server`. Note that `Database` is better than `Db` in this
case, because the former can also check if we're writing from a write
buffer and can easily reject unsequenced writes. The `pub` modifiers
were adjusted to make it impossible to write through `Db` directly.
2021-11-26 09:26:43 +01:00
c2a8baf824
refactor: clean up write buffer end2end tests
...
Remove the ones that are covered by `write_pb.rs` or that are obsolete
(i.e. rely on behavior that only exist in the legacy router code).
Migrate the remaining ones to use an actual router.
2021-11-25 14:18:37 +01:00
6e1f86ca48
Merge branch 'main' into crepererum/remove_routing_from_database_mode_1
2021-11-24 14:15:36 +00:00
9bece6808d
Merge branch 'main' into crepererum/issue3154_b
2021-11-24 13:29:45 +00:00
eca7798308
refactor: update influxdb_iox/src/influxdb_ioxd/server_type/database/rpc/storage/service.rs
2021-11-24 10:47:33 +00:00
3eb5d42fe7
fix: address review comments
2021-11-24 11:37:45 +01:00
311375bdac
refactor: add debug logging for meta queries
2021-11-24 10:31:09 +00:00
539c08125b
feat: HTTP delete support for router
...
Closes #3154 .
2021-11-24 11:13:56 +01:00
d3b1557dbc
refactor: isolate and parametrize some management CLI tests
2021-11-24 11:12:22 +01:00
cae8b8572d
refactor: remove remote API from database node
...
We can easily bring it back in case we need it, but at the moment its
not required for database nodes to manage remotes.
2021-11-24 11:12:22 +01:00
ed54100e60
refactor: convert `write_api` end2end tests to use new router mode
2021-11-24 11:12:22 +01:00
aa636eaed9
refactor: remove methods from `ManagementService` that were moved to `RemoteService`
...
Tests are covered by `remote_api.rs`.
Ref #2980 .
2021-11-24 10:01:32 +01:00
5ecaa6d8a9
refactor: use serialized predicates instead of strings for gRPC deletes
...
IOx is the only consumer of this API so we might just use the serialized
form. Cloud2 uses the HTTP API which supports SQL-like predicates.
Fixes #3192 .
2021-11-24 09:30:32 +01:00
53139edef9
Merge branch 'main' into crepererum/issue2980h
2021-11-24 08:21:07 +00:00
976bccfafe
Merge branch 'main' into er/feat/grpc_binary_tags
2021-11-23 22:32:36 +00:00
42e3a41825
fix: flaky test_chunk_access_time ( #3197 ) ( #3200 )
...
* fix: flaky test_chunk_access_time (#3197 )
* chore: fix lint
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-23 17:39:57 +00:00
dc6b44818d
refactor: remove methods from `ManagementService` that were moved to `DeploymentService`
...
Tests are covered by `deployment_api.rs`.
Ref #2980 .
2021-11-23 18:04:41 +01:00
3a7cb119c6
feat: add support for binary tag key format
2021-11-23 17:00:28 +00:00
dde1db40d1
refactor: tidy up db_name move
2021-11-23 17:00:28 +00:00
8784ae0d93
refactor: move entire request message into impl
2021-11-23 17:00:28 +00:00
c9a5726d7b
Merge branch 'main' into er/refactor/grpc_tag_order
2021-11-23 16:39:32 +00:00
2ef3ab8544
refactor: remove methods from `ManagementService` that were moved to `DeleteService`
...
Tests are covered by `delete_api.rs`.
Ref #2980 .
2021-11-23 16:40:00 +01:00
5b200ee564
feat: impl delete gRPC interface for router
2021-11-23 15:39:53 +01:00
df19d67532
test: allow to specify `WriteBufferConfigFactory` when creating a test `RouterServer`
2021-11-23 15:39:53 +01:00
65dc0bd4b3
test: update tests
2021-11-23 14:15:06 +00:00
521a8e3636
test: fix tests
2021-11-22 21:40:41 +00:00
d16a7759ca
Merge branch 'main' into cn/workspace-hack
2021-11-22 17:05:31 +00:00
5c6deb81db
chore(deps): bump serde_json from 1.0.70 to 1.0.71
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.70 to 1.0.71.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.70...v1.0.71 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-22 01:21:06 +00:00
9fd4a560f5
feat: Results of running cargo hakari manage-deps
2021-11-19 09:21:57 -05:00
25d55cd08a
feat: Move server config paths beneath 'nodes' ( #3144 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-19 09:54:32 +00:00
a2454b542d
fix: Small cleanups in Cargo.tomls ( #3160 )
...
* fix: Add tokio rt-multi-thread feature so cargo test -p client_util compiles
* fix: Alphabetize dependencies
* fix: Add the data_types_conversions feature to get tests passing
* fix: Remove dev dependencies already listed under normal dependencies
* fix: Make sure the workspace is using the new resolver
2021-11-18 22:26:33 +00:00
f69d37e9a8
fix: Remove database delete/restore entirely
2021-11-17 12:03:11 -05:00
7783e4a7ff
fix: Make delete/restore aliases for release/claim; remove tombstone
...
Fixes #2680
2021-11-17 11:41:08 -05:00
8155747735
feat: add write buffer delete encoding ( #2731 ) ( #3127 )
...
* feat: add write buffer delete encoding (#2731 )
* chore: fix doc
* chore: review feedback
* chore: review feedback
* chore: fmt
* chore: review feedback
2021-11-17 16:12:19 +00:00
d6c6e9a6c7
fix: Default kafka timeout to be shorter than gRPC timeout (60 sec --> 10 sec) ( #3131 )
...
* fix: Default kafka timeout to be shorter than gRPC timeout
* docs: fix link style
2021-11-17 12:19:53 +00:00
38ca9e1339
fix: capture all panic messages in logs ( #3130 )
2021-11-16 21:59:05 +00:00
553e412226
refactor: DMLOperation write path ( #2731 ) ( #3121 )
...
* refactor: DMLOperation write path (#2731 )
* chore: fmt
* chore: review feedback
2021-11-16 12:42:19 +00:00
bc11244828
feat: Rename database disown/adopt to release/claim ( #3111 )
...
* fix: Rename 'disown' to 'release' database
Connects to #3110
* fix: Rename 'adopt' to 'claim' database
Fixes #3110 .
2021-11-15 20:28:09 +00:00
d759d98612
fix: Update new code with API that changed since branching from main
2021-11-15 14:32:50 -05:00
cc693a780e
Merge branch 'main' into cn+jpg/adopt
2021-11-15 19:22:07 +00:00
3545f6d65a
fix: Pass through error for already-owned database
2021-11-15 14:15:56 -05:00
1d68980e4f
fix: typo
...
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-11-15 18:37:31 +01:00
c88930a6a5
refactor: clarify `ServerType` background worker handling
...
Ref #3020 .
2021-11-15 18:28:32 +01:00
3cd7d2eda2
refactor: improve usability of proto conversion traits ( #3109 )
...
* refactor: improve usability of proto conversion traits
* chore: review feedback
2021-11-15 16:10:29 +00:00
af28cfa2a6
feat: Add an adopt database API
...
Fixes #2679 .
2021-11-15 09:26:06 -05:00
236edc0e7e
Merge branch 'main' into crepererum/improve_router_client_errors
2021-11-15 11:36:57 +00:00
9868430c12
Merge branch 'main' into crepererum/fix_server_id_on_router_startup
2021-11-15 10:29:21 +00:00
3d091208af
refactor: move delete predicate into data_types ( #2731 ) ( #3094 )
...
* refactor: move delete predicate into dml (#2731 )
* refactor: move DeletePredicate to data_types
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-15 10:28:58 +00:00
a6d83a3026
feat: WriteBufferReader use DmlOperation ( #2731 ) ( #3096 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-15 10:19:54 +00:00
48d0388ea0
feat: improve `RouterClient` errors
2021-11-15 11:18:46 +01:00
3485d70409
chore(deps): bump serde_json from 1.0.69 to 1.0.70
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.69 to 1.0.70.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.69...v1.0.70 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-15 08:48:44 +00:00
641ee156ce
Merge branch 'main' into jpg/unused-deps
2021-11-15 08:25:04 +00:00
d5d07b6eeb
Merge branch 'main' into crepererum/issue3075b
2021-11-15 08:17:16 +00:00
d20b7ca390
fix: Remove unused dependencies
...
Thanks, `cargo udeps`!
2021-11-12 16:42:02 -05:00
376471591f
Merge branch 'main' into cn/disown
2021-11-12 19:31:03 +00:00
8bdfb253ce
fix: Remove context from disown operation
2021-11-12 11:10:20 -05:00
815347b18a
fix: Disown in the disown tests, not delete
2021-11-12 09:47:25 -05:00
3220970fea
fix: server IDs for routers on startup
2021-11-12 11:31:52 +01:00
f2c90d9259
tests: allow creation of test servers w/ server IDs set on startup
2021-11-12 11:31:52 +01:00
c1827cf2c3
refactor: `s/writer_id/server_id/g`
2021-11-12 11:31:52 +01:00
7f021f556d
tests: improve `TestConfig` to better support server types
2021-11-12 11:31:52 +01:00
8c1cf00164
feat: improve and test handling of updating a server ID a 2nd time
2021-11-12 11:31:52 +01:00
7ef762a88e
tests: add `test_serving_readiness_router`
2021-11-12 11:31:52 +01:00
84a5df44e3
fix: Let structopt handle parsing Uuid
2021-11-11 14:37:19 -05:00
95c46e58bc
Merge branch 'main' into jpg/restore-uuid-bytes
2021-11-11 15:34:23 +00:00
19d5200d1c
test: Make assertion more concise and avoid an allocation, thank you clippy
2021-11-11 09:56:43 -05:00
3667cb36fa
refactor: align HTTP error handling w/ cloud 2
...
Streamline HTTP error by using the same representation as cloud 2. Also
ensure that only internal server errors result in an error-level log
entry.
Fixes #3075 .
2021-11-11 14:48:50 +01:00
6f268f8260
refactor: extract DML types ( #2731 ) ( #3084 )
...
* refactor: extract DML types (#2731 )
* chore: fmt
2021-11-11 12:34:07 +00:00
f0136e9791
ci: run clippy for all features
...
This indeed fixes some issues in our object store implementation.
Closes #537 .
2021-11-11 10:48:21 +01:00
7e13cb77ac
feat: add transactions for context to database owner info on disown
2021-11-10 16:49:32 -05:00
fb7bde527f
feat: Implement disown in Server and Database
2021-11-10 16:49:32 -05:00
489fad0040
feat: Add disown database to the gRPC API
2021-11-10 16:49:31 -05:00
b1e81186d3
feat: Add disown database commands to the database CLI
2021-11-10 16:49:31 -05:00
81e5c33a72
fix: Make error handling match a bit nicer to read and more consistent
2021-11-10 16:49:31 -05:00
bebb95830a
fix: Make create database CLI output consistent with delete database CLI output
2021-11-10 16:49:31 -05:00
c80088bf12
feat: HTTP write interface for router
2021-11-10 17:55:46 +01:00
721011ab51
docs: improve wording
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-11-09 13:11:32 +01:00
00b3282180
refactor: partially split up HTTP implementation and tests
...
This should make it easier to to wire up the HTTP write interface for
the router. No functional change intended, this just moves code around.
2021-11-09 12:22:31 +01:00
83dbfe1333
feat: gRPC PB-write interface for new router
2021-11-08 15:49:13 +01:00
b226d629c1
feat: router write logic
2021-11-08 14:09:33 +01:00
e4d943d79c
chore(deps): bump comfy-table from 4.1.1 to 5.0.0
...
Bumps [comfy-table](https://github.com/nukesor/comfy-table ) from 4.1.1 to 5.0.0.
- [Release notes](https://github.com/nukesor/comfy-table/releases )
- [Changelog](https://github.com/Nukesor/comfy-table/blob/master/CHANGELOG.md )
- [Commits](https://github.com/nukesor/comfy-table/compare/v4.1.1...v5.0.0 )
---
updated-dependencies:
- dependency-name: comfy-table
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-08 10:55:57 +00:00
7214fa9d6a
chore(deps): bump serde_json from 1.0.68 to 1.0.69
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.68 to 1.0.69.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.68...v1.0.69 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-08 10:31:33 +00:00
60f0deaf1e
feat: remove flatbuffer entry ( #3045 )
2021-11-05 20:19:24 +00:00
de672b7599
fix: Store the restore database UUID as bytes instead of as a string
2021-11-05 15:20:18 -04:00
7d070f8582
feat: remove influxdata.iox.write.v1.WriteService ( #3044 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-05 17:51:01 +00:00
88868e7496
feat: remove legacy write service from influxdb_iox_client ( #3043 )
...
* feat: remove legacy write service from influxdb_iox_client
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-05 17:26:18 +00:00
c8d80e5c28
fix: Change database paths to be under /dbs/ instead of under /[server id]/
2021-11-05 10:14:06 -04:00
898567e221
feat: migrate server to DbWrite ( #2724 ) ( #3035 )
...
* feat: migrate server to DbWrite (#2724 )
* chore: print perf log output
* fix: don't suppress CI status code
* chore: review feedback
* fix: don't error on empty line protocol write payloads
* fix: test
* fix: test
2021-11-05 11:09:33 +00:00
06dfdc4af8
feat: simplify sharding ( #3031 )
...
* feat: simplify sharding
* chore: fix lint
2021-11-04 16:57:19 +00:00
78101eb52d
feat: implement `RemoteService` for new router mode
2021-11-04 15:35:05 +01:00
53422dbd48
Merge branch 'main' into crepererum/router_mode_get_set_cfg
2021-11-04 14:06:44 +00:00
d28749bd93
refactor: remove Entry from Db interfaces ( #2724 ) ( #3027 )
2021-11-04 13:28:47 +00:00
4840dc11cb
feat: add basic CRUD operations for router configs
2021-11-04 12:03:14 +01:00
07ba629e2b
feat: migrate write buffer producer to MutableBatch and pbdata ( #2743 ) ( #3021 )
...
* feat: migrate write buffer producer to MutableBatch and pbdata (#2743 )
* fix: Kafka message content type header
* chore: fix doc
2021-11-04 10:20:40 +00:00
0f62bb1686
fix: Distinguish between restoring twice and name collisions on restore
...
And return them in the correct situations.
2021-11-03 15:02:22 -04:00
6615001dc7
fix: Extract loading the database name out of the rules file into a function
2021-11-03 15:02:21 -04:00
52d9df5caf
fix: Thread a Not Found error on restore database through all APIs
2021-11-03 15:02:21 -04:00
ac75be6b78
fix: Don't require name for database restore CLI; it's now unused
2021-11-03 15:02:21 -04:00
3e44f17698
fix: Return a useful error on attempt to restore an active database
2021-11-03 15:02:18 -04:00
f6327b1940
fix: Send back the correct error when attempting to restore the same database twice
2021-11-03 15:01:56 -04:00
a0b4b6b8f9
test: Add assertions that ensure UUID doesn't change when DatabaseRules are updated
2021-11-03 15:01:27 -04:00
207d701bdb
fix: Have DatabaseState keep track of the UUID after reading it from object storage
2021-11-03 15:01:27 -04:00
62e7376394
fix: Restore database should only need UUID, not database name too
2021-11-03 15:01:26 -04:00
e3e99c45cf
fix: remove stale comment
2021-11-03 16:35:54 +01:00
1cf697f235
refactor: make server fixture creation less confusing
2021-11-03 16:34:51 +01:00
a91e572b43
feat: implement basic router server type
...
This only implements the deployment service and nothing else yet.
2021-11-03 16:34:51 +01:00
39a7f3069f
feat: add alternative string encodings to pbdata ( #2724 ) ( #3006 )
...
* feat: add alternative string encodings to pbdata (#2724 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-02 18:14:34 +00:00
0d0c0cb42b
refactor: move write buffer configs to new home
...
Write buffer configs will partially be shared by database and router
nodes, so lets move them into a shared home.
2021-11-02 10:17:01 +01:00
011af2d6ba
feat: add `DeploymentService`
...
Ref #2980 .
2021-11-01 17:55:40 +01:00
8c3446ac87
Merge branch 'main' into crepererum/issue2980b
2021-11-01 16:39:32 +00:00
f1a6468e7b
feat: migrate write buffer consumer to use DbWrite ( #2724 ) ( #3003 )
...
* feat: migrate write buffer consumer to use DbWrite (#2724 )
* fix: doc
* chore: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-01 16:38:48 +00:00
1b377f0377
chore(deps): bump tokio-stream from 0.1.7 to 0.1.8
...
Bumps [tokio-stream](https://github.com/tokio-rs/tokio ) from 0.1.7 to 0.1.8.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.7...tokio-stream-0.1.8 )
---
updated-dependencies:
- dependency-name: tokio-stream
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 11:38:59 +00:00
c724bb2ee8
Merge branch 'main' into dependabot/cargo/tokio-1.13.0
2021-11-01 11:30:33 +00:00
c540b40f05
chore(deps): bump tokio from 1.12.0 to 1.13.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.12.0...tokio-1.13.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 11:21:59 +00:00
bcd66c555a
feat: add `RemoteService`
...
Ref #2980 .
2021-11-01 11:38:46 +01:00
a1f978e280
chore(deps): bump tokio-util from 0.6.8 to 0.6.9
...
Bumps [tokio-util](https://github.com/tokio-rs/tokio ) from 0.6.8 to 0.6.9.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.6.8...tokio-util-0.6.9 )
---
updated-dependencies:
- dependency-name: tokio-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 01:28:41 +00:00
fc25e313c6
Merge branch 'main' into cn/iox-obj-sto-refactors
2021-10-29 13:53:26 +00:00
c02e476175
fix: Split database name from rules at the last possible function
...
When updating database rules, the database name should come from the
rules. Enforce this by only taking the rules as a parameter, rather than
having two arguments that could possibly disagree with each other.
2021-10-29 09:46:23 -04:00
f3edda11e7
fix: Print all delete database output to stdout
2021-10-29 09:46:02 -04:00
83e4514a43
feat: add `DeleteService`
...
Ref #2980 .
2021-10-29 11:57:31 +02:00
2caed5456e
Merge branch 'main' into crepererum/run_modes_boilerplate3
2021-10-29 07:06:12 +00:00
7b938fa521
docs: extend `CommonServerState` docs
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-10-29 09:05:29 +02:00
f340738aa5
docs: explain `influxdb_ioxd::main`
2021-10-29 09:04:39 +02:00
3c22d93e38
fix: remove TODO error handling
2021-10-29 08:59:04 +02:00
8a2410e161
feat: mutable batch write entry ( #2724 ) ( #2973 )
...
* feat: mutable batch write entry (#2724 )
* chore: lint
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-28 20:15:28 +00:00
088cdd8f97
fix: Share comfy_table style and give it a more descriptive name and docs
2021-10-28 16:00:58 -04:00
0bccd35f8c
fix: List details about databases only prints active dbs and their UUIDs
...
We can no longer list deleted databases because the server no longer
knows about them, and we now have UUIDs that are useful to know about,
so change the detailed listing of databases to return UUID instead of
possible deleted at time.
2021-10-28 13:20:29 -04:00
ce55cf401c
feat: Return UUID when creating a database
...
Seems polite and is useful for some test setup
2021-10-28 13:20:29 -04:00
4077d3a20a
fix: Take a UUID when restoring a database
2021-10-28 13:20:29 -04:00
95ba84edca
fix: Return UUID when a database is deleted
...
So that it can be immediately used to restore elsewhere.
2021-10-28 13:20:29 -04:00
e80b902d15
test: Delete tests either irrelevant or covered elsewhere
...
Now that the database paths use UUIDs instead of names+generation ids,
and that IoxObjectStore checks there are no files in a directory before
it'll create a database directory (and has tests for that).
2021-10-28 13:20:28 -04:00
09ad30cbfd
fix: For dumping a db catalog, look up db location in server config first
...
Rather than requiring the user of this command to know the database UUID
or otherwise having the knowledge of how to construct a database object
store location in this debugging command.
2021-10-28 13:20:28 -04:00
f160712f5e
fix: Remove the concept of generations
...
From iox_object_store and everywhere that fails to compile as a result.
Also make path handling in iox_object_store a bit more consistent.
2021-10-28 13:20:28 -04:00
8198c1ff2a
refactor: Rename IoxObjectStore constructors to better match what server does with Databases
2021-10-28 13:20:27 -04:00
641c7b3051
refactor: make `influxdb_ioxd` server-type-generic
2021-10-28 17:38:06 +02:00
9996cfc6b3
fix: things I forgot to rename
2021-10-28 16:33:03 +02:00
9cd3dfb624
refactor: make gRPC server code reusable
...
Sadly `tonic` is a bit of a type nightmare (see
<23c1392fb7/tonic/src/transport/server/mod.rs (L452-L470)
2021-10-28 13:19:24 +02:00
a6e0a069cc
refactor: rename "run mode" to "server type"
2021-10-28 11:16:41 +02:00
843551d291
docs: improve
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-10-28 11:16:41 +02:00
c465d0af8f
refactor: abstract HTTP inteface for different run modes
...
First iteration of a common interface for the different server run
modes. This splits the existing HTTP server into a part that can be used
by all run modes (e.g. health checks, metrics endpoint, profiling) and a
part that is specific to the the database run mode.
Note that the interface is not final and might require more iterations,
but we have to start somewhere.
2021-10-28 11:16:41 +02:00
6ebca13ed7
Merge branch 'main' into crepererum/prepare_run_mode_cli
2021-10-27 11:04:07 +00:00
cdc0c63cb7
feat: Add LP batch size histogram metric
...
Part of #2752
2021-10-27 12:14:48 +02:00
6e29cec502
refactor: rename `query` mode to `database`
2021-10-27 09:31:46 +02:00
41bc373fe8
refactor: offer different modes via "run" CLI
...
Running a server is now using `influxdb_iox run MODE [args]`, e.g.
`influxdb_iox run query --server-id 1`. Another mode that will follow
soon is `router`.
The old syntax `influxdb_iox run [args]` (w/o the mode part) is still
supported but a deprecation note will be printed.
2021-10-27 09:31:46 +02:00
a0cdca7781
chore: move `influxdb_iox` into a proper workspace package
2021-10-26 11:02:33 +02:00
Marco Neumann
Andrew Lamb
Edd Robinson
Nga Tran
Carol (Nichols || Goulding)
kodiakhq[bot]
dependabot[bot]
Raphael Taylor-Davies
Marko Mikulicic
Carol (Nichols || Goulding)
Jake Goulding
Carol (Nichols || Goulding)