Commit Graph

1494 Commits (38abe9735f37db2ffd8d08dadf7e3b452b0119a7)

Author SHA1 Message Date
Andrew Lamb 38abe9735f
Merge branch 'main' into dom/iox-api-client 2021-01-12 13:19:49 -05:00
Andrew Lamb 2938c8f8fc
feat: implement chunk listing and snapshotting in mutable buffer (#641)
* feat: implement chunk listing and snapshotting in mutable buffer

* fix: update to use latest version of string interner and remove custom clone

* docs: fix comment
2021-01-12 12:46:18 -05:00
Dom ac88b5a92b refactor: fix lints 2021-01-12 17:23:19 +00:00
Dom e06b989fa6
Merge branch 'main' into dom/iox-api-client 2021-01-12 17:10:51 +00:00
Dom ba7e3dfcbd refactor: remove static lifetimes from const 2021-01-12 17:09:20 +00:00
Dom a6f1832a81 feat: set writer ID method
Adds an API call to set the server's writer ID.
2021-01-12 16:48:55 +00:00
Andrew Lamb c1a7778d85
refactor: move id and deps out of query crate (#646) 2021-01-12 11:47:43 -05:00
Dom 62349edb94 feat: create IOx API client
Initialises a new library crate and implements a basic IOx API client.

The API client supports:
	- ping
	- create database

Care has been taken to abstract away the underlying HTTP client used
(reqwest) and avoid leaking it into the public API (error types is a
common leak!) This makes updating the HTTP client and/or swapping it for
something else a backwards compatible change for end users of the crate.

Outstanding items:
	- move shared API types into a sensible location
	- discriminate between various IOx error responses

The former doesn't need doing until we publish the crate and will likely
be rather invasive / conlict prone so aiming to merge this PR and then
move things around in a follow-up.

The latter would allow us to expose error conditions to the user such
that they can take actions to remidy the situation / know if the request
can or should be retried / etc. Currently we expose a string error
message when requests fail, requiring string matching and/or passing the
string higher in the stack (and thus punting the problem to the caller).
It would be very nice to have typed errors, but a detail I have left for
later.
2021-01-12 16:38:33 +00:00
Dom 7ca127f270
Merge pull request #645 from influxdata/dom/api-set-writer-id
feat: add set_writer_id API endpoint
2021-01-12 16:02:30 +00:00
Dom 8764e00d63
Merge branch 'main' into dom/api-set-writer-id 2021-01-12 15:48:26 +00:00
Dom f270c48c6e
Merge pull request #640 from influxdata/dom/refactor-cli-config
refactor: replace config system with structopt
2021-01-12 15:17:29 +00:00
Dom 1a1a14308d feat: API set writer ID endpoint
Adds a new endpoint /iox/api/v1/id that accepts a JSON object in the form:

    {"id":42}

And calls into the server's set_id method to assign the writer ID to the server.
2021-01-12 15:16:59 +00:00
Dom e60906c5d5 refactor: remove async from ID methods
Both set / get ID methods are not async, so this removes the annotation &
awaits.
2021-01-12 15:01:51 +00:00
Dom e856b740e1
Merge branch 'main' into dom/refactor-cli-config 2021-01-12 15:01:44 +00:00
Dom 72becf3317 fix: remove config subcommand & config ignore 2021-01-12 14:16:20 +00:00
Dom 63ff0943eb fix: strip "server" from args 2021-01-12 14:04:44 +00:00
Dom e38df076b4
fix: do not overwrite databases (#644)
* fix: do not overwrite databases

Do not overwrite an existing database when attempting to create a DB with an
existing name.

This should probably update the existing database config, but for the time being
this prevents the existing database from being silently dropped.

Fixes #643

* test: ensure duplicate DB names returns an error

Covers #643 with a test case.
2021-01-12 08:55:29 -05:00
Dom e88114425d docs: update help text
Updates the main help text and adds config precedence order to server help text.
2021-01-12 11:52:56 +00:00
Dom 71daaffce6 docs: update README config section 2021-01-12 11:35:41 +00:00
Dom f8e74126c9 fix: source .env to display values in help text
Sources the .env (ignoring errors) before constructing the top-level Clap
config. This ensures any values from the .env file are loaded and displayed.
2021-01-12 11:32:43 +00:00
Andrew Lamb fd28d8a01b
refactor: Use u32 for Chunk ids consistently (#639) 2021-01-11 16:07:22 -05:00
Dom bdc832d040 refactor: replace config system with structopt
Replaces the hand-rolled config system with a StructOpt managed config struct.

I've got most of it ported across, but the interaction between all the logging
config bits is complex! I've left what is there and hooked in the value from
the config struct (which directly replaces the env var in usage, as it also
sources from the env).
2021-01-11 18:43:14 +00:00
Andrew Lamb de6c385d85
fix: Run CI checks only once on PRs, not twice (#546)
* fix: Run CI tests only once on PRs, not twice

* fix: run on pull_request rather than push
2021-01-11 12:01:07 -05:00
Edd Robinson 00bea0a388
Merge pull request #631 from influxdata/er/feat/read_buffer/select_database
feat: read filter API for lazily streaming out of Read Buffer
2021-01-11 16:24:31 +00:00
Edd Robinson 9ec0ae26e1 refactor: implement From<ArrayRef> 2021-01-11 16:11:21 +00:00
Edd Robinson 9eef7b4d7f feat: add enum for selecting 'all' columns 2021-01-11 16:11:21 +00:00
Edd Robinson 61466fed44 refactor: add outline for read_aggregate 2021-01-11 16:11:21 +00:00
Edd Robinson 5a15a11a5c feat: lazily return record batches for read_filter 2021-01-11 16:11:21 +00:00
Edd Robinson c3019a91bd feat: add support for determining logical column types 2021-01-11 16:11:20 +00:00
Dom 4dff270c34
Merge pull request #633 from influxdata/dom/fix-api-panic
fix: invalid API request body causes panic
2021-01-11 13:42:14 +00:00
Dom f98a33bbe9 fix: invalid API request body causes panic
Removes a stray unwrap() that was causing a request with invalid JSON / empty
body to panic the handler thread.

Fixes #632
2021-01-11 13:25:22 +00:00
Edd Robinson 3ccb36a9ca
Merge pull request #628 from influxdata/er/feat/read_buffer/partition-map
feat: add partition concept to ReadBuffer
2021-01-08 21:20:12 +00:00
Edd Robinson 1d972e01c8 refactor: Update read_buffer/src/lib.rs
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-01-08 21:03:38 +00:00
Edd Robinson 23f27e10fa feat: add partition concept to ReadBuffer 2021-01-08 21:03:38 +00:00
Carol (Nichols || Goulding) 0ef604ad17
Merge pull request #627 from influxdata/cn/fewer-servers 2021-01-08 14:25:34 -05:00
Carol (Nichols || Goulding) 2be992b2f6
Merge branch 'main' into cn/fewer-servers 2021-01-08 12:41:48 -05:00
Carol (Nichols || Goulding) b66ad643d5 refactor: Extract panic logging to its own crate for ease of reuse 2021-01-08 12:36:56 -05:00
Andrew Lamb 6d0c538eca
refactor: pull DBChunk into its own module (#623)
* refactor: pull DBChunk into its own module

* refactor: consolidate impl blocks
2021-01-08 12:27:21 -05:00
Carol (Nichols || Goulding) 6f3b5a6a9e refactor: Alphabetize workspace members to match dir listing 2021-01-08 12:20:31 -05:00
Carol (Nichols || Goulding) cd03f39280 refactor: Move code in the server::server module to the root
This gets rid of the redundant server::server imports.
2021-01-08 12:19:58 -05:00
Carol (Nichols || Goulding) 155485a752 refactor: Rename commands/server to commands/influxdb_ioxd 2021-01-08 12:19:16 -05:00
Edd Robinson d2717e97c1
Merge pull request #616 from influxdata/er/feat/record-batch-read-buffer
feat: add support for writing chunks to ReadBuffer
2021-01-08 16:41:06 +00:00
Edd Robinson 46f85bb6a6 refactor: adrress PR comments
Co-authored-by: Andrew Lamb <alamb@influxdata.com>

Co-authored-by: Dom <dom@itsallbroken.com>
2021-01-08 16:19:19 +00:00
Edd Robinson 4ce6821d90 feat: implement table_names on 2021-01-08 16:19:19 +00:00
Edd Robinson 590b74e386 refactor: add test for adding chunk to database 2021-01-08 16:19:19 +00:00
Edd Robinson 6df2de62bb refactor: provide an API for table row groups 2021-01-08 16:19:19 +00:00
Edd Robinson 954da31e83 test: fix tests 2021-01-08 16:19:19 +00:00
Edd Robinson 2178a6eae4 feat: hook up record batch -> chunk to store 2021-01-08 16:19:19 +00:00
Edd Robinson b1ab6a189d feat: record batch -> read buffer column 2021-01-08 16:19:19 +00:00
Edd Robinson 8382501440 refactor: validate column types 2021-01-08 16:19:19 +00:00