f90a95fd80
fix: unambigious bucket/org to DB mappings
...
Previosuly the $ORG and $BUCKET was joined as:
$ORG + "_" + $BUCKET
Which is fine unless either $ORG or $BUCKET includes a "_", such as:
$ORG = "org_a"
$BUCKET = "bucket"
and
$ORG = "org"
$BUCKET = "a_bucket"
This change continues to join $ORG and $BUCKET with an underscore, but
disallows underscores in either $ORG or $BUCKET. It appears these values
are non-zero u64s in the gRPC protocol converted to their base-10 string
representations for the DB name, so this seems safe to enforce.
In addition, this change introduces a `DatabaseName` type to avoid
passing bare strings around, and allow consuming code to ensure only
valid database names are provided at compile type. This type works with
both owned & borrowed content so doesn't force a string copy where we
can avoid it, and derefs to `str` to make it easier to use with existing
code.
I've been minimally invasive in pushing the `DatabaseName` through the
existing code and figured I'd see what the sentement is first.
Candidates for conversion from `str` to `DatabaseName` that seem to make
sense to me include:
- `DatabaseStore` trait
- `RemoteServer` trait
- Others? Basically anywhere other than the "edge" API inputs
Fixes #436 (thanks @zeebo)
2020-12-03 16:10:15 +00:00
8c0e14e039
refactor: rename src/server/rpc/storage.rs to src/server/rpc/service.rs ( #513 )
...
* refactor: rename src/server/rpc/storage.rs src/server/rpc/service.rs
* refactor: update references
2020-12-03 09:59:00 -05:00
592c5c3679
Merge pull request #522 from influxdata/dom/ci-reduce-size
...
ci: remove IOx pre-building in rust build container
2020-12-03 13:25:08 +00:00
3589aec136
Merge branch 'main' into dom/ci-reduce-size
2020-12-03 13:14:52 +00:00
54ae680780
Merge pull request #520 from influxdata/er/refactor/read-filter-result
...
refactor: encapsulate results from segment/table into nicer types
2020-12-03 12:51:20 +00:00
7136e5853a
ci: remove IOx pre-building in rust build container
...
Stops adding the IOx source code and performing a cargo build/test/clippy each
night. Previously this build would compile the IOx source & dependencies,
populating the incremental build cache and allowing builds that used the same
dependencies to complete quicker. This build caching was moved to
per-dependency-set caching in #496 , and this pre-build is no longer used.
This should reduce the build image size substantially, making the whole CI
process a bit faster.
2020-12-03 11:58:13 +00:00
254dfc14d8
refactor: apply suggestions from code review
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2020-12-03 11:47:41 +00:00
4f32778596
refactor: implement ReadFilterResults type
...
The `ReadFilterResults` type encapsulates results from multiple
segments. It implements `Display` to allow visualisation of results from
segments in a `select` call.
2020-12-03 11:23:12 +00:00
7ad0b4ad9a
refactor: encapsulate read filter results in type
...
This commit also adds `Display` and `Debug` implementations for
`ReadFilterResult`. These can be used for visualising the contents of
the result of a `read_filter` call on a segment.
The former trait elides the column names.
2020-12-03 11:23:09 +00:00
a088f33c35
Merge pull request #519 from influxdata/er/refactor/time-predicate
...
refactor: avoid requiring time predicate in Segment
2020-12-03 10:06:29 +00:00
05c420cc9e
Merge branch 'main' into er/refactor/time-predicate
2020-12-02 19:13:12 +00:00
381c3038aa
refactor: update segment_store/src/segment.rs
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2020-12-02 19:13:00 +00:00
8cb8276819
fix: Update gRPC definitions so tag_key=_field requests work in IOx ( #517 )
...
* fix: Update gRPC definitions so tag_key=_field requests work in IOx
* docs: Update src/server/rpc.rs
* fix: fixup test
* fix: Apply suggestions from code review
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
* fix: consistent type annotations
* fix: refactor redundant test code into test_helpers
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-12-02 13:58:48 -05:00
4dc5cc46a9
refactor: DRY up the predicate logic
2020-12-02 17:59:45 +00:00
ab83288067
refactor: segment doesn't require time range
2020-12-02 17:56:59 +00:00
ecc4eee8e1
refactor: Move SQL functions into is own trait ( #511 )
...
* refactor: remove uneeded function table_to_arrow from Trait
* refactor: Move SQL functions into is own trait
2020-12-02 08:23:37 -05:00
c99d389a70
refactor: Remove vestigial gRPC IOX API ( #508 )
...
* refactor: remove unused IOx gRPC interface
* refactor: rename file
* refactor: rename gRPC service IOx to IOxTesting
2020-12-02 06:55:15 -05:00
321fa0aa28
Merge pull request #496 from influxdata/dom/separate-ci
...
ci: use GitHub for CI
2020-12-02 10:36:27 +00:00
5ef499bb63
refactor: rename Database --> TSDatabase to better reflect its purpose ( #510 )
...
* refactor: rename Database --> TSDatabase to better reflect its purpose
* refactor: rename field_columns to field_column_names
* fix: clippy?
2020-12-01 12:37:11 -05:00
a2e5af1508
feat: Log gRPC predicates in a human readable form ( #507 )
...
* feat: log predicate passed to storage system
* fix: clippy
2020-12-01 09:31:39 -05:00
60f933c3f7
Merge pull request #509 from influxdata/dom/gz-flate2
...
chore: use flate2 everywhere
2020-12-01 11:29:19 +00:00
c3a0e893ae
test: use flate2
2020-12-01 11:01:10 +00:00
e7fa6ab20a
perf(ingest): use flate2
2020-12-01 11:01:09 +00:00
62eb6eb43e
perf(influx_tsm): use flate2
2020-12-01 11:01:09 +00:00
1646397891
refactor: consolidate GroupedSeriesSet and SeriesSet ( #502 )
2020-11-30 14:23:58 -05:00
e7d88d7383
Merge pull request #505 from influxdata/dom/gz-flate2
...
perf(gzip): switch to flate2
2020-11-30 16:51:31 +00:00
7383c5aa7b
Merge pull request #504 from influxdata/er/refactor/logical_types
...
refactor: restrict logical scalar type
2020-11-30 16:08:22 +00:00
19b0ff284d
chore(server): restrict gzip deflated body size
...
Restricts the size of the decompressed request body when using gzip.
2020-11-30 15:20:20 +00:00
867aba847a
perf(convert): use flate2 for gzip decompression
...
Switches from `libflate` to `flate2` for the top-level commands (specifically
TSM conversion).
2020-11-30 15:18:25 +00:00
9dc9e505ff
refactor: add From<&str> implementation for Value
2020-11-30 13:33:56 +00:00
681b0f0660
refactor: implement From trait for Value
...
This commit adds a set of helper `From` trait implementations for
numerical scalar types.
2020-11-30 13:28:34 +00:00
dfdf7082d9
refactor: remove non-64-bit Scalar types
...
Supports: #501
This commit removes scalar types that are not 64-bit, since we don't
plan to expose these datatypes outside of a column.
2020-11-30 13:14:25 +00:00
b17b6be0a3
Merge pull request #499 from influxdata/dom/db-name-validation
...
chore: validate database names
2020-11-30 13:03:53 +00:00
ccc84de894
refactor: remove logical f32 type
...
Supports: #501
This commit removes the logical `f32` type.
2020-11-30 12:52:36 +00:00
8d1d653193
refactor: reduce set of supported logical types
...
Supports: #501
This commit removes logical integer types other than `i64` and `u64`.
2020-11-30 12:52:31 +00:00
37fea83ca9
test: add newline test case
2020-11-30 11:57:48 +00:00
b9f933763e
Merge pull request #500 from influxdata/dom/fix-line-proto-bench
...
test(bench): fix bad line_parser prometheus assert
2020-11-30 11:55:55 +00:00
9fbe3e1767
test(bench): fix bad line_parser prometheus assert
2020-11-30 11:47:27 +00:00
f08846ca50
chore: validate database names
...
Ensure database names contain only {alphanumeric, -, _} characters.
Fixes #278 .
2020-11-30 11:27:33 +00:00
20f421e9c6
fix: Do not send GroupFrames in response to read_window_aggregate ( #497 )
...
* fix: Do not send GroupFrames in response to read_window_aggregate
* fix: clippy and test
2020-11-30 05:59:05 -05:00
3cbabf0f80
docs(README): document clang requirement ( #498 )
...
Found while investigating #465 - clang is needed to build the croaring
dependency.
2020-11-27 15:32:29 -05:00
46d58dfec5
fix: allow empty `offset` widows for read_window_aggregate offset ( #493 )
...
* fix: allow empty `offset` widows for read_window_aggregate offset
* refactor: Use an enum for clarity
2020-11-27 09:31:22 -05:00
7e1e24ca21
ci: only run CircleCI for /perf branches
...
Filters the triggered CircleCI workflow to only run against branches with /perf
in their name.
No changes to the scheduled build image generation.
2020-11-27 11:34:56 +00:00
9c785a3b0f
ci: add GitHub actions
...
Adds a GitHub CI action:
* Uses the rust-toolchain file to install the requested version.
* Performs a build with default cargo target
* Runs workspace tests
* Checks cargo fmt is happy
* Runs clippy for lints, denying warnings
All commands taken from the existing CircleCI pipeline.
Currently no caching / reuse of build container.
2020-11-27 11:34:56 +00:00
da3b616368
Merge pull request #494 from influxdata/dom/extra-ci-docs
...
docs(ci): extra CI documentation
2020-11-26 13:49:57 +00:00
e93941e487
docs(ci): extra CI documentation
...
Adds some extra quick-glance docs for the CI flow.
2020-11-26 13:46:07 +00:00
559df0e37c
Merge pull request #485 from influxdata/er/refactor/segment-lifetimes
...
refactor: remove some lifetimes
2020-11-26 11:47:42 +00:00
129aa208ba
refactor: More use of datafusion expr building functions ( #488 )
2020-11-26 06:40:57 -05:00
aca65414c7
Merge pull request #492 from influxdata/alamb/moar-logging
...
feat: Log error message returned to gRPC in IOx logs
2020-11-26 11:25:18 +00:00
9c726530ed
Merge pull request #489 from influxdata/dom/storage-fixture-random-port
...
test: random ports for storage fixture server
2020-11-26 10:04:11 +00:00
Dom
Andrew Lamb
Edd Robinson