14cb4b4fdc
fix: Fix copy&pasted check_response descriptions ( #740 )
...
Copy pasta is best pasta
2021-02-04 18:28:08 -05:00
5c3661dd91
chore: refactor how columns are kept in the table in the mutable buffer
...
This one is a bit of a yak shave in advance of adding column names to the summary statistics. I needed the column and its name (or identifier) to be together, rather than the id to index map that existed before. I think the table_id and column_id stuff should be refactored out over time since they add a ton of complexity to the code and don't add much value. Having those as Strings would be much easier and probably be a drop in the bucket for memory usage. Basically, I don't think they need to be interned. But that would be an even more massive refactor touching so many things in the MutableBuffer, I leave it as a later exercise.
Hopefully this makes the code simpler and cleaner in the interim and it gives me the column_id with the column so that I can easily look up the name when generating the summary statistics for a chunk.
2021-02-04 16:31:55 -05:00
1f8043a3f8
feat: add approximate memory size tracking to mutable buffer
...
This updates the mutable buffer, partitions, chunks, dictionary, tables, and individual columns to be able to return their approximate memory size used. This doesn't aim to be exact. There are spots where I'm not counting table or column pointers or the partition key. My expectation is that the data size will dominate and a few pointers here and there won't matter.
2021-02-04 13:50:43 -05:00
13b097728f
fix: Improve code, comments and error messages for consistency ( #737 )
...
* fix: Improve code, comments and error messages for consistency
* fix: update error message
2021-02-04 12:05:24 -05:00
1ff9a3ab87
Merge pull request #741 from influxdata/listdb
...
feat: Implement list databases API
2021-02-04 17:06:47 +01:00
202a4187a1
feat: Implement list databases API
...
Ref #722
2021-02-04 16:50:40 +01:00
42de58497e
Merge pull request #742 from influxdata/routerify-error-handling
...
feat: consistent global error handling and logging
2021-02-04 15:04:25 +00:00
29314a6118
feat: consistent global error handling and logging
2021-02-04 13:15:17 +00:00
d5ebf9c3da
chore: Update deps again ( #738 )
2021-02-04 06:02:05 -05:00
6906e6bdd3
refactor: standardize erorr handling in rpc service ( #728 )
2021-02-03 14:50:00 -05:00
d66eae1a44
feat: Implement TableProvider for Trait for `Db` ( #730 )
...
* feat: Implement TableProvider for Db
Gets us selection pushdown in plans, sets us up for predicate pushdown
Includes: SendableRecordBatchStreams for mutable buffer and read buffer results
fixup snapshots
* docs: comments
2021-02-03 14:18:47 -05:00
4b524dbb35
Merge pull request #712 from influxdata/jg/flight-server
2021-02-03 10:25:51 -05:00
1f1ebefd7a
fix: Switch to database name instead of org/bucket for the Flight API
2021-02-03 10:02:06 -05:00
fc091041c5
fix: Propagate query planning error instead of unwrapping
2021-02-03 09:56:56 -05:00
b0a3ac1a65
docs: Leave a TODO note
2021-02-03 09:56:56 -05:00
19673657c2
refactor: Extract the Flight server into its own module
2021-02-03 09:56:56 -05:00
51d18e6ac7
refactor: Move GrpcService definition up a module
2021-02-03 09:56:56 -05:00
9ff2ee99d8
refactor: Move gRPC make_server up a module
2021-02-03 09:56:56 -05:00
8975b1cbf5
fix: into_iter in the assert_table_eq macro
...
This makes it so the macro can take `&[String]` or `&[&str]`.
2021-02-03 09:56:56 -05:00
39338b195b
feat: Basic scaffolding of a flight server do_get
2021-02-03 09:56:56 -05:00
95d5f759df
chore: Organize use imports
2021-02-03 09:56:56 -05:00
a5e09366b0
feat: Export arrow-flight from arrow-deps
2021-02-03 09:56:56 -05:00
4954cef08d
Merge pull request #675 from influxdata/cn+jg/osp-types
2021-02-03 09:56:30 -05:00
0f8ef9c7d5
Merge branch 'main' into cn+jg/osp-types
2021-02-03 09:09:04 -05:00
abc26a33c1
chore: Update dependencies (again) ( #718 )
...
* chore: Update dependencies (again)
* refactor: update for changes in DataFusion API
* fix: fmt
* fix: clippy
2021-02-02 18:33:01 -05:00
47101896c8
Merge pull request #719 from mkmik/editorconfig
...
style: Add editor config
2021-02-02 17:34:59 +00:00
941eb60b43
style: Add editor config
...
hopefully with some uncontroversial rules.
2021-02-02 13:28:02 +01:00
485a59b2f8
feat: Implement logfmt (Heroku) formatted log output ( #716 )
...
* feat: add option to output logs formatted via logfmt
* refactor: Apply suggestions from code review
Co-authored-by: Edd Robinson <me@edd.io>
* fix: add tests for span inclusion
* feat: Also log spans
* fix: bug in normalizer
Co-authored-by: Edd Robinson <me@edd.io>
2021-02-01 16:43:01 -05:00
5d1c7dfe82
docs: Improve descriptive code comments as suggested in review
2021-02-01 14:56:49 -05:00
5c8b351f57
fix: Address clippy suggestions
2021-02-01 14:56:49 -05:00
f9539f2b74
fix: Remove blanket trait impl now causing a stack overflow
2021-02-01 14:56:49 -05:00
ff6955a433
refactor: Extract a trait for ObjectStoreApi with associated path
...
This is the promised cleanup. This structure gets rid of a lot of
intermediate structures and encodes through associated types how the
object stores and path types are related.
The enums are still necessary to avoid having generics leak all over
the place, but the object store variants and path variants should always
match because they'll always come from the object store trait
implementations that use the associated types.
2021-02-01 14:56:47 -05:00
c40205b37e
test: Move DirsAndFileName functionality tests with the definition
2021-02-01 14:39:18 -05:00
596a73f56a
refactor: Extract a FilePath type for use in file storage
...
Enforces that on-disk storage will only ever use file paths.
More cleanup coming!
2021-02-01 14:39:18 -05:00
d39131ab49
refactor: Extract a CloudPath type for use in cloud storage
...
This is the start of using the type system to enforce that only
CloudPaths will be used with S3, GCS, and Azure.
Still some mess in here, cleanup coming.
2021-02-01 14:39:16 -05:00
7d3b4db234
fix: InMemory doesn't need pagination
2021-02-01 14:35:47 -05:00
fdbe602e57
refactor: Always get a path to build from the object store
2021-02-01 14:30:21 -05:00
5e74ae2ee7
refactor: Segments should know how to persist themselves to object storage
2021-02-01 14:30:20 -05:00
288861e646
feat: implement table_schema in partition chunk, mutable buffer, read buffer ( #705 )
...
fix: sort output schema by name
fix: Update data_types/src/schema.rs
Co-authored-by: Edd Robinson <me@edd.io>
refactor: Update read_buffer/src/lib.rs
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: Edd Robinson <me@edd.io>
2021-02-01 13:54:58 -05:00
9560e33c5d
Merge pull request #715 from influxdata/er/feat/read-buffer/bool
...
feat: add Boolean column support
2021-02-01 17:13:32 +00:00
2885b0528b
refactor: address PR comments
2021-02-01 16:56:56 +00:00
d44099d242
refactor: update read_buffer/src/column/bool.rs
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-02-01 15:17:51 +00:00
7e1ac34906
refactor: recordbatch boolean -> Bool
2021-02-01 12:29:04 +00:00
172c8d146c
refactor: wire up bool encoding to read buffer column
2021-02-01 12:02:14 +00:00
36d9541cbc
feat: an arrow-backed boolean encoding
2021-02-01 12:02:14 +00:00
0dae29f9ea
Merge pull request #714 from influxdata/er/fix/read_buffer/arrow_fixed_null
...
refactor: wire up nullable column encodings
2021-02-01 12:01:38 +00:00
679e04fdb3
refactor: remove redundant From implementations
2021-02-01 10:52:27 +00:00
42e2178110
refactor: simpify arrow -> read buffer column
2021-01-31 21:19:18 +00:00
28b596b883
refactor: wire up fixed null encoding
2021-01-31 21:12:47 +00:00
0195dfc03a
test: add coverage for NULL values in queries
2021-01-31 21:12:06 +00:00
Marko Mikulicic
Paul Dix
Andrew Lamb
Raphael Taylor-Davies
Raphael Taylor-Davies
Carol (Nichols || Goulding)
Carol (Nichols || Goulding)
Jake Goulding
Dom
Edd Robinson