Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
2,432 Commits
409 Branches
512 Tags
356 MiB
Commit Graph

92 Commits (78aa66d6997cb6b7dd565b47b1d8ea24bac91c49)

Author SHA1 Message Date
Marco Neumann 40decd0154 fix: fix Rust 2021 compat warning 
This fixes the following warning:

```text
warning: panic message is not a string literal
   --> mutable_buffer/src/table.rs:591:13
    |
591 |             format!("didn't match returned error: {:?}", response)
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `#[warn(non_fmt_panic)]` on by default
    = note: this is no longer accepted in Rust 2021
    = note: the panic!() macro supports formatting, so there's no need for the format!() macro here
help: remove the `format!(..)` macro call
    |
591 |             "didn't match returned error: {:?}", response
    |            --                                           --
```
 2021-04-19 14:00:04 +02:00
Andrew Lamb 529c99c93f
fix: don't clone arrays to make TimestampNanosecondArrays (#1241) 
* fix: avoid clone

* fix: remove another clone
 2021-04-16 18:40:22 +00:00
Andrew Lamb e226b5a820
feat: Use TimestampNanosecondArray for timestamps in IOx (#1230) 
* refactor: Create Arrow arrays using iterators

* feat: use Timestamp64(TimeUnit::Nanosecond) for timestamps

* feat: add support for timestamp array

* fix: update more tests

* fix: remove unecessary code

Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-04-16 15:55:33 +00:00
Marko Mikulicic c9cbc74857 feat: Use a DID newtype instead of u32 for dictionary ids 
Rationale
---------

We use `u32` throughout the codebase to reference for interned dictionary strings.
We also use `u32` for other reasons and it would be nice to get some help from the compiler
to avoid mixing them up
 2021-04-13 22:38:41 +00:00
Raphael Taylor-Davies 7ef490694c
refactor: reduce module coupling in mutable buffer (#1199) 
* refactor: reduce module coupling in mutable buffer

* refactor: tweak visibility

* chore: formatting
 2021-04-13 20:59:41 +00:00
Raphael Taylor-Davies 61cd745ab6
refactor: remove mutable buffer predicate logic (#1186) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-04-13 19:09:36 +00:00
Raphael Taylor-Davies 55a77914b1
feat: basic snapshot caching (#1184) 2021-04-13 17:10:28 +00:00
Paul Dix 4d22982b5f fix: fix broken tests from column refactoring 2021-04-13 12:52:14 +00:00
Paul Dix 7e28f8ef66 feat: Implement Entry writing to Db 
This removes the old ReplicatedWrite structure and implements the writing of an Entry to the Db. I also call out in `server/lib.rs` and in the `Db` where sharding and replication might happen.

I've also added helpers in various places to write line protocol to chunks, tables, and databases. That enabled removing a good amount of code from the test helpers crate.
 2021-04-13 12:52:14 +00:00
Raphael Taylor-Davies 1997324344
feat: mutable buffer snapshotting (#1179) 
* feat: mutable buffer snapshotting

* chore: review feedback
 2021-04-13 12:14:54 +00:00
Paul Dix d67739c873 chore: updates for pr feedback 2021-04-12 18:43:14 +00:00
Paul Dix 5893c17905 refactor: PR feedback and change ClockValue to actual type. 2021-04-12 18:43:14 +00:00
Paul Dix 31115742ec feat: Add writing of Entry structures to MB Chunk 
This adds writing of Entry of a vec of TableWriteBatch to the Mutable Buffer Chunk. This is additional to the previous method of writing via ReplicatedWrite. The next step is to remove the old ReplicatedWrite bits.

Test helpers for parsing line protocol into Entry and writing line protocol directly to Chunks have also been added.
 2021-04-12 18:43:14 +00:00
kodiakhq[bot] 319e66706e
Merge branch 'main' into cn/no-more-test-results 2021-04-07 16:45:32 +00:00
Carol (Nichols || Goulding) 82588d5c72 fix: Don't return Result from test functions 2021-04-07 12:40:00 -04:00
Raphael Taylor-Davies c2355aca6d
feat: add basic memory tracking (#1125) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-04-07 15:38:24 +00:00
Jacob Marble 80d55d0829 chore: rename tracing_deps to observability_deps 
OpenTelemetry makes this necessary.
 2021-04-02 13:14:30 -07:00
Carol (Nichols || Goulding) 0b880d3534 chore: Group all tracing-related crates under one crate for easier upgrade management 2021-04-02 09:54:39 -04:00
Andrew Lamb 569f90d937
feat: Add ability to get PartitionSummary statistics from a Db (#1090) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-03-31 14:18:53 +00:00
Andrew Lamb 7154dfd5f6
feat: Add timestamps to ChunkSummary (#1079) 
* refactor: Move timestamps from mutable_buffer::Chunk to catalog::Chunk

* feat: Add timestamps to ChunkSummary

* feat: Add timestamp conversion logic to protobuf types

* test: Add tests

* fix: Update data_types test

* fix: handle negative nanos during conversion

* fix: clippy

* fix: more clippy

* fix: even more clippy

* fix: even more clippy
 2021-03-30 19:03:23 +00:00
Andrew Lamb 0b7dea09d5
refactor: Remove `MutableBufferDb` and Partition (#1068) 
* refactor: Move test for missing tags as null to db

* refactor: move last_write_at and create_time to catalog partition

* refactor: remove mutable buffer partition and port tests

* fix: port over partition sorting logic

* fix: remove stale comment

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-03-30 12:51:37 +00:00
Andrew Lamb 6a48001d13
refactor: Manage storage directly in the Catalog (#1057) 
* refactor: Manage mutable buffer chunks directly

* fix: do not use mutable_buffer for listing table names
 2021-03-29 17:55:07 +00:00
Raphael Taylor-Davies 1a4c793270
feat: lifecycle rules (#1045) 2021-03-29 15:32:36 +00:00
Andrew Lamb 02ae743e8e
refactor: Remove async from Database (#1063) 2021-03-29 12:48:12 +00:00
Andrew Lamb 44d67db733
feat: Rework Db to use Catalog for chunk state (#1043) 
* feat: Rework Db to use Catalog for chunk state

* docs: Update server/src/db.rs

* fix: fmt

* fix: fmt

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-03-24 17:57:11 +00:00
Carol (Nichols || Goulding) c47a252c15 feat: Check in generated WAL flatbuffers code 2021-03-22 09:38:59 -04:00
Carol (Nichols || Goulding) 52e415ae1b fix: Verify flatbuffers when restoring from a file, and not after that 2021-03-22 09:38:59 -04:00
Carol (Nichols || Goulding) 7c6f543d35 fix: wb::ColumnValue is now a set of associated consts, not a real Rust enum 2021-03-22 09:38:58 -04:00
Carol (Nichols || Goulding) 24af745b23 chore: Upgrade to flatbuffers 0.8 2021-03-22 09:38:58 -04:00
Andrew Lamb 6e1795fda0
refactor: Move some types (not yet exposed to clients) into internal_types (#1015) 
* refactor: Move some types (not yet exposed to clients) into internal_types

* docs: Add README.md explaining the rationale

* refactor: remove some stragglers

* fix: fix benches

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: add clippy lints

* fix: fmt

* docs: Apply suggestions from code review

fix typos

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-03-19 16:27:57 +00:00
Andrew Lamb 4781317647
feat: Management API + CLI command to create a new chunk (rollover partition) (#990) 
* feat: Management API + CLI command to create a new chunk (rollover partition)

* fix: Update tests/end_to_end_cases/management_api.rs

fix typo

* fix: logical merge conflict

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-03-15 16:41:18 +00:00
Raphael Taylor-Davies 65f7a1ac5b
fix: use consistent crate versions (#989) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-03-15 15:42:19 +00:00
Andrew Lamb 6ac7e2c1a7
feat: Add management API and CLI to list chunks (#968) 
* feat: Add management API and CLI to list chunks

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: add comment to protobuf

* fix: fix comment

* fix: fmt, fixup merge errors

* fix: fascinating type dance with prost generated types

* fix: clippy

* fix: move command to influxdb_iox database chunk list

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2021-03-12 13:56:14 +00:00
Jacob Marble 488252d9f1 fix: test 2021-03-09 07:52:52 -08:00
Jacob Marble 09c4f88758 fix: cargo fmt 2021-03-09 07:45:05 -08:00
Jacob Marble abb991ffca fix: add tests to unsigned integer parsing in LP and MUB 2021-03-08 15:29:05 -08:00
Jacob Marble ac1b0c04ae fix(line-protocol): add unsigned integer field type 
Fixes #904

The line protocol parser was lacking the unsigned integer type, which
suffixes values with `u`. This adds unsigned integer support to the line
protocol parser, and fills a few corresponding gaps in the mutable
buffer.
 2021-03-08 09:59:12 -08:00
Andrew Lamb 746373a687
refactor: Remove mutable_buffer crate dependency on query crate (#927) 2021-03-05 11:34:27 +00:00
Andrew Lamb a6965769b4
refactor: Remove impl of `query::PartititionChunk` in mutable_buffer: This PR (#923) 2021-03-05 10:51:08 +00:00
Andrew Lamb b1e5cfedf7
refactor: Move chunk predicate creation from query::Predicate into server crate (#922) 
* refactor: Create `ChunkPredicate` predicate in the Server crate

* refactor: move chunk predicate out of chunk

* fix: remove comment in server/src/db/pred.rs

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-03-04 23:23:43 +00:00
Andrew Lamb 3be5c26f92
refactor: Remove impl of `query::Database` in mutable_buffer (#914) 2021-03-04 22:02:42 +00:00
Andrew Lamb 945d2f8d45
refactor: Remove query stuff from mutable buffer (#912) 
* refactor: remove query code from mutable_buffer database.rs

* refactor: remove query code from mutable_buffer table.rs

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-03-04 21:42:56 +00:00
Andrew Lamb 8b1f100df3
feat: make read_group and read_window_aggregate work across chunks (#905) 
* feat: make read_group and read_window_aggregate work across chunks

* refactor: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* refactor: Update query/src/frontend/influxrpc.rs

Improve logic and use strings directly

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: fmt

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-03-04 17:06:31 +00:00
Andrew Lamb 7d8d00781c
feat: Make read_filter work for mutable buffer and read buffer (#882) 
* feat: port read_filter to InfluxRPCPlanner

* fix: remove commented out vestigal test

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: fmt

* fix: Update arrow_deps/src/util.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2021-03-01 16:50:29 +00:00
Nga Tran 18de3bdcab chore: merge main into branch 
Merge branch 'main' into ntran/optimize_column_selection
 2021-02-26 15:29:43 -05:00
Nga Tran f37e5846aa feat: fmt auto fix 2021-02-26 14:56:10 -05:00
NGA TRAN eb81975151 feat: Optimize Column Selection 2021-02-26 14:28:46 -05:00
Andrew Lamb 12deacd8a0
refactor: move SeriesSetPlans into its own module (#878) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-02-25 23:12:39 +00:00
Andrew Lamb 8fb7651719
feat: Port tag_values to the InfluxRPCPlanner (#859) 
* feat: Port tag_values to the InfluxRPCPlanner

* refactor: merge imports

* refactor: rename column_names to tag_column_names for clarity

* fix: Update query/src/frontend/influxrpc.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* refactor: use ensure!

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* refactor: less silly whitespace

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: code review comments

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-02-24 23:11:22 +00:00
Andrew Lamb ed7859e182
refactor: rename tag_column_names --> tag_keys in InfluxRPCPlanner (#860) 2021-02-23 17:04:53 +00:00