Commit Graph

138 Commits (be9b3a48534e2292d9fc26c4fd1c88c1db72254f)

Author SHA1 Message Date
Andrew Lamb a614fef5bc
chore: remove more unused dependencies (#1658)
* chore: remove more unused deps

* refactor: move benchmarks into server_benchmarks crate
2021-06-09 10:17:20 +00:00
kodiakhq[bot] 87297f7db4
Merge branch 'main' into cn/delete 2021-06-07 13:32:42 +00:00
Raphael Taylor-Davies 1e7ef193a6
refactor: use field metadata to store influx types (#1642)
* refactor: use field metadata to store influx types

make SchemaBuilder non-consuming

* chore: remove unused variants

* chore: fix lints
2021-06-07 13:26:39 +00:00
Carol (Nichols || Goulding) 4d6569583e fix: Partially restore SequencedEntry as Entry+sequencer_id+sequence_num 2021-06-04 14:40:19 -04:00
Andrew Lamb 32c6ed1f34
refactor: More cleanup related to multi-table chunks (#1604)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-06-02 17:00:23 +00:00
Andrew Lamb d8fbb7b410
refactor: Remove last vestiges of multi-table chunks from PartitionChunk API (#1588)
* refactor: Remove last vestiges of multi-table chunks from PartitionChunk API

* fix: remove test that can no longer fail

* fix: update tests + code review comments

* fix: clippy

* fix: clippy

* fix: restore test_measurement_fields_error test
2021-06-01 16:12:33 +00:00
Andrew Lamb 64328dcf1c
feat: cache schema on catalog chunks too (#1575) 2021-06-01 12:42:46 +00:00
Andrew Lamb 00e735ef0d
chore: remove unused dependencies (#1583) 2021-05-29 10:31:57 +00:00
Raphael Taylor-Davies db432de137
feat: add distinct count to StatValues (#1568) 2021-05-28 17:41:34 +00:00
Raphael Taylor-Davies d8f19348bf
feat: per-column dictionaries in MUB (#1570)
* feat: per-column dictionaries in MUB

* chore: fmt

* refactor: remove chunk-level dictionary

* chore: remove redundant sort

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-28 13:51:56 +00:00
Raphael Taylor-Davies 792bff07d1
feat: only store ChunkSnapshot in Closed state (#1560)
* feat: only store ChunkSnapshot in Closed state

* chore: review feedback

* feat: record MUB size as closed size

* chore: document column ordering assumption

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-27 10:36:47 +00:00
Raphael Taylor-Davies 4fcc04e6c9
chore: enable arrow prettyprint feature (#1566) 2021-05-27 10:28:14 +00:00
Raphael Taylor-Davies c03b8a3963
refactor: remove tables from ChunkSnapshot (#1295) (#1558) 2021-05-26 10:37:40 +00:00
Andrew Lamb 14ba25f86d
chore: Update datafusion and use released version of arrow crates (#1546)
* chore: Update datafusion and use released version of arrow crate

* fix: Update for change in API
2021-05-24 15:37:22 +00:00
Andrew Lamb ee06ca4c7d
refactor: remove MutableChunk::has_table (#1539)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-22 12:02:42 +00:00
Raphael Taylor-Davies 37880ee89a
refactor: store chunk IDs only in catalog (#1521)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-20 04:07:14 +00:00
Andrew Lamb c1ea6be101
fix: Add context to panic error on tools (#1501)
* fix: Add context to panic error on tools

* fix: log new unwrap that has snuck in

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-17 21:05:01 +00:00
Raphael Taylor-Davies f9178dbb5f
feat: push metrics into catalog (#1488)
* feat: push metrics into catalog

* chore: minor cleanup

* fix: include db labels in chunk metric domains

* chore: fmt

* fix: don't allow dropping moving chunks

* chore: further tweaks

* chore: review feedback

* feat: use new_unregistered() for metric instruments instead of default

* chore: use &[KeyValue] instead of &Vec<KeyValue>

* refactor: make GauageValue non default constructible
2021-05-14 17:37:39 +00:00
Raphael Taylor-Davies cfa8160644
fix: don't test snapshotting when disabled (#1496) (#1498) 2021-05-14 16:15:55 +00:00
Raphael Taylor-Davies 8ab71be5eb
feat: use PackedStringArray in mutable buffer (#1476)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-13 15:25:19 +00:00
Andrew Lamb 9d0c3a2b1a
refactor: Remove multi-table per chunk code in MUB (#1471)
* refactor: Remove multi-table per chunk code in MUB

* fix: clippy

* fix: bench build

* fix: merge conflicts
2021-05-11 17:49:07 +00:00
Raphael Taylor-Davies c85d1574eb
feat: move dictionary and bitset into arrow_utils (#1459)
* feat: move dictionary and bitset into arrow_utils

* chore: review feedback

* chore: remove redundant dictionary methods

* chore: consistent type parameter name in PackedStringArray

* chore: review comments

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-11 16:43:38 +00:00
Marco Neumann 795f5bfcb7 refactor: make `StatValues::{min,max}` optional + handle NaNs
This will allow us to:

- handle all-NULL columns correctly
- be in-line with Parquet (where min/max are optional)
- handle NaNs at least somewhat sane (they do not "poison" stats
  anymore)
2021-05-10 17:12:25 +02:00
Andrew Lamb b5ea71f45f
feat: Expose the storage usage for each column in system.chunk_columns (#1441)
* feat: Expose the storage usage for each column in system.chunk_columns

* fix: fixup logical conflicts

* refactor: move coalsce logic into the read buffer

* fix: Update system_tables to not use coalese

* fix: Improve comments

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-05-07 12:36:49 +00:00
Andrew Lamb 884baf7329
feat: add column_type and influxdb_column_type, remove row_count from system.columns (#1415)
* feat: add column_type and influxdb_column_type, remove row_count from system.columns

* fix: update tests

* fix: more test update

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: fmt

* fix: copy/paste type conversion to avoid cross dependency between data_types and internal_types

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-05-06 12:59:30 +00:00
Raphael Taylor-Davies 10f89a3e8d
refactor: split entry out into separate crate (#1428)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-06 11:36:23 +00:00
Raphael Taylor-Davies ca1c698fd0
chore: update hashbrown (#1430)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-05 22:32:46 +00:00
Raphael Taylor-Davies 411cf134e9
refactor: explode arrow_deps (#1425)
* refactor: explode arrow_deps

* chore: workaround doctest bug
2021-05-05 16:59:12 +00:00
Andrew Lamb 40b9b09cdc
refactor: rename assert_table_eq to assert_batches_eq (#1368) 2021-04-30 10:51:08 +00:00
Andrew Lamb 0788892413
feat: add row_count to system.chunks and Chunk managment API (#1373)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-29 13:53:25 +00:00
Carol (Nichols || Goulding) 9aefcd216f fix: Validate that ClockValue is never 0 2021-04-28 13:54:55 -04:00
Marco Neumann eddc9319ff docs: deny broken intradoc links 2021-04-27 13:22:28 +02:00
Raphael Taylor-Davies 20117de078
feat: string dictionary encoding (#1220) (#1262)
* feat: string dictionary encoding (#1220)

* chore: review comments

* chore: fix lint

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-27 09:36:58 +00:00
Carol (Nichols || Goulding) 272cdb85ce fix: Use the ServerId type everywhere, for writing, querying, anything 2021-04-26 18:44:32 +00:00
Raphael Taylor-Davies 0a835436ac
feat: use bitmasks within MUB (#1274) (#1289)
* feat: use bitmasks within MUB (#1274)

* chore: review feedback

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-26 18:00:16 +00:00
Edd Robinson 146b48325b refactor: use bool::then 2021-04-22 21:59:41 +00:00
Raphael Taylor-Davies 74c25f541d
feat: fast MUB dictionary arrow conversion (#1273)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-22 20:31:44 +00:00
Raphael Taylor-Davies fe4fa29930
feat: MUB benchmarks (#635) (#1271) 2021-04-22 11:15:32 +00:00
Carol (Nichols || Goulding) 58bbe55fd6 fix: Remove error type not used anywhere 2021-04-21 17:43:03 +00:00
Carol (Nichols || Goulding) adb96ee758 fix: Make error types more granular; cover field-to-tag and tag-to-field mismatch 2021-04-21 14:49:37 +00:00
Carol (Nichols || Goulding) 4407fdd322 fix: Make line lengths less long
rustfmt doesn't touch macro calls 😬
2021-04-21 14:49:37 +00:00
Carol (Nichols || Goulding) d360a08dff fix: Remove error type that is no longer used 2021-04-21 14:49:37 +00:00
Paul Dix e08795ce6f fix: update schema conflicts to display column name 2021-04-21 14:49:37 +00:00
Raphael Taylor-Davies 1cf3ab3978
feat: enable disabling MUB snapshot caching (#1203) (#1264)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-21 08:21:31 +00:00
Carol (Nichols || Goulding) c9772db01b fix: Allow this upper case acronym; it could be confusing otherwise 2021-04-19 08:48:11 -04:00
Carol (Nichols || Goulding) f136931225 fix: Inconsistent ordering lints 2021-04-19 08:48:11 -04:00
Marco Neumann 40decd0154 fix: fix Rust 2021 compat warning
This fixes the following warning:

```text
warning: panic message is not a string literal
   --> mutable_buffer/src/table.rs:591:13
    |
591 |             format!("didn't match returned error: {:?}", response)
    |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `#[warn(non_fmt_panic)]` on by default
    = note: this is no longer accepted in Rust 2021
    = note: the panic!() macro supports formatting, so there's no need for the format!() macro here
help: remove the `format!(..)` macro call
    |
591 |             "didn't match returned error: {:?}", response
    |            --                                           --
```
2021-04-19 14:00:04 +02:00
Andrew Lamb 529c99c93f
fix: don't clone arrays to make TimestampNanosecondArrays (#1241)
* fix: avoid clone

* fix: remove another clone
2021-04-16 18:40:22 +00:00
Andrew Lamb e226b5a820
feat: Use TimestampNanosecondArray for timestamps in IOx (#1230)
* refactor: Create Arrow arrays using iterators

* feat: use Timestamp64(TimeUnit::Nanosecond) for timestamps

* feat: add support for timestamp array

* fix: update more tests

* fix: remove unecessary code

Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-16 15:55:33 +00:00
Marko Mikulicic c9cbc74857 feat: Use a DID newtype instead of u32 for dictionary ids
Rationale
---------

We use `u32` throughout the codebase to reference for interned dictionary strings.
We also use `u32` for other reasons and it would be nice to get some help from the compiler
to avoid mixing them up
2021-04-13 22:38:41 +00:00