1fa08d0de5
test: add test for float encoding rules
2021-05-14 13:24:53 +01:00
1ac949e7ea
feat: implement predicate pushdown on RLE
2021-05-14 13:23:42 +01:00
0cf445991e
refactor: all read buffer tests passing
2021-05-14 13:14:12 +01:00
7525f6e9e3
feat: teach read buffer to create RLE float columns
2021-05-14 13:14:10 +01:00
9a666fac00
feat: implement RLE methods for materialising
2021-05-14 13:05:02 +01:00
c55dce3af5
feat: implement stat methods
2021-05-14 13:05:02 +01:00
958219d63e
feat: skeleton scalar RLE
2021-05-14 13:05:02 +01:00
41abeba7a0
Merge pull request #1494 from influxdata/er/feat/rb_raw_size
...
feat: add uncompressed read buffer size metric
2021-05-14 12:47:57 +01:00
91fda41f8e
refactor: pdate read_buffer/src/column/boolean.rs
...
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
2021-05-14 12:11:54 +01:00
d80e71ad86
feat: add new metric to track raw size
2021-05-14 10:34:54 +01:00
51c9c15026
refactor: include raw size in log message
2021-05-14 09:42:24 +01:00
966093deec
feat: expose size_raw via ReadBuffer API
2021-05-14 09:42:24 +01:00
984f505267
feat: implement raw column size on bool columns
2021-05-14 09:42:24 +01:00
1a20f3fb4a
feat: implement raw column size on float columns
2021-05-14 09:42:24 +01:00
301df03e72
feat: implement raw column size on integer columns
2021-05-14 09:42:24 +01:00
850db3f6c2
feat: implement raw size on string columns
2021-05-14 09:42:22 +01:00
68b3577b0d
Merge pull request #1492 from influxdata/pointsmetric
...
fix: Add ingest_fields_total
2021-05-13 16:08:49 +00:00
35c2ca17fc
fix: Add ingest_fields_total
...
ingest_lines_total count lines (which apparently are the same as points, quite confusingly)
No yaks harmed in the making of this PR.
(NOTE: the code around metric, especially dealing with happy and error paths is very painful;
to be done in another PR)
2021-05-13 17:55:07 +02:00
8ab71be5eb
feat: use PackedStringArray in mutable buffer ( #1476 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-13 15:25:19 +00:00
553bc83eef
Merge pull request #1486 from influxdata/ntran/parquet_os
...
feat: we now can read parquet files from all kind of object stores
2021-05-13 10:27:40 +00:00
9583636748
feat: we now can read parquet files form all kind of object stores
2021-05-12 18:05:34 -04:00
8be95856ab
test: Add a test with multiple threads using a process clock
2021-05-12 13:31:26 -04:00
cecb4afc58
docs: Add some documentation on the assumptions around this design
2021-05-12 13:31:26 -04:00
b3fb61a0b3
refactor: Rename now_nanos to system_clock_now for clarity
2021-05-12 13:31:26 -04:00
425aacc391
refactor: Extract ProcessClock into its own type
2021-05-12 13:31:26 -04:00
b749353d21
refactor: Use a compare_exchange loop instead of Arc Mutex
2021-05-12 10:58:08 -04:00
5dfd152549
test: Use the now_nanos helper function more in tests
2021-05-12 10:58:08 -04:00
f28c9ae04c
docs: Add unit and semantic information about the process clock
2021-05-12 10:58:08 -04:00
513d4731be
feat: Add a process clock to Db and use it for Sequenced Entries
...
Connects to #1157 .
2021-05-12 10:58:06 -04:00
f98807936d
test: Some tests don't call await, so they don't need to be async
2021-05-12 10:57:05 -04:00
ebea554e5a
feat: Report num_cpus seen by IOx on startup ( #1484 )
2021-05-12 13:42:16 +00:00
0b1ef52481
chore: upgrade arrow and datafusion ( #1483 )
2021-05-12 13:08:37 +00:00
658c34009a
Merge pull request #1480 from influxdata/er/feat/read_buffer/num_rle
...
refactor: shuffle encodings around in Read Buffer
2021-05-12 10:30:13 +00:00
1416097a35
Merge branch 'main' into er/feat/read_buffer/num_rle
2021-05-11 23:30:55 +01:00
aa83669740
refactor: move encodings to scalar module
2021-05-11 22:49:20 +01:00
482e4dab86
refactor: shuffle string encodings
2021-05-11 22:47:42 +01:00
f86e0641fd
refactor: clarify benchmark
2021-05-11 22:47:42 +01:00
f5fe270e43
refactor: move benchmark
2021-05-11 22:47:36 +01:00
b02105e47b
feat: construct StringDictionary from PackedStringArray ( #1475 )
...
* feat: construct StringDictionary from PackedStringArray
* chore: fix formatting
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-11 20:16:25 +00:00
67984fbff5
Merge pull request #1474 from influxdata/er/feat/rb_metrics
...
fix: ensure metrics not overwriting
2021-05-11 20:06:16 +00:00
696e4e0cfd
fix: ensure metrics not overwriting
2021-05-11 20:57:31 +01:00
4409d2c8af
feat: instrument catalog locks ( #1464 )
...
* feat: instrument catalog locks (#1355 )
* chore: add metrics test
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-11 18:59:11 +00:00
9d0c3a2b1a
refactor: Remove multi-table per chunk code in MUB ( #1471 )
...
* refactor: Remove multi-table per chunk code in MUB
* fix: clippy
* fix: bench build
* fix: merge conflicts
2021-05-11 17:49:07 +00:00
c85d1574eb
feat: move dictionary and bitset into arrow_utils ( #1459 )
...
* feat: move dictionary and bitset into arrow_utils
* chore: review feedback
* chore: remove redundant dictionary methods
* chore: consistent type parameter name in PackedStringArray
* chore: review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-11 16:43:38 +00:00
d1da954fe4
feat: don't store encoded strings twice in RLE dictionaries ( #1469 )
2021-05-11 15:22:25 +00:00
e00b4aaa24
feat: associate labels with a domain ( #1463 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-11 13:03:45 +00:00
be74eab3fd
Merge pull request #1468 from influxdata/er/feat/rb_metrics
...
feat: add in-depth column encoding metrics
2021-05-11 12:45:08 +00:00
32abe2e777
feat: wire up stats to metrics
2021-05-11 13:38:32 +01:00
c4987028fb
feat: expose all column stats
2021-05-11 13:00:52 +01:00
88ed58aa8a
feat: column statistics for int/float
2021-05-11 13:00:52 +01:00
Edd Robinson
kodiakhq[bot]
Marko Mikulicic
Raphael Taylor-Davies
Nga Tran
Carol (Nichols || Goulding)
Andrew Lamb
Marco Neumann