Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
2,952 Commits
212 Branches
453 Tags
356 MiB
Commit Graph

2952 Commits (f028a356f4e2c7190f76c9169f9d4b61f2c3d675)

Author SHA1 Message Date
Edd Robinson 2963d63b5e feat: implement byte trimming on nullable encodings 2021-05-17 14:32:55 +01:00
Edd Robinson 6a72274517 feat: extend implementations to more Arrow arrays 2021-05-17 14:32:55 +01:00
Edd Robinson 2b98bca9ca test: allow from slice to be testable 2021-05-17 14:32:55 +01:00
Edd Robinson b7ea53f5db refactor: remove unnecessary from imps 2021-05-17 14:32:55 +01:00
Marco Neumann 18f0a7f614 docs: reference open issue 2021-05-17 14:01:51 +02:00
Marco Neumann cdf0ada6a6 test: test preserved catalog <-> Db write wiring 2021-05-17 13:57:31 +02:00
Raphael Taylor-Davies 91a45fd380
feat: simplify shutdown (#1502) 
* feat: simplify shutdown

* chore: fix lint

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-05-17 11:50:14 +00:00
Marco Neumann 4299371cf2 refactor: remove some code 2021-05-17 12:32:48 +02:00
Marco Neumann 68729dd5ee refactor: avoid string allocation 2021-05-17 12:32:34 +02:00
Marco Neumann adcd8132e7 docs: more comments regarding catalog transaction handling 2021-05-17 12:05:08 +02:00
Marco Neumann a99d53e771 docs: document `OpenTransaction::handle_action*` 2021-05-17 11:48:51 +02:00
Marco Neumann 840c11dab2 feat: wire up catalog preservation write path 
Required a bit of refactoring:

- Add an extra layer between DB an catalog which is the "preserved
  catalog" wrapper. This is required to make the ownership model
  somewhat sane, because during the read operations the "preserved
  catalog" is going to act on the in-mem catalog.
- Move "parquet file written" logic into binding `preserved catalog <->
  catalog state`, so we have a single place where new parquet files are
  announced. For now this only works for chunks that are already known
  (i.e. the writing->written transation when coming from read buffer),
  however in the next PR this will be extended to also handle totally
  new parquet files during transaction playback.

**NOTE: This does NOT include the read path yet!**

Issue: #1382.
 2021-05-17 11:33:22 +02:00
Marco Neumann 4fb800c7a6 refactor: make PreservedCatalog easier to integrate 2021-05-17 11:33:22 +02:00
Marco Neumann f4d7154746 fix: table summaries must include timestamp as well 2021-05-17 11:33:22 +02:00
Marco Neumann ff004c1a51 feat: add `ObjectStore.path_from_dirs_and_filename` 2021-05-17 11:33:22 +02:00
Marco Neumann 7cced3242f feat: add a way to parse infos from parquet paths 2021-05-17 11:33:22 +02:00
Marco Neumann 5969caccb0 feat: return parquet metadata from `write_to_object_store` 2021-05-17 11:33:22 +02:00
Andrew Lamb 07db4932ee
refactor: rename data_types/src/chunk.rs -> data_types/src/chunk_metadata.rs (#1500) 2021-05-15 10:18:01 +00:00
Andrew Lamb 133ce12827
chore: update deps (#1499) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-05-14 20:22:16 +00:00
Raphael Taylor-Davies f9178dbb5f
feat: push metrics into catalog (#1488) 
* feat: push metrics into catalog

* chore: minor cleanup

* fix: include db labels in chunk metric domains

* chore: fmt

* fix: don't allow dropping moving chunks

* chore: further tweaks

* chore: review feedback

* feat: use new_unregistered() for metric instruments instead of default

* chore: use &[KeyValue] instead of &Vec<KeyValue>

* refactor: make GauageValue non default constructible
 2021-05-14 17:37:39 +00:00
Raphael Taylor-Davies cfa8160644
fix: don't test snapshotting when disabled (#1496) (#1498) 2021-05-14 16:15:55 +00:00
kodiakhq[bot] 73c43f1710
Merge pull request #1497 from influxdata/dom/rle-size-target 
refactor: RLE selector target
 2021-05-14 15:34:33 +00:00
Dom db6c7728c7 refactor: use 10% target reduction for RLE 
Comments say 10% but const was 30% - a 10% computed size reduction
sounds sensible!
 2021-05-14 15:08:54 +01:00
Dom 874d7a1118 test: run rle_rows test 
The rle_rows test was missing a #[test] annotation preventing it from
running.
 2021-05-14 14:41:17 +01:00
kodiakhq[bot] d98d66e718
Merge pull request #1444 from influxdata/cn/wb-clock 
feat: Add a process clock to Db and use it for Sequenced Entries
 2021-05-14 13:04:26 +00:00
kodiakhq[bot] fdc8461c7f
Merge branch 'main' into cn/wb-clock 2021-05-14 13:00:06 +00:00
kodiakhq[bot] 02ae69dffc
Merge pull request #1493 from influxdata/er/feat/read_buffer/num_rle 
feat: implement Read Buffer run-length encoding for scalars
 2021-05-14 12:42:51 +00:00
Edd Robinson 0d21d9e2e0 refactor: implement from_iter, reduce code! 2021-05-14 13:32:02 +01:00
Edd Robinson ac4fa1e527 refactor: update read_buffer/src/column/encoding/scalar/rle.rs 
Co-authored-by: Dom <dom@itsallbroken.com>

Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
 2021-05-14 13:24:55 +01:00
Edd Robinson 1fa08d0de5 test: add test for float encoding rules 2021-05-14 13:24:53 +01:00
Edd Robinson 1ac949e7ea feat: implement predicate pushdown on RLE 2021-05-14 13:23:42 +01:00
Edd Robinson 0cf445991e refactor: all read buffer tests passing 2021-05-14 13:14:12 +01:00
Edd Robinson 7525f6e9e3 feat: teach read buffer to create RLE float columns 2021-05-14 13:14:10 +01:00
Edd Robinson 9a666fac00 feat: implement RLE methods for materialising 2021-05-14 13:05:02 +01:00
Edd Robinson c55dce3af5 feat: implement stat methods 2021-05-14 13:05:02 +01:00
Edd Robinson 958219d63e feat: skeleton scalar RLE 2021-05-14 13:05:02 +01:00
Edd Robinson 41abeba7a0
Merge pull request #1494 from influxdata/er/feat/rb_raw_size 
feat: add uncompressed read buffer size metric
 2021-05-14 12:47:57 +01:00
Edd Robinson 91fda41f8e refactor: pdate read_buffer/src/column/boolean.rs 
Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
 2021-05-14 12:11:54 +01:00
Edd Robinson d80e71ad86 feat: add new metric to track raw size 2021-05-14 10:34:54 +01:00
Edd Robinson 51c9c15026 refactor: include raw size in log message 2021-05-14 09:42:24 +01:00
Edd Robinson 966093deec feat: expose size_raw via ReadBuffer API 2021-05-14 09:42:24 +01:00
Edd Robinson 984f505267 feat: implement raw column size on bool columns 2021-05-14 09:42:24 +01:00
Edd Robinson 1a20f3fb4a feat: implement raw column size on float columns 2021-05-14 09:42:24 +01:00
Edd Robinson 301df03e72 feat: implement raw column size on integer columns 2021-05-14 09:42:24 +01:00
Edd Robinson 850db3f6c2 feat: implement raw size on string columns 2021-05-14 09:42:22 +01:00
kodiakhq[bot] 68b3577b0d
Merge pull request #1492 from influxdata/pointsmetric 
fix: Add ingest_fields_total
 2021-05-13 16:08:49 +00:00
Marko Mikulicic 35c2ca17fc
fix: Add ingest_fields_total 
ingest_lines_total count lines (which apparently are the same as points, quite confusingly)

No yaks harmed in the making of this PR.

(NOTE: the code around metric, especially dealing with happy and error paths is very painful;
to be done in another PR)
 2021-05-13 17:55:07 +02:00
Raphael Taylor-Davies 8ab71be5eb
feat: use PackedStringArray in mutable buffer (#1476) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-05-13 15:25:19 +00:00
kodiakhq[bot] 553bc83eef
Merge pull request #1486 from influxdata/ntran/parquet_os 
feat: we now can read parquet files from all kind of object stores
 2021-05-13 10:27:40 +00:00
Nga Tran 9583636748 feat: we now can read parquet files form all kind of object stores 2021-05-12 18:05:34 -04:00