Commit Graph

413 Commits (ef6eda639912290414a21b31708bad9696c2ab8b)

Author SHA1 Message Date
Dom Dwyer 5585dd3c21 refactor: switch to using DynObjectStore
Changes all consumers of the object store to use the dynamically
dispatched DynObjectStore type, instead of using a hardcoded concrete
implementation type.
2022-03-15 16:32:52 +00:00
Dom Dwyer 1d5066c421 refactor: rename ObjectStore -> ObjectStoreImpl
Frees up the name for so we can use `dyn ObjectStore` throughout the
code instead of `ObjectStoreApi`.
2022-03-15 16:29:43 +00:00
Carol (Nichols || Goulding) 1dacf567d9
feat: Add a function to the catalog to fetch level 1 parquet files
Fixes #3946.
2022-03-11 15:40:34 -05:00
Carol (Nichols || Goulding) f184b7023c
feat: Update specified parquet file records to compaction level 1
Fixes #3950.
2022-03-11 15:34:40 -05:00
Carol (Nichols || Goulding) fabd262442
feat: Add a function to the catalog to fetch level 0 parquet files
Connects to #3946.
2022-03-11 15:34:05 -05:00
Nga Tran 5a29d070ea
feat: Implement the compact function for NG Compactor (#4001)
* feat: initial implementation of compact a given list of overlapped parquet files

* feat: Add QueryableParquetChunk and some refactoring

* feat:  build queryable parquet chunks for parquet files with tombstones

* feat: second half the implementation for Compactor's compact. Tests will be next

* fix: comments for trait funnctions fof QueryChunkMeta

* test: add tests for compactor's compact function

* fix: typos

* refactor: address Jake's review comments

* refactor: address Andrew's comments and add one more test for files in different order in the vector

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-11 20:25:19 +00:00
Andrew Lamb b24ae7d23b
refactor: extract out compactor creation from config (#4018)
* refactor: extract out compactor creation from config

* fix: fmt

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-11 14:46:34 +00:00
Carol (Nichols || Goulding) 944f628e29
fix: Remove data_types as a dependency of ng compactor (#3993)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-09 17:03:02 +00:00
Nga Tran 09fba1d2c0
feat: NG Compactor - main function for finding and compacting parquet files (#3973)
* feat: main function for finding and compacting parquet files

* chore: Apply suggestions from code review

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* refactor: rename file and struct

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-08 16:34:43 +00:00
Andrew Lamb b870b9340b
chore: remove uneeded dependencies (#3929)
* chore: remove unused deps in compactor

* chore: remove unused deps in influxdb_ioxd

* chore: remove unused deps in object_store

* chore: remove unused deps in server

* fix: object_store needs observability deps when compiled with aws

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-03-04 17:39:46 +00:00
Luke Bond 34e06e8689
fix: compactor server stays up; removed unused delegates (#3855)
* fix: compactor server stays up; removed unused delegates

* chore: fmt

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-24 16:30:44 +00:00
dependabot[bot] ad3868ed7c
chore(deps): Bump tokio from 1.16.1 to 1.17.0 (#3814)
* chore(deps): Bump tokio from 1.16.1 to 1.17.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.16.1 to 1.17.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.16.1...tokio-1.17.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build: update workspace-hack

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom Dwyer <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-22 16:27:43 +00:00
Luke Bond 0f012de70c
feat: adding compactor CLI command and crate
Closes: #3777
2022-02-21 12:24:09 +00:00