Commit Graph

127 Commits (96fb595cc0a4dde98451eb923c2e4300e4af9622)

Author SHA1 Message Date
Nga Tran 402d9c748c chore: cargo fmt 2021-04-28 16:52:52 -04:00
Nga Tran 2a2760bd18 feat: complete tests where data in both RUB and OS 2021-04-28 16:14:07 -04:00
Nga Tran 140d96dbea feat: tests ffor loading data to object store and make sure twe still query read buffer 2021-04-28 15:59:17 -04:00
Marco Neumann eddc9319ff docs: deny broken intradoc links 2021-04-27 13:22:28 +02:00
Carol (Nichols || Goulding) 272cdb85ce fix: Use the ServerId type everywhere, for writing, querying, anything 2021-04-26 18:44:32 +00:00
Carol (Nichols || Goulding) b8face3335 refactor: Organize use statements 2021-04-26 18:44:32 +00:00
Jake Goulding 67f5ad841d refactor: Introduce ServerId and CurrentServerId types 2021-04-26 18:44:32 +00:00
Nga Tran 657bfa1b20 refactor: address Andrew's comments 2021-04-16 17:44:46 -04:00
Nga Tran b3e110a241 refactor: address Jake's comment 2021-04-16 17:27:40 -04:00
Nga Tran 4c23ca8888 feat: full implementation of parquet's read_filter for review 2021-04-16 16:03:24 -04:00
Andrew Lamb e226b5a820
feat: Use TimestampNanosecondArray for timestamps in IOx (#1230)
* refactor: Create Arrow arrays using iterators

* feat: use Timestamp64(TimeUnit::Nanosecond) for timestamps

* feat: add support for timestamp array

* fix: update more tests

* fix: remove unecessary code

Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-16 15:55:33 +00:00
Nga Tran 231ebb54d4 chore: fix a format 2021-04-14 16:32:25 -04:00
Nga Tran 4e2d59d9a5 feat: saimplement a few more functions as part of supporting query dfrom parquet files 2021-04-14 16:06:47 -04:00
Nga Tran 05bf28ce85 feat: Add 2 main functions table_schema and table_names for Parquet Chunk ato pay a foundation for querying it 2021-04-13 18:23:55 -04:00
Nga Tran 4a6d6bd7ad feat: initial work for querying data from parquet file in object store 2021-04-13 13:57:46 -04:00
Raphael Taylor-Davies 1997324344
feat: mutable buffer snapshotting (#1179)
* feat: mutable buffer snapshotting

* chore: review feedback
2021-04-13 12:14:54 +00:00
Nga Tran 453aeaf1a0 feat: Add tests for writing RB chunks to Object Store 2021-04-09 17:39:23 -04:00
Nga Tran f501a74aea refactor: Address review comments 2021-04-07 21:28:03 -04:00
Nga Tran be6e1e48e4 feat: add writer_id and object_store in Db 2021-04-07 18:36:07 -04:00
Raphael Taylor-Davies c2355aca6d
feat: add basic memory tracking (#1125)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-07 15:38:24 +00:00
Nga Tran 6e01fbc382 feat: ause TableSummary as metadata for parquet chunk's tables and read buffer's read_filter ot get data 2021-04-05 15:37:34 -04:00
Nga Tran 4bdf8963e6 feat: continue buidling foundation for writing RB chunks to parquet files 2021-04-02 16:06:25 -04:00
Nga Tran 49267114d3 chore: merge main into branch and resolve conflicts 2021-04-01 13:22:49 -04:00
Nga Tran 1463c6645f feat: Add ChunkState::ObjectStore and rename ParquetChunk to Chunk 2021-04-01 11:53:03 -04:00
Nga Tran 19a453a483 feat: finally have some framework with clear todos for writing a chunk into parquet files 2021-03-31 16:21:53 -04:00
Nga Tran cd409b471f feat: continue the implementation 2021-03-30 21:31:51 -04:00
Nga Tran 0bcd52d5c9 feat: Add more changes 2021-03-30 18:31:09 -04:00