36d50d083f
feat: sharder trait & impl
...
This commit defines the Sharder trait that should allow us to implement
multiple sharding strategies over a defined set of input types (such as
a MutableBatch for writes, DeletePredicate for deletes, etc).
This commit also includes a jump hash implementation that consistently
shards (table name, namespace) tuples to a given shard for all input
types.
2022-01-20 11:10:37 +00:00
3122aec71a
refactor: use static DatabaseName instances
2022-01-20 11:10:37 +00:00
2cd063698f
refactor: API agnostic DML delete handler
...
Changes the DmlHandler::delete() trait method to accept required params,
and accept a DeletePredicate instead of a HttpDeleteRequest so that it
can be re-used in the gRPC handler.
2022-01-20 11:10:37 +00:00
362446b885
Merge pull request #3495 from influxdata/ntran/persist_merge_snapshots
...
refactor: add new fields and comments in ingest data buffer
2022-01-19 23:20:25 +00:00
029f4bb41e
fix: comment
2022-01-19 18:11:00 -05:00
dcf952bb27
chore: merge main to branch
2022-01-19 17:59:05 -05:00
4ede10b3a0
refactor: add new fields and comments in ingest data buffer
2022-01-19 17:53:58 -05:00
df3db021d9
Merge pull request #3494 from influxdata/pd/kafka-get-by-name
...
feat: add get kafka_topic by name to catalog
2022-01-19 22:47:43 +00:00
bfc085c20d
feat: add get kafka_topic by name to catalog
2022-01-19 17:32:23 -05:00
33de1dc69a
Merge pull request #3493 from influxdata/pd/sequencer-get
...
feat: add sequencer get_by_kafka_topic_id_and_partition to catalog
2022-01-19 22:25:42 +00:00
860e5a30ca
refactor: update ingester to get sequencer record and not attempt to create
2022-01-19 17:15:10 -05:00
172d75c6d7
feat: add sequencer get_by_kafka_topic_id_and_partition to catalog
2022-01-19 16:45:06 -05:00
ad0fc25e55
Merge pull request #3492 from influxdata/pd/clear-schema-fix
...
fix: clear postgres schema in test wasn't deleting parquet_file
2022-01-19 21:42:08 +00:00
28db06297f
fix: clear postgres schema in test wasn't deleting parquet_file
2022-01-19 16:30:54 -05:00
75625a7726
Merge pull request #3490 from influxdata/pd/tombstone-catalog
...
feat: Add parquet file records to iox_catalog
2022-01-19 20:00:38 +00:00
d825dab8e2
fix: hakari workspace hack
2022-01-19 14:48:00 -05:00
41038721e1
feat: Add parquet file records to iox_catalog
...
* Adds ParquetFile and scaffolding to IOx catalog
* Changed the file_location in parquet_file to object_store_id which is a uuid
2022-01-19 14:14:54 -05:00
4728d9e762
Merge pull request #3474 from influxdata/ntran/ingester
...
feat: add ingester crate and a few basic data structures for its data lifecycle
2022-01-19 18:34:39 +00:00
be3e523312
fix: use PersistingBatch
2022-01-19 13:25:03 -05:00
9977f174b7
refactor: use wrapper ID
2022-01-19 12:51:04 -05:00
9d61580136
chore: Merge branch 'main' into ntran/ingester
2022-01-19 12:37:47 -05:00
edb97f51cf
refactor: add persisting struct
2022-01-19 12:36:18 -05:00
8a17e1c132
refactor: address review comments
2022-01-19 11:20:20 -05:00
a514449db7
Merge pull request #3483 from influxdata/pd/tombstone-catalog
...
feat: Add Tombstone to Catalog
2022-01-19 14:38:32 +00:00
ce7005ec75
chore: merge main to branch and resolve conflict
2022-01-18 22:06:38 -05:00
e8294d21ec
fix: add .lock
2022-01-18 22:02:27 -05:00
fe9a41ee9a
chore: remove non-longer needed dependency
2022-01-18 21:45:20 -05:00
b89c250ccc
refactor: use RepoColection instead of MemCatalog
2022-01-18 21:39:22 -05:00
b57f027e35
refactor: address review comments
2022-01-18 20:57:13 -05:00
f36d66deb7
feat: Add Tombstone to Catalog
...
* Adds TombstoneId and Tombstone to the iox_catalog with associated interfaces
* Adds SequenceNumber new type for use with Tombstone
* Adds Timestamp new type for use with Tombstone
* Adds constraint to the Postgres schema to enforce tombstone uniqueness by table_id, sequencer_id, and sequence_number
2022-01-18 18:17:21 -05:00
367a9fb812
fix: add workspace-hack
2022-01-18 18:10:42 -05:00
1c970a2064
fix: format
2022-01-18 18:01:47 -05:00
667ec5bfc5
fix: the code is now compile without warnings
2022-01-18 18:01:06 -05:00
d0705001e6
Merge pull request #3482 from influxdata/pd/fix-partitionid-desc
...
fix: typo in partitionid description
2022-01-18 22:47:16 +00:00
b20d1757d0
feat: initialize ingester data
2022-01-18 17:43:03 -05:00
8067316c33
fix: typo in partitionid description
2022-01-18 17:33:05 -05:00
d3e517bf13
chore: Merge branch 'main' into ntran/ingester
2022-01-18 16:11:57 -05:00
125285ae9a
feat: commit in order to pull and merge new commit from main
2022-01-18 16:11:25 -05:00
f33aaf700a
Merge pull request #3481 from influxdata/pd/kafka-partition
...
refactor: add new type for Kafka Partition in Catalog
2022-01-18 20:23:54 +00:00
b1510675ae
refactor: add new type for Kafka Partition in Catalog
2022-01-18 15:13:18 -05:00
e34b24fc7f
Merge pull request #3480 from influxdata/pd/partitions-catalog
...
feat: add Partition scaffolding to iox_catalog crate
2022-01-18 20:11:45 +00:00
f6a901f90d
Merge branch 'main' into pd/partitions-catalog
2022-01-18 20:00:42 +00:00
e395ef7066
feat: add Partition scaffolding to iox_catalog
2022-01-18 14:50:49 -05:00
5d3c9de7aa
chore: Merge branch 'main' into ntran/ingester
2022-01-18 14:04:44 -05:00
23290fd2ff
fix: new data structures suggested by reviewers
2022-01-18 14:04:07 -05:00
65c5b8248b
Merge pull request #3478 from influxdata/pd/catalog-ids
...
refactor: change all ids in catalog to their own types
2022-01-18 18:54:23 +00:00
40cac21e21
refactor: change all ids in catalog to their own types
2022-01-18 12:42:10 -05:00
15d87b4087
Merge pull request #3475 from influxdata/pd/mem-catalog
...
feat: implement memory backed iox_catalog
2022-01-18 15:20:05 +00:00
5e464727d1
refactor: make get_schema_by_name bare function
2022-01-18 10:09:16 -05:00
4f87600081
chore: make iox_catalog tests generic for any backend implementation
2022-01-17 17:20:11 -05:00
Dom
kodiakhq[bot]
NGA-TRAN
Paul Dix