History

wei liu 9fc5f1176c fix: Drop segment meta info with prefix (#29856 ) If segment has more than 128 log fils, drop segment will exceed etcd txn ops limit, which will failed the drop segment request This PR drop segment meta info with prefix, to avoid drop segment meta failed --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>		2024-01-11 12:22:51 +08:00
..
broker	…
OWNERS	…
README.md	…
allocator.go	…
allocator_test.go	…
build_index_policy.go	…
channel.go	…
channel_checker.go	…
channel_checker_test.go	…
channel_manager.go	enhance:change some logs (#29579 )	2024-01-05 16:12:48 +08:00
channel_manager_factory.go	…
channel_manager_test.go	…
channel_store.go	…
channel_store_test.go	…
cluster.go	…
cluster_test.go	…
compaction.go	fix: compacted segment status was flushing instead flushed and L0 segment trigger gc slowly (#29587 )	2024-01-09 10:52:49 +08:00
compaction_l0_view.go	…
compaction_l0_view_test.go	enhance:change some logs (#29579 )	2024-01-05 16:12:48 +08:00
compaction_scheduler.go	…
compaction_scheduler_test.go	…
compaction_test.go	fix: compacted segment status was flushing instead flushed and L0 segment trigger gc slowly (#29587 )	2024-01-09 10:52:49 +08:00
compaction_trigger.go	enhance:change some logs (#29579 )	2024-01-05 16:12:48 +08:00
compaction_trigger_test.go	feat: Support multiple vector indexes in a collection (#27700 )	2023-12-29 11:44:45 +08:00
compaction_trigger_v2.go	…
compaction_trigger_v2_test.go	…
compaction_view.go	…
compaction_view_manager.go	…
compaction_view_manager_test.go	…
const.go	…
errors.go	…
errors_test.go	…
garbage_collector.go	enhance: Add concurrency for datacoord segment GC (#29561 )	2024-01-03 13:16:57 +08:00
garbage_collector_test.go	enhance: Add concurrency for datacoord segment GC (#29561 )	2024-01-03 13:16:57 +08:00
handler.go	enhance: Explicitly pass LevelZero segment ids in vchan info (#29612 )	2024-01-04 16:46:45 +08:00
index_builder.go	…
index_builder_test.go	…
index_engine_version_manager.go	…
index_engine_version_manager_test.go	…
index_meta.go	feat: Support multiple vector indexes in a collection (#27700 )	2023-12-29 11:44:45 +08:00
index_meta_test.go	feat: Support multiple vector indexes in a collection (#27700 )	2023-12-29 11:44:45 +08:00
index_service.go	feat: Support multiple vector indexes in a collection (#27700 )	2023-12-29 11:44:45 +08:00
index_service_test.go	feat: Support multiple vector indexes in a collection (#27700 )	2023-12-29 11:44:45 +08:00
indexnode_manager.go	…
indexnode_manager_test.go	…
meta.go	fix: compacted segment status was flushing instead flushed and L0 segment trigger gc slowly (#29587 )	2024-01-09 10:52:49 +08:00
meta_test.go	fix: Drop segment meta info with prefix (#29856 )	2024-01-11 12:22:51 +08:00
meta_util.go	…
metrics_info.go	…
metrics_info_test.go	…
mock_allocator_test.go	…
mock_channel_store.go	…
mock_channelmanager.go	…
mock_cluster.go	…
mock_compaction_meta.go	…
mock_compaction_plan_context.go	…
mock_handler.go	…
mock_index_engine_version_manager.go	…
mock_scheduler.go	…
mock_session_manager.go	…
mock_test.go	…
mock_trigger_manager.go	…
policy.go	…
policy_test.go	…
segment_allocation_policy.go	…
segment_allocation_policy_test.go	…
segment_info.go	…
segment_manager.go	enhance:change some logs (#29579 )	2024-01-05 16:12:48 +08:00
segment_manager_test.go	…
server.go	enhance: Use ChannelManger interface in Server (#29629 )	2024-01-08 17:46:47 +08:00
server_test.go	enhance: Use ChannelManger interface in Server (#29629 )	2024-01-08 17:46:47 +08:00
services.go	enhance: Use ChannelManger interface in Server (#29629 )	2024-01-08 17:46:47 +08:00
services_test.go	enhance: Use ChannelManger interface in Server (#29629 )	2024-01-08 17:46:47 +08:00
session.go	…
session_manager.go	enhance:change some logs (#29579 )	2024-01-05 16:12:48 +08:00
session_manager_test.go	…
util.go	feat: Support multiple vector indexes in a collection (#27700 )	2023-12-29 11:44:45 +08:00
util_test.go	…

README.md

Data Coordinator

Data cooridnator(datacoord for short) is the component to organize DataNodes and segments allocations.

Dependency

KV store: a kv store has all the meta info datacoord needs to operate. (etcd)
Message stream: a message stream to communicate statistics information with data nodes. (Pulsar)
Root Coordinator: timestamp, id and meta source.
Data Node(s): could be an instance or a cluster, actual worker group handles data modification operations.