milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
milvus/internal/datanode
History
XuanYang-cn 40d5c902b6
Enable getting multiple segments in plan result (#28350) 
Compaction plan result contained one segment for one plan. For l0
compaction would write to multiple segments, this PR expand the segments
number in plan results and refactor some names for readibility.

- Name refactory: - CompactionStateResult -> CompactionPlanResult -
CompactionResult -> CompactionSegment

See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 		2023-11-14 15:56:19 +08:00
..
allocator Replace manual composed grpc call with Broker methods (#27676) 2023-10-13 09:55:34 +08:00
broker Not convert legacy error code to new merr (#28232) 2023-11-08 14:42:20 +08:00
iterators Add iterators (#27643) 2023-10-18 19:34:08 +08:00
metacache Change MetaCache interface (#28388) 2023-11-14 15:08:19 +08:00
syncmgr Fix atomic.Int64 not found in go 1.18 (#28216) 2023-11-07 15:16:21 +08:00
writebuffer Add `WriteBuffer` to provide abstraction for delta policy (#27874) 2023-11-04 12:10:17 +08:00
OWNERS [skip ci]Update OWNERS files (#11898) 2021-11-16 15:41:11 +08:00
README.md [skip ci]Change uppercase of MinIO (#9539) 2021-10-09 11:22:58 +08:00
binlog_io.go Add revive sub-lints and fix existing problems (#27495) 2023-10-07 20:53:38 +08:00
binlog_io_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
buffer.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
buffer_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
cache.go Use typeutil.ConcurrentMap instead of sync.Map (#25846) 2023-07-24 10:23:01 +08:00
cache_test.go Make duplicate flush recalls success and other logic update (#16755) 2022-05-06 17:49:51 +08:00
channel_manager.go Refine state check (#27541) 2023-10-11 21:01:35 +08:00
channel_manager_test.go [skip e2e] Fix unstable ut (#27618) 2023-10-11 07:21:32 +08:00
channel_meta.go Use merr to prevent datanode panic (#28121) 2023-11-04 00:58:21 +08:00
channel_meta_test.go Refine chunk manager errors (#27590) 2023-10-31 12:18:15 +08:00
compaction_executor.go Enable getting multiple segments in plan result (#28350) 2023-11-14 15:56:19 +08:00
compaction_executor_test.go Enable getting multiple segments in plan result (#28350) 2023-11-14 15:56:19 +08:00
compactor.go Enable getting multiple segments in plan result (#28350) 2023-11-14 15:56:19 +08:00
compactor_test.go Enable getting multiple segments in plan result (#28350) 2023-11-14 15:56:19 +08:00
data_node.go Refine datanode Timetick Sender (#28393) 2023-11-13 21:46:20 +08:00
data_node_test.go Replace manual composed grpc call with Broker methods (#27676) 2023-10-13 09:55:34 +08:00
data_sync_service.go Reduce the goroutine in flowgraph to 2 (#28233) 2023-11-13 10:50:17 +08:00
data_sync_service_test.go Not convert legacy error code to new merr (#28232) 2023-11-08 14:42:20 +08:00
event_manager.go Reduce the goroutine in flowgraph to 2 (#28233) 2023-11-13 10:50:17 +08:00
event_manager_test.go Refine datanode Timetick Sender (#28393) 2023-11-13 21:46:20 +08:00
flow_graph_dd_node.go Reduce the goroutine in flowgraph to 2 (#28233) 2023-11-13 10:50:17 +08:00
flow_graph_dd_node_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
flow_graph_delete_node.go Enhance newDataSyncService (#27277) 2023-09-27 11:07:25 +08:00
flow_graph_delete_node_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
flow_graph_dmstream_input_node.go Enhance newDataSyncService (#27277) 2023-09-27 11:07:25 +08:00
flow_graph_dmstream_input_node_test.go Support to replicate the mq message (#27240) 2023-10-20 14:26:09 +08:00
flow_graph_insert_buffer_node.go Use single instance for mergedTimeTickerSender (#27730) 2023-11-13 10:18:17 +08:00
flow_graph_insert_buffer_node_test.go Replace manual composed grpc call with Broker methods (#27676) 2023-10-13 09:55:34 +08:00
flow_graph_manager.go Remove recollect segment stats during starting datacoord (#27410) 2023-10-16 10:26:09 +08:00
flow_graph_manager_test.go Not convert legacy error code to new merr (#28232) 2023-11-08 14:42:20 +08:00
flow_graph_message.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
flow_graph_message_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
flow_graph_node.go Make flow graph retry longer (#24355) 2023-05-25 09:53:30 +08:00
flow_graph_time_tick_node.go Fix datanode ttNode goroutine leak (#27878) 2023-10-24 18:22:10 +08:00
flow_graph_time_ticker.go Use single instance for mergedTimeTickerSender (#27730) 2023-11-13 10:18:17 +08:00
flush_manager.go Not convert legacy error code to new merr (#28232) 2023-11-08 14:42:20 +08:00
flush_manager_test.go Not convert legacy error code to new merr (#28232) 2023-11-08 14:42:20 +08:00
flush_task.go Support to replicate the mq message (#27240) 2023-10-20 14:26:09 +08:00
flush_task_counter.go Fix flushManager.isFull is too slow (#28141) 2023-11-03 14:42:17 +08:00
flush_task_counter_test.go Fix flushManager.isFull is too slow (#28141) 2023-11-03 14:42:17 +08:00
flush_task_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
io_pool.go Fix getting incorrect CPU num (#28146) 2023-11-06 06:02:16 +08:00
io_pool_test.go Fix copylocks linter errors (#26217) 2023-08-08 20:41:15 +08:00
meta_service.go Replace manual composed grpc call with Broker methods (#27676) 2023-10-13 09:55:34 +08:00
meta_service_test.go Replace manual composed grpc call with Broker methods (#27676) 2023-10-13 09:55:34 +08:00
meta_util.go Use go-api/v2 for milvus-proto (#24770) 2023-06-09 01:28:37 +08:00
metrics_info.go Refine state check (#27541) 2023-10-11 21:01:35 +08:00
mock_test.go Enable getting multiple segments in plan result (#28350) 2023-11-14 15:56:19 +08:00
rate_collector.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
rate_collector_test.go Move some modules from internal to public package (#22572) 2023-04-06 19:14:32 +08:00
segment.go Fix panic while flushing dropped/compacted segment (#27927) 2023-10-25 22:10:13 +08:00
segment_sync_policy.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
segment_sync_policy_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
segment_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
services.go Enable getting multiple segments in plan result (#28350) 2023-11-14 15:56:19 +08:00
services_test.go Enable getting multiple segments in plan result (#28350) 2023-11-14 15:56:19 +08:00
timetick_sender.go Refine datanode Timetick Sender (#28393) 2023-11-13 21:46:20 +08:00
timetick_sender_test.go Refine datanode Timetick Sender (#28393) 2023-11-13 21:46:20 +08:00
util.go Format the code (#27275) 2023-09-21 09:45:27 +08:00

README.md

Data Node

DataNode is the component to write insert and delete messages into persistent blob storage, for example MinIO or S3.

Dependency

  • KV store: a kv store that persists messages into blob storage.
  • Message stream: receive messages and publish imformation
  • Root Coordinator: get the latest unique IDs.
  • Data Coordinator: get the flush information and which message stream to subscribe.