0b905078e7
Use writebuffer, sync manager refactory in datanode ( #28320 )
...
See also #27675
This PR make previously merged refactory of datanode go online
- Use write node to replace insert/delete node
- Use write buffer manager to control all buffers
- Use sync manager to control sync tasks instead of flush manager
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-15 15:24:18 +08:00
40d5c902b6
Enable getting multiple segments in plan result ( #28350 )
...
Compaction plan result contained one segment for one plan. For l0
compaction would write to multiple segments, this PR expand the segments
number in plan results and refactor some names for readibility.
- Name refactory: - CompactionStateResult -> CompactionPlanResult -
CompactionResult -> CompactionSegment
See also: #27606
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-14 15:56:19 +08:00
c41df18b6d
Add compaction meta in `SyncSegmentsRequest` ( #28199 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-07 10:06:17 +08:00
e386a62fae
Remove recollect segment stats during starting datacoord ( #27410 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-16 10:26:09 +08:00
82b2edc4bd
Replace manual composed grpc call with Broker methods ( #27676 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-13 09:55:34 +08:00
0f14d18201
Optimize the codec code of session ( #27360 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-10-01 10:33:30 +08:00
5c5f9aa05e
Enhance newDataSyncService ( #27277 )
...
- Add flowgraph.Assemble assembles nodes in flowgraph.go
- remove fgCtx in newDataSyncService
- Add newServiceWithEtcdTickler func, reduce param numbers to 3
- Remove unnecessary params
- config.maxQueueLength, config.maxParallelish
See also: #27207
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-27 11:07:25 +08:00
7dd0be1b2c
Enable bulkinsert binlog data with partitionkey ( #27241 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-09-26 10:25:25 +08:00
7f7c71ea7d
Decoupling client and server API in types interface ( #27186 )
...
Co-authored-by:: aoiasd <zhicheng.yue@zilliz.com>
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-09-26 09:57:25 +08:00
676024ff38
Add rpc to notify channel operation ( #27172 )
...
- Add NotifyChannelOperation
- Add CheckChannelOperationProgress
See also: #25309
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-25 14:05:27 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
916528f51a
Add initCtx in getDataSyncService ( #27199 )
...
Passing initCtx to all IO funcs in newDataSyncService,
so when ctx.Canceled, newDataSyncService would return.
See also: #25309
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-09-20 16:03:23 +08:00
4b2802033d
Fix datanode panic due to concurrent compaction and delete processing ( #27167 )
...
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-09-18 19:35:29 +08:00
dd2cb1d44a
Flush by flush channels ( #26859 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-12 21:07:19 +08:00
3c503afe7c
Use typeutil.ConcurrentMap instead of sync.Map ( #25846 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-24 10:23:01 +08:00
96c987ed62
Bulkinsert supports partition keys ( #25284 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-07-11 15:18:28 +08:00
5c3b744b0c
Fix potential crash bug of bulkinsert ( #24763 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-06-13 10:22:38 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
5d172d0f4f
Refine errors of datanode ( #22852 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-28 19:04:00 +08:00
93bc805933
Enhance ID allocator in DataNode ( #22905 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-03-23 19:43:57 +08:00
4a90490a67
Fix the `segment not found` error ( #22772 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-03-17 17:27:56 +08:00
d126f06946
Decouple mq module from internal proto definition ( #22536 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-04 23:21:50 +08:00
148a024e05
Add tickle for datacoord watch event ( #21193 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-02-15 16:20:34 +08:00
6a722396bd
Integration test framework ( #21283 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-01-12 19:49:40 +08:00
3a58ac9fba
Change node stop to TearDownTest ( #21422 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-12-28 20:11:30 +08:00
a7b24cbc53
Move APIs of types.DataNode to services.go ( #21042 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-12-12 10:57:22 +08:00
congqixia
XuanYang-cn
jaime
Jiquan Long
wayblink
SimFG
yihao.dai
groot
aoiasd