Commit Graph

36 Commits (1c0e1ccfbf1fe7c26a5dd8a2a6b071c78438dac0)

Author SHA1 Message Date
Ten Thousand Leaves b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
zhagnlu 59f8e96522
Change compactor policy from loading all binlogs to loading serially (#13716) (#19280)
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
2022-09-25 20:12:52 +08:00
yah01 ee9d634a43
Compaction requires the source segments indexed (#19368)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-22 18:34:50 +08:00
yah01 71d4a32ce6
Fix deadlock (#19340)
The IsParentDropped() acquires read lock of meta,
which leads to recursively acquire read lock

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-22 15:48:50 +08:00
SimFG d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
yah01 11b352c3ae
Guarantee flushed segments are all indexed (#19062)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-16 11:32:48 +08:00
Enwei Jiao 096149e33f
fix GetCompactionState hang (#18829)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-26 14:22:55 +08:00
cai.zhang c924f73105
Refactor for IndexCoord to support cloud (#18643)
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
jaime c9174d55ba
Refine merge operation during compacting phase (#18399)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-07-28 14:52:31 +08:00
yah01 ef6859a4d6
Refactor handoff handler (#17806)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-29 23:26:19 +08:00
Xiaofan 6fdf88f452
Fix compaction selection policy (#17690)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-22 19:00:14 +08:00
cai.zhang ed94ecf847
Segments can't be compacted when they have reference lock (#17649)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-20 21:56:12 +08:00
Xiaofan 54aeb077e9
Refine compaction selection poliy (#17486)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-15 23:14:10 +08:00
XuanYang-cn c70af73005
Fix datacoord set wrong state for node registering (#17376)
Fix datacoord datarace

See also: #17335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-06 15:10:05 +08:00
Xiaofan bd7d294c6d
Refine log level in data (#15821)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 15:35:55 +08:00
congqixia 5751759c85
Implement mix compaction logic (#15542)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-18 14:51:49 +08:00
Xiaofan be42627ab2
Move Logutil to Util dir (#15447)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-09 18:55:46 +08:00
godchen f2baa987fb
Handle trigger compaction error (#13636)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-07 17:41:40 +08:00
godchen 3874edf163
[skip e2e]Sort imports (#14815)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:16:17 +08:00
Xiangyu Wang 80ffa9c4d0
[skip e2e]Fix license missing issue in datacoord (#14489)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-29 11:38:57 +08:00
Xiaofan 1408926cd9
Remove logs without enough info (#14440)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 10:06:47 +08:00
godchen b8dcbf6604
[skip e2e]Sort imports (#14318)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-27 18:46:42 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
congqixia 74ae8d3923
Fix nil segment caused panic in handleSignal (#13990)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-22 21:23:10 +08:00
congqixia 1dcd06cc78
Add log file info for all kind of segment binlogs (#13606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
edward.zeng 098e1375fb
[skip ci] Refine datacoord compaction trigger comment (#13491)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-16 09:59:12 +08:00
edward.zeng d6b0a47248
[skip ci] Refine datacoord compaction trigger comment (#13483)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-16 09:57:25 +08:00
congqixia 8476791492
Process only flushed/flushing segment in global compaction (#13339)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-14 15:15:10 +08:00
Bingyi Sun 3a2aace6f0
Fix ci issue caused by only compact flushed segments (#13317)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-14 13:55:07 +08:00
congqixia aea5a61e9a
Add switch for auto compaction (#12974)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-08 19:47:05 +08:00
edward.zeng f8c826c597
Fix typo in datacoord compaction trigger (#12636)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-02 19:09:37 +08:00
Bingyi Sun b8c5239784
Add compaction retention config (#11793)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-16 11:47:10 +08:00
Bingyi Sun 54b40da4c5
Add segment drop logic (#11621)
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 00:22:42 +08:00
Bingyi Sun 5574a104c9
Add triggerInfo in compactionTask (#11517)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-11 15:54:42 +08:00
sunby 05681364c9
Fix some review comments in compaction. related pr #9906 (#11399)
issue: #9904
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2021-11-08 21:45:00 +08:00
sunby c26cd79532
Add compaction (#9906)
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00