Commit Graph

33 Commits (hotfix-2.2.5)

Author SHA1 Message Date
zhenshan.cao b0b7e5ba4d
Revert "Restrain gc files by collection (#22504) (#22518)" (#23383)
This reverts commit a58abe1665.

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-04-12 23:18:29 +08:00
congqixia 0cdbf0ec71
Make datacoord handle GetIndexedSegment error from indexcoord (#22673)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-10 10:57:53 +08:00
congqixia a58abe1665
Restrain gc files by collection (#22504) (#22518)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-02 16:55:49 +08:00
Ten Thousand Leaves a28b3c28d2
Fix channel checkpoint issues and fix GC (#22027)
/kind bug

issue: #21897

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-02-08 21:00:32 +08:00
congqixia 5c5c755643
Add Rated group for checkpoint gc skip case (#21949) (#21956)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-06 16:01:54 +08:00
congqixia 474d2e11e8
Add segment gc restraint of dml position before channel cp (#21773)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-17 18:31:43 +08:00
SimFG a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
yah01 89c9cb3680
Fix scan may break GC limitation (#19670)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-10-15 15:13:24 +08:00
jaime ab88dd77e7
Remove unnecessary log for datacoord gc (#19698)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-10-13 14:43:23 +08:00
congqixia 2a2d32c4fc
Fix datacoord panics when collection info not found (#19706)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-10-11 21:39:24 +08:00
jaime 1919353f02
Fix parse segment id error from delta log (#19566)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-09-30 14:18:55 +08:00
congqixia 838a633584
Add ctx parameter for ChunkManager methods (#19546)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-29 16:18:56 +08:00
Ten Thousand Leaves b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
Xiaofan 928a213e31
Fix Chunk Manager Path mis use (#19141)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-09-23 14:40:51 +08:00
SimFG d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
yah01 11b352c3ae
Guarantee flushed segments are all indexed (#19062)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-16 11:32:48 +08:00
jaime efa5dfaa7b
Fix Parse binlog path failure (#18583)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-08-10 17:04:37 +08:00
xing.zhao d90313a1a8
Use chunkManager rather than minio.Client in datanode garbage collection (#18092)
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
2022-07-22 22:10:28 +08:00
Xiaofan 6d82ef8c20
fix garbage collector err handling (#18277)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-07-15 13:54:26 +08:00
cai.zhang bcf3b7426a
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
Xiaofan bd7d294c6d
Refine log level in data (#15821)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 15:35:55 +08:00
Bingyi Sun daf25bcb81
Add log when remove keys in garbage collector. (#15453)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-02-18 11:59:49 +08:00
godchen a0534d3139
[skip e2e]Sort imports (#14814)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:14:21 +08:00
godchen d29243cbc5
[skip e2e]Sort imports (#14317)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-30 16:49:37 +08:00
guoxiangzhou 1dc4418059
[skip e2e] Modify log text according to naming conventions (#13710)
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2021-12-20 17:42:07 +08:00
congqixia 1dcd06cc78
Add log file info for all kind of segment binlogs (#13606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
edward.zeng dbae087d21
[skip ci] Refine datacoord garbage collector comment (#13430)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-15 18:25:43 +08:00
congqixia 7b870e3392
Limit datacoord gc scan prefixes (#12231)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-24 09:55:15 +08:00
Bingyi Sun 3be9442c0f
Clear dropped segment meta on etcd (#12191)
issue: #9613
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-23 11:23:15 +08:00
congqixia 8c107ab15f
Set segment DroppedAt when compacted (#12182)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-22 20:11:14 +08:00
congqixia b99164f43d
Add datacoord GC configuration (#12061)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-18 22:29:40 +08:00
congqixia 7e43b568ee
Add dropped ts and apply when gc scan dropped segments (#11838)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-16 14:23:21 +08:00
congqixia 3c9610cb1b
Add garbage collector for DataCoord (#11554)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-10 19:03:38 +08:00