ee9d634a43
Compaction requires the source segments indexed ( #19368 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-22 18:34:50 +08:00
71d4a32ce6
Fix deadlock ( #19340 )
...
The IsParentDropped() acquires read lock of meta,
which leads to recursively acquire read lock
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-22 15:48:50 +08:00
5141e05c47
Add policy to seal long time idle segment ( #19222 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-09-20 20:54:50 +08:00
0306f71b49
Remove incorrect formatting, unused conditions ( #19239 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-20 19:16:49 +08:00
d7f38a803d
Separate some proto files ( #19218 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
11b352c3ae
Guarantee flushed segments are all indexed ( #19062 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-09-16 11:32:48 +08:00
e65ba6331e
Only store segmentID for flushed segments event ( #19075 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-09-09 15:52:35 +08:00
3927ae9952
Return all flushed segments and time-of-seal in Flush interface ( #18882 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-09-09 09:58:37 +08:00
362c663b22
Remove deprecated inner compaction type ( #19024 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-09-07 19:16:34 +08:00
a5e2d6b6fb
Refactor RootCoord ( #18930 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
2022-09-05 13:29:11 +08:00
29c72cad15
Use segment statslog instead of separate statslog ( #18775 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-02 10:34:59 +08:00
43c14055f6
Fix test compact logic and data race ( #18932 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-31 12:10:57 +08:00
096149e33f
fix GetCompactionState hang ( #18829 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-26 14:22:55 +08:00
ef9098f84a
Refine save dropped segments into kv metastore ( #18779 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-08-25 16:48:53 +08:00
c924f73105
Refactor for IndexCoord to support cloud ( #18643 )
...
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
6fd83464cd
Fix watchDmChannel may be out-of-date after compaction issue and add context ( #18681 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-08-23 17:44:56 +08:00
d9c7519107
refactor compaction concurency logic ( #18660 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-23 15:50:52 +08:00
641afae22f
Watch channels with start positions ( #18744 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-08-22 14:42:52 +08:00
70254c3ec5
Unified catalog interface for segment ( #18289 ) ( #18290 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-08-20 10:24:51 +08:00
fc42ee5bb3
Fix # of rows of recovering segment ( #18736 )
...
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-08-19 19:50:50 +08:00
4c86cc63ba
Fix DN flowgraph delete buffer logs ( #18657 )
...
1. Remove logs about not existing segments.
2. Group logs by timestamp.
3. Log changed segments only.
4. Pair the segments reference lock and unlock log by taskID.
Resolves : #18655
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-08-15 16:28:48 +08:00
8a70f2badb
Add ShowConfigurations for all remaining session ( #18594 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2022-08-12 13:20:39 +08:00
0e95f8a690
Refine indexcoord logs ( #18589 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-08-11 10:00:38 +08:00
efa5dfaa7b
Fix Parse binlog path failure ( #18583 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-08-10 17:04:37 +08:00
12023e8569
Fix a potential nullptr issue with metric ( #18540 )
...
/kind improvement
related to: #18530
but not the root cause
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-08-09 21:12:42 +08:00
ff0200210a
Support Role-Based Access Control ( #18425 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-04 11:04:34 +08:00
28aadc988d
Use vchannel name in datanode subName ( #18519 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-03 17:10:36 +08:00
c9174d55ba
Refine merge operation during compacting phase ( #18399 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-07-28 14:52:31 +08:00
e9d777b336
Fix ParseSegmentIDBinlog panicks with bad input ( #18413 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-26 19:32:30 +08:00
d90313a1a8
Use chunkManager rather than minio.Client in datanode garbage collection ( #18092 )
...
Signed-off-by: xingzhao <xing.zhao@zilliz.com>
Co-authored-by: xingzhao <xing.zhao@zilliz.com>
2022-07-22 22:10:28 +08:00
cd2f64d30c
Fix a nil pointer issue ( #18343 )
...
/kind bug
issue: #18270
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-07-20 17:56:30 +08:00
0ebe407221
Make DataNode not panic if stale segment is not found during flush ( #18322 )
...
/kind improvement
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-07-20 15:06:30 +08:00
638f6c36e9
Support to get real row count of segment ( #18115 )
...
Signed-off-by: xaxys <zheng.guan@zilliz.com>
2022-07-18 09:58:28 +08:00
6d82ef8c20
fix garbage collector err handling ( #18277 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-07-15 13:54:26 +08:00
e5fe46124b
Fix isCompacting flag lost after segment clone ( #18248 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-07-13 10:08:28 +08:00
72910d65e0
Fix inconsistency in DataCoord ( #18170 ) ( #18171 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-08 13:48:21 +08:00
43ae6791e6
Make VChannelInfo read compatible with 2.0.2 ( #17995 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-07-07 21:58:22 +08:00
0ad15a22c8
IndexCoord handle events correctly ( #17878 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-07-07 14:44:21 +08:00
d4a1e94f32
Fix flush failure caused by empty stale segments ( #18097 )
...
/bug
issue: #18096
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-07-06 18:10:21 +08:00
9bd35dd84e
Refine logs to reduce print ( #17759 ) ( #18028 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-07-04 11:10:20 +08:00
0baeb609dc
Fix bugs in monitoring metrics ( #17894 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-06-30 17:26:19 +08:00
ef6859a4d6
Refactor handoff handler ( #17806 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-29 23:26:19 +08:00
ff3c4ab2a8
Fix compaction panic ( #17840 )
...
See also: #17823
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-29 16:54:17 +08:00
f0b036a35a
Move bulk load segment lock happen early ( #17612 )
...
issue: #17600
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-27 13:56:17 +08:00
230404499d
Check duplication before adding channel info in DataCoord ( #17742 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-24 16:30:15 +08:00
05273309e8
Drop empty sealed segment ( #17708 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-06-24 15:16:14 +08:00
21b52bfcf2
Enhance channel manager ut ( #17682 )
...
Resolves : #17676
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-06-23 18:08:14 +08:00
50d5d04552
Refine some codes about the datanode ( #17720 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-23 09:58:14 +08:00
6fdf88f452
Fix compaction selection policy ( #17690 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-22 19:00:14 +08:00
746aeea35b
fix typo ( #17668 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-06-21 18:54:13 +08:00
yah01
Jiquan Long
SimFG
cai.zhang
wayblink
jaime
congqixia
Enwei Jiao
codeman
Ten Thousand Leaves
XuanYang-cn
aoiasd
xing.zhao
Jeng.Gwan
Xiaofan
zhenshan.cao
bigsheeper
Bingyi Sun