26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
4f6122dd4a
enable ctx traceId for assignsegment on dc( #26972 ) ( #27108 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-09-18 09:53:22 +08:00
d7cd1f2a6d
Fix bug for get index state with compaction segment ( #26822 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-09-04 17:29:48 +08:00
bed034a44b
Protect DataCoord from calculating segment lines by stale log entries num ( #26523 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-08-22 14:16:22 +08:00
3c62acde05
Add coll and segState lebels to DataCoordNumStoredRows monitoring ( #26299 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-08-17 15:52:17 +08:00
7619db1edc
Fast GC compacted dropped segment ( #26207 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-08-09 19:39:15 +08:00
b533c68632
Forbid update checkpoint without msgID ( #25694 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-07-25 10:43:04 +08:00
f4e72cb170
remove sync segmentLastExpire every time when assigning( #25271 ) ( #25316 ) ( #25557 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-07-24 14:11:07 +08:00
95f91e6e35
Fix segments meta not consistent between catalog and in-memory ( #25799 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: czs007 <zhenshan.cao@zilliz.com>
2023-07-21 14:21:05 +08:00
bccdef1ad7
Fix segment meta only record last flushed binlog ( #25707 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-07-19 15:28:57 +08:00
66fdc71479
Refactor logs in DataCoord & DataNode ( #25574 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-07-14 15:56:31 +08:00
b22d6d2a5d
Avoid update all binlogs every a segment is updated ( #25526 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-07-13 14:08:29 +08:00
54acb5b725
Remove alter segment interface ( #25382 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-07-06 20:46:25 +08:00
fe22720ff3
Avoid rewriting all binlogs every time a segment is updated ( #25339 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-07-06 18:20:26 +08:00
33c2012675
Add more metrics ( #25081 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-26 17:52:44 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
19164ea753
Make binlog size metric to collect healthy binlog only ( #24557 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-31 16:45:29 +08:00
fc010ee86b
Improve the metric of stored binlog size to the level of collection and segment ( #23859 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-07 20:52:38 +08:00
b517bc9e6a
refine balance mechanism including:( #23454 ) ( #23763 ) ( #23791 )
...
1. balance granuity to replica to avoid influence unrelated replicas
2. avoid balance back and forth
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-05-04 12:22:40 +08:00
ed8836cd15
Add disk quota at the collection level ( #23704 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-26 21:52:36 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
1287ca699a
Refine usage of TimeRecorder.Record ( #23142 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-03-30 18:56:22 +08:00
4a90490a67
Fix the `segment not found` error ( #22772 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-03-17 17:27:56 +08:00
aa6212d5b0
[Cherry-pick] Fix channel checkpoint issues and fix GC ( #22559 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: Ten Thousand Leaves <69466447+soothing-rain@users.noreply.github.com>
2023-03-09 14:13:52 +08:00
efdc77bce6
Fix not fill timestamp when use collection created ts ( #22629 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-08 18:53:51 +08:00
d126f06946
Decouple mq module from internal proto definition ( #22536 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-04 23:21:50 +08:00
319f1773af
Fix segment not found if forward delete to empty segment ( #22528 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-03 14:13:49 +08:00
3d6bded115
Fix Row Number Mismatch ( #22307 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-02-27 10:41:46 +08:00
fef12b53ef
Fix 2.2.2 possible broken channel cp ( #22205 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-16 16:46:35 +08:00
f9495fef00
Minor fix on segment metric ( #21976 )
...
/kind improvement
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-02-03 19:33:52 +08:00
f1daef22a1
Fix failed bulkload segment marked as sealed ( #21799 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-31 12:41:53 +08:00
9fd9bed2b9
Remove collection meta after GC finished ( #21595 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-01-12 09:55:42 +08:00
8c2c04517e
Fix an issue where flushed segments can turn into sealed ( #21566 )
...
/kind improvement
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-01-11 18:57:43 +08:00
73d8509600
Improve ListSegments performance for datacoord catalog ( #21477 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-01-06 14:33:36 +08:00
e5f408dceb
Merge IndexCoord and DataCoord ( #21267 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-01-04 19:37:36 +08:00
5d044c4f29
Add check for stats log and bin log number ( #21264 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-12-26 16:35:30 +08:00
89b810a4db
Refactor all params into ParamItem ( #20987 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
3cde5b53ad
Fix a panic issue while reporting metrics ( #20776 )
...
issue: #20771
/kind bug
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-11-24 17:37:13 +08:00
6d9d24b4ca
Make memory and etcd data as consistent as possible ( #20683 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-11-24 10:19:12 +08:00
a8c7199a52
Fix segment_num, stored_rows_num and stored_rows_count metrics ( #20704 )
...
TODO: better track segment with state other than flushed and dropped.
/kind bug
issue: #20146 , #20145
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-11-22 19:21:13 +08:00
2390095232
Fix load uses compacted segments' binlogs ( #20655 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-11-17 20:37:10 +08:00
e73bdbe17d
Disk quota only checks healthy segments ( #20574 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-14 21:03:09 +08:00
fc15789da9
Ensure compatibility of channel seek position and move syncPeriod to config ( #20504 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-12 21:09:04 +08:00
cd19d99ad7
Add channel level checkpoint ( #20350 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-10 22:13:04 +08:00
9c6436d72d
Add time log for methods of starting the node ( #20313 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-11-08 20:13:03 +08:00
5708352a80
Improve DataCoord and DataNode logs ( #20265 )
...
/kind improvement
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-11-03 12:15:35 +08:00
8d47356aa9
Use compaction generated segment's checkpoint as compacted segment's seekPosition ( #20240 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-11-01 21:01:34 +08:00
2bfecf5b4e
Refine bloomfilter and memory usage ( #20168 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-31 17:41:34 +08:00
11efa0bb5f
Fix compacted segment not dropped ( #20128 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-27 17:15:32 +08:00
def5972e01
fix duplicated handoff request ( #20108 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-27 15:15:32 +08:00
SimFG
MrPresent-Han
cai.zhang
yah01
yihao.dai
zhenshan.cao
aoiasd
Jiquan Long
Enwei Jiao
Bingyi Sun
xige-16
congqixia
jaime
wayblink
Xiaofan
Ten Thousand Leaves