milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
15,142 Commits
136 Branches
115 Tags
566 MiB
Commit Graph

106 Commits (b923994b7865599c54a9aa24f715525bad804baa)

Author SHA1 Message Date
zhenshan.cao 9bd35dd84e
Refine logs to reduce print (#17759) (#18028) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-04 11:10:20 +08:00
bigsheeper 6b79d4f7bc
Log segment type as possible in QueryNode (#17846) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-28 10:16:17 +08:00
congqixia 68d385137a
Fix: Miss channel info when load growing segment (#17789) (#17797) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: zhagnlu <1542303831@qq.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-06-27 12:02:16 +08:00
yah01 a38790f586
Predict the loading memory usage more accurate (#17451) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-09 18:16:07 +08:00
yah01 70f8bea4b4
Avoid growing slice as deserializing binlogs (#17421) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-08 11:46:06 +08:00
yah01 4e83f37899
Improve load performance (#17273) 
Load segment's fields concurrently

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-02 17:56:03 +08:00
bigsheeper f55fe6be3c
Merge historical replica and streaming replica in QueryNode (#17217) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-31 13:42:03 +08:00
xige-16 e196caf604
Force reclamation of memory after loading (#17244) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-27 12:50:00 +08:00
xige-16 56778787be
Reverse data from scalar index (#17145) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-26 14:58:01 +08:00
zhenshan.cao 08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-23 16:41:58 +08:00
Xiaofan 338edcd3cc
Load binlog for different field in parallel (#17005) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-15 16:11:54 +08:00
Letian Jiang b3eb2b1d0d
Support deltaLog loading on growing segment (#16903) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-12 11:59:53 +08:00
Ten Thousand Leaves 216c45fbd6
Add tests where segments load delta logs (#16825) 
issue: #16821

/kind improvements

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-10 17:59:52 +08:00
Xiaofan 92b6293be4
Fix QueryNode log level (#16604) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-07 10:27:51 +08:00
xige-16 515d0369de
Support string type in segcore (#16546) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 13:35:49 +08:00
jaime 68b1b82faf
Remove DataKV interface (#16692) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-28 21:03:47 +08:00
yah01 80ae6de323
Add comments for concurrency package (#16654) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-26 11:33:45 +08:00
yah01 7af02fa531
Improve load performance, load binlogs concurrently per file, deserialize binlogs concurrently per field/segment (#16514) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-25 15:57:47 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-24 22:03:44 +08:00
xige-16 99984b88e1
Support delete varChar value (#16229) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-04-02 17:43:29 +08:00
Jiquan Long ba37531456
Add support for loading multiple indexes (#16138) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-30 21:11:28 +08:00
bigsheeper 7fee4743f5
Improve log print in OOM error info (#16170) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-24 20:17:25 +08:00
yah01 63a71e247a
Loading segments concurrency level fallbacks to the value that won't cause OOM (#16172) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-24 13:39:25 +08:00
godchen 78557ca6d7
Replace minio kv with minio chunk manager (#15936) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-17 18:03:23 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00
jaime 97b1ed7bca
Remove reader API from Mqstream and replace it with consumer API (#15971) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-15 14:45:22 +08:00
yah01 4f3d5fb6eb
Make load operation parallel improve performance (#15928) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-11 14:28:01 +08:00
顾炎 4f8c08f912
fix checkSegmentSize overflows int64 (#15858) 
Signed-off-by: guyan <632722781@qq.com>
 2022-03-09 13:29:58 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-03 21:57:56 +08:00
bigsheeper 12eb8c3da5
Add prometheus metrics for QueryNode (#15649) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-02 14:49:55 +08:00
bigsheeper 6a48071a5c
Set BaseMsg for LoadSegmentRequest, prevent crashing in log printing (#15725) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-02-26 20:09:54 +08:00
xige-16 55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-02-08 21:57:46 +08:00
Cai Yudong b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 17:29:34 +08:00
godchen 9e51591b3a
Fix pulsar reader not close (#14543) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-04 14:45:19 +08:00
XuanYang-cn 3d24007efa
Add more logs for load sealed segment (#14079) 
Add logs before&after flowgraph_ddnode forwarding delete msg

See also: #13879

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-24 10:50:07 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
congqixia 1dcd06cc78
Add log file info for all kind of segment binlogs (#13606) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-19 20:00:42 +08:00
bigsheeper 5e5787ddd2
Update memory util in metricinfo (#13320) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-16 17:25:13 +08:00
XuanYang-cn b1ec235644
[skip ci]Update license for qn segment (#13193) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-10 20:59:42 +08:00
xige-16 8b2cd271c7
Improve querynode log (#13158) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-10 18:51:08 +08:00
congqixia 8f87cc7ef1
Fix JoinIDPath logic problem (#13039) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-09 12:05:04 +08:00
Cai Yudong 1a806a174a
Support load collection with multiple vector column (#12630) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-03 11:29:33 +08:00
godchen c4f0837d84
Fix insert pks overwrite (#12365) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-29 22:01:41 +08:00
godchen f31ed089b5
Add timeout for reader next (#12308) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-26 22:45:24 +08:00
godchen e3f49858c5
Add subscription prefix (#12251) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-24 17:47:15 +08:00
godchen 3fa5b4a9ce
Delete for segment indexing (#12197) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-23 17:17:16 +08:00
godchen 9d5bcd3e3a
Close event and binlog reader (#12173) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-22 17:27:14 +08:00
godchen 02bf77aabf
Close reader stream (#12167) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-22 16:17:14 +08:00
godchen 116a503b4a
From dml cp load delete msg (#12144) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-21 07:33:14 +08:00
Xiaofan f654621650
Remove the etcd deletion of segment data path on querynode (#11863) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-17 12:13:10 +08:00