Commit Graph

75 Commits (ed94ecf847fbd8a32ec7970146640f9a9bb4f026)

Author SHA1 Message Date
Jiquan Long 875b6f88b0
Add channel information to flow graph node (#17349)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-15 21:04:10 +08:00
yah01 4e83f37899
Improve load performance (#17273)
Load segment's fields concurrently

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-02 17:56:03 +08:00
xige-16 8c69790383
Fix lost delete msg caused by loadSegment after watchDeltaChannel (#17308)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-02 16:56:04 +08:00
bigsheeper f55fe6be3c
Merge historical replica and streaming replica in QueryNode (#17217)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-31 13:42:03 +08:00
congqixia c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
bigsheeper a4ea2fb18a
Handle errors in DataNode and QueryNode flow graph (#17096)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-24 21:11:59 +08:00
zhenshan.cao 08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
Letian Jiang b3eb2b1d0d
Support deltaLog loading on growing segment (#16903)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-05-12 11:59:53 +08:00
Ten Thousand Leaves 216c45fbd6
Add tests where segments load delta logs (#16825)
issue: #16821

/kind improvements

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-10 17:59:52 +08:00
xige-16 515d0369de
Support string type in segcore (#16546)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
2022-04-29 13:35:49 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-24 22:03:44 +08:00
congqixia 69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470)
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
xige-16 27b4cbc098
Cherry pick remove translateHits commit to mater (#16436)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-04-08 20:27:31 +08:00
godchen bb7a0766fe
Add dependency factory (#16204)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
Letian Jiang 09b90fd8bf
Fix QueryCollection retrieve unit test (#16376)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-04-04 21:07:30 +08:00
xige-16 99984b88e1
Support delete varChar value (#16229)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-04-02 17:43:29 +08:00
Jiquan Long ba37531456
Add support for loading multiple indexes (#16138)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-30 21:11:28 +08:00
godchen 478f6ca11e
Add cache for vector chunk (#15912)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-26 22:05:26 +08:00
xiyichan ca129d4308
Support configurable msgstream (#16131)
Signed-off-by: xiyichan <2863768433@qq.com>
2022-03-24 10:15:25 +08:00
Jiquan Long f8d9bc919d
Unify interface of vector index & scalar index. (#15959)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-21 14:23:24 +08:00
godchen 78557ca6d7
Replace minio kv with minio chunk manager (#15936)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00
jaime 97b1ed7bca
Remove reader API from Mqstream and replace it with consumer API (#15971)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-15 14:45:22 +08:00
jaime 29975a7a26
Remove Consume() API from mqstream interface (#15886)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-11 20:09:59 +08:00
bigsheeper 9ad5c14a4a
Return error when create message stream failed in QueryNode (#15317)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-09 15:17:59 +08:00
zhenshan.cao dff08dbf47
Refactor querynode unittest (#15929)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-08 17:39:58 +08:00
bigsheeper 6fec7ed6fb
Add search benchmark in QueryNode (#15854)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-08 10:15:59 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00
Cai Yudong 92c8e32ebd
Let MemoryKV.Load return error when key not exist (#15814)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-02 18:51:55 +08:00
bigsheeper 12eb8c3da5
Add prometheus metrics for QueryNode (#15649)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-02 14:49:55 +08:00
godchen fcf0887d92
Add and implement chunk manager interface. (#15541)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-02-24 23:53:53 +08:00
xige-16 55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-08 21:57:46 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
xige-16 c132302129
Clean flowGraph if watchChannel failed (#15303)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-20 10:01:38 +08:00
xige-16 dfc6670f0f
Unsubscribe channel after query node down (#15230)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-17 17:37:37 +08:00
bigsheeper 0c98f21d4d
Remove global sealed segments in historical (#15177)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-01-12 17:01:38 +08:00
Cai Yudong b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 17:29:34 +08:00
Cai Yudong 7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-09 23:45:33 +08:00
Cai Yudong f8f61fb116
Use BaseParams etcd configuration for all components (#15012)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-07 16:15:26 +08:00
bigsheeper 8a3a1b556a
Free OS memory in QueryNode and IndexNode (#14305)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-12-30 20:03:29 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
congqixia 1dcd06cc78
Add log file info for all kind of segment binlogs (#13606)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
bigsheeper 1ca08d9138
Simplify flow graph in QueryNode (#13613)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-12-17 20:12:42 +08:00
bigsheeper bf6405ca93
Simplify tSafe in query node (#13241)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-12-17 14:41:33 +08:00
XuanYang-cn dbf757e08b
Change genKey into JoinIDPath (#12959)
1. Simplify genKey(), make genKey a rpc call
2. Unify the usage of JoinIDPath

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-09 11:09:06 +08:00
XuanYang-cn 02347baf2d
[skip ci]Update license for qn mock (#12817)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-06 21:10:09 +08:00
zhenshan.cao af22977cd3
Fixbug: grpc connection is closed by mistake (#12307)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-12-03 15:15:32 +08:00
Cai Yudong 1a806a174a
Support load collection with multiple vector column (#12630)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-03 11:29:33 +08:00
godchen 116a503b4a
From dml cp load delete msg (#12144)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-21 07:33:14 +08:00
Xiaofan f654621650
Remove the etcd deletion of segment data path on querynode (#11863)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-11-17 12:13:10 +08:00