milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,811 Commits
139 Branches
115 Tags
577 MiB
Commit Graph

908 Commits (1e7205fa03f14098dcb02e48fa78f7ce230657c5)

Author SHA1 Message Date
sunby ddddd65d10 Delete internal/errors package 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-05 10:15:27 +08:00
bigsheeper 447a15207e Add zap log to query node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-05 09:21:35 +08:00
dragondriver 801396e3ef Disable register link logic, use proxy node directly 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-04 22:27:12 +08:00
xige-16 ebaa5d4f5b Fix error of repeated loading partition failures 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-04 19:21:44 +08:00
sunby 6dc938e6f4 Refactor code 
1. add error as return value
2. check assertion success

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-04 18:58:16 +08:00
FluorineDog ef98dab2a9 Support segcore config 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-04 17:09:48 +08:00
sunby dec0b40a66 Add log panic in log module 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-04 16:01:30 +08:00
yukun 36bae90635 Fix singlenode search timeout caused by selectcase in rmq_msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-04 15:53:17 +08:00
neza2017 ba21b11e9e Fix masterservice :drop index 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-04 14:54:16 +08:00
xige-16 6cdd4d02fc Refactor seek function 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-04 14:49:51 +08:00
godchen 3ba0dc2f44 Change doc 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-04 10:35:28 +08:00
ophunter233 4a921ffd7e Stop listen to dd channel in query node 
Signed-off-by: ophunter233 <chengming.li@zilliz.com>
 2021-03-03 16:06:02 +08:00
sunby ae432436db Init start position when opening new segment 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-02 15:52:42 +08:00
XuanYang-cn c0edffc63f Fix datanode update segment statistics bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-02 15:47:37 +08:00
xige-16 795cb74c85 Fix message stream miss ack 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-02 13:15:49 +08:00
FluorineDog b7a00e30e2 Fix memory leak 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-01 11:14:51 +08:00
bigsheeper 456d0e8c23 Refactor time tick align strategy 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-27 18:33:29 +08:00
FluorineDog 66146223ca Support flat 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-02-27 12:46:37 +08:00
zhenshan.cao 966cf42b54 Enable indexnode to build index asynchronously 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-27 10:45:03 +08:00
bigsheeper d3db2b0d82 Add zap log in query service 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-27 10:11:52 +08:00
BossZou 7893ebdb40 Fix test insert case 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-26 17:59:13 +08:00
godchen 7b9fdd7f29 Add opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-26 17:44:24 +08:00
xige-16 50a5f97421 Delete componentStateResponse in queryService.proto 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-26 15:26:21 +08:00
cai.zhang f5977a1302 Change retry times and add retry logic 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-26 15:17:47 +08:00
neza2017 4ea27335fd Ignore index param from create_collection 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-26 14:40:53 +08:00
zhenshan.cao 007e7ea858 Add tso unittest and EnableMaxLogic 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-26 13:05:52 +08:00
zhenshan.cao 12ea873216 Fix bug: drop empty index should be considered as legal operation 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-26 11:19:10 +08:00
bigsheeper 9edbff0a6b Fix fields check 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-26 11:15:44 +08:00
XuanYang-cn d34e3e77d1 Fix typo 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-26 11:07:25 +08:00
XuanYang-cn 10374f15d7 Update log in datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-26 10:13:36 +08:00
sunby e95cb78ec8 Add LoadCollection from master 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-26 09:23:39 +08:00
neza2017 c36c532c70 Remove unused test code 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-25 19:12:40 +08:00
xige-16 650bf1d993 Refactor release collection warn message 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-25 18:26:11 +08:00
FluorineDog bf75c2fbb4 Fix bug (#1053) 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-02-25 17:56:43 +08:00
Xiangyu Wang 38266b8569 Lock channel at produce and consume 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-25 17:44:29 +08:00
Cai Yudong 84110d2684 Add tests/benchmark and tests/python_test using new python SDK 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-02-25 17:35:36 +08:00
neza2017 d5631b09e8 Describe collection by id 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-25 16:08:56 +08:00
bigsheeper 7cb28428ce Prevent consuming after loading collection 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-25 15:08:50 +08:00
neza2017 4c3486c770 Fix timestamp allocator 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-25 14:31:38 +08:00
yukun a391d4fbbf Fix stuck problem of create_index 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-25 12:03:35 +08:00
cai.zhang 1d899513dd Unify service name in config file 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-24 18:46:19 +08:00
cai.zhang 9be4a107f9 Fix bug for free memory 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-24 18:25:34 +08:00
zhenshan.cao bbc65e0b14 Remove checking of VectorFieldMetricType in create_collection 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-24 18:14:59 +08:00
sunby c149995bcd Change return value of GetCollectionStatistics. 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-24 17:56:03 +08:00
bigsheeper 08a020798e Release collection and partitions 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-24 17:24:51 +08:00
zhenshan.cao 0679954922 Refactor tso and global id allocator 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-24 17:12:06 +08:00
neza2017 e396c3f4ad Add zap log 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-24 16:25:40 +08:00
xige-16 4c491471ee Add release collection and release partition interface for query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-24 15:58:55 +08:00
BossZou b02a9d6b43 Enlarge rocksmq stream buffer size 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-24 15:16:09 +08:00
XuanYang-cn fd562f9f9c Update doc: add proxy graph 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-24 09:48:17 +08:00
zhenshan.cao 7c5e2d89f6 Read node port from config 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 18:08:17 +08:00
zhenshan.cao 6b392cbe58 Fix memory leak 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 14:35:00 +08:00
xige-16 d5379a82a8 Fix load binlog error when segment not flushed 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-23 14:13:33 +08:00
dragondriver b506754364 Add workflow of drop index in index service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-23 11:57:18 +08:00
zhenshan.cao bbfcbbdd68 Fix ci: paramstable and startup logic refactor 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 11:40:30 +08:00
xige-16 7a7a73e89c Fix high memory usage in pulsarTtStream 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-23 11:40:12 +08:00
FluorineDog 15dd17488e Support benchmark 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-02-23 10:47:21 +08:00
sunby ea1d9ea99a Replace log in dataservice 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-23 09:58:06 +08:00
dragondriver 68518fec38 Fix memory leak in indexnode 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-22 18:33:40 +08:00
neza2017 f0c16a228e Fix show segments 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-22 16:40:26 +08:00
yukun e786ad6aa8 Fix inconsistency of produce msg and consume msg with rmq 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-22 16:34:15 +08:00
sunby b07a371f25 Create a goroutine to receive msg for every consumer 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-22 09:58:34 +08:00
xige-16 861576f77a Checkout field ids when load segment in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-22 10:44:38 +08:00
BossZou 7ac1821cf1 Add RocksMQ throughout test 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-20 18:30:37 +08:00
BossZou ae0e8b1a19 Init data service param before new DataServive 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-20 16:45:49 +08:00
cai.zhang 5c8747cf40 Make proxy retry time more frequently 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-20 16:14:10 +08:00
yukun 346b0a9549 Add unittest for RmqTtMsgStream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-20 15:58:19 +08:00
neza2017 1d3fdebcf4 Add drop index for proxy 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-20 15:38:44 +08:00
cai.zhang 4c5a898e9a Add masterservice docker 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-20 11:15:19 +08:00
dragondriver 6ebb03791f Fix drop index interface 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-20 10:37:11 +08:00
zhenshan.cao b3891ca199 Correct docker file 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-20 10:15:37 +08:00
xige-16 f62078c027 Fix crash error when search 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-20 10:14:03 +08:00
xige-16 a6442cb600 Fix querynode memory leak 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-20 09:20:51 +08:00
sunby 5769c000a1 Find failed python tests and add skip mark "r0.3-test" 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-19 15:37:04 +08:00
bigsheeper d8f41db823 Fix search crash error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-19 13:56:01 +08:00
sunby 5b61182140 Add zap log 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-18 19:53:55 +08:00
zhenshan.cao d404d87633 Fix bug of getindexstate 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-19 09:52:06 +08:00
xige-16 450ea631bc Fix queryService assign queryChannel failure 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-18 16:26:02 +08:00
neza2017 549bda93ea Fix get index by name 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-18 15:58:59 +08:00
groot af3736aff6 Fix rocksmq bug 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-18 14:13:55 +08:00
neza2017 85c1404393 Refactor master:index ids 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-11 08:41:59 +08:00
xige-16 70fe5233cf Watch queryChannel on query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-09 17:09:26 +08:00
groot 833f1d49ab Let singlenode use rocksmq 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-09 15:57:10 +08:00
neza2017 e2d8358cb4 Refactor master:index ids 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-09 13:11:55 +08:00
neza2017 c2956dbe92 Change master meta prefix 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-09 10:36:31 +08:00
zhenshan.cao 7901d98d5e Fix bug: return wrong IndexID 0 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-08 18:42:33 +08:00
sunby c74221e4da Change ttMsgStream of dataservice to MsgStream 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-08 16:42:56 +08:00
neza2017 4035083188 Support Index name 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-08 14:49:12 +08:00
sunby 90d69658d9 Fix master meta table bugs 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-08 15:19:48 +08:00
groot dc3736281a Integrate message stream 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-08 14:30:54 +08:00
zhenshan.cao faa23fde86 Fix getindexstate failed to fetch indexstate 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-08 14:20:29 +08:00
dragondriver da94fbed0a Parse to map when the key of index params is named params 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-07 21:32:37 +08:00
bigsheeper 12b2eaf196 Rename apis 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 21:26:03 +08:00
zhenshan.cao 1ba8e2448f Add tsLoop for indexservice 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-07 18:38:58 +08:00
bigsheeper a64c831ed9 Fix segment replacement bug 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 18:37:18 +08:00
bigsheeper 7342e0758c Fix index and flush errors, and fix master crash error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 17:02:13 +08:00
zhenshan.cao 2c9e226703 Add log for segment logic in proxy 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-07 16:44:11 +08:00
xige-16 c7ac238aac Fix error when query node release collection 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-07 15:56:57 +08:00
xige-16 2ca53fa668 Fix msgstream deadlock when loadCollection 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-07 15:47:10 +08:00
yukun 563a8d66ae Add RmqTtMsgStream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-07 15:10:13 +08:00
sunby 6ee6557981 Record segment flushed state 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-07 14:51:52 +08:00
sunby eba5b9ae27 Fix queryservice client init bug 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-07 13:53:40 +08:00
bigsheeper b07b2484dc Fix error when loading 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 10:29:58 +08:00
dragondriver 8f4995dab5 Fix DropCollection task in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-07 09:56:39 +08:00
xige-16 cdcf006690 Fix loadCollection error when query service assign insert channels and query nodes 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-07 09:30:48 +08:00
zhenshan.cao 39161cac9a Add collectonScheam when calling load_collection and load_partition 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-06 21:17:18 +08:00
xige-16 a12b184e02 Add field schema to LoadCollection and LoadPartition photo 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-06 17:37:10 +08:00
bigsheeper e685f1825b Fix unittest caused by param table 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-06 15:16:10 +08:00
groot b034740c6c Make paramstable initialize once by using do_once 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-06 13:39:15 +08:00
yukun 099220394e Add rmq_msgstream unittest and fix bugs 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-06 11:39:12 +08:00
bigsheeper e9ee9a273e Refactor load service and check insertion binLog periodically 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-06 11:35:35 +08:00
sunby 0111cba8ae Fix startup bugs in some modules 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-06 10:28:53 +08:00
XuanYang-cn dd72dfcbd1 Fix queryservice start 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-05 17:57:41 +08:00
godchen c231f6da79 Fix collectionName not found error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-05 16:32:03 +08:00
xige-16 57ba236117 Add mutex for message stream’s asConsumer 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-05 15:42:53 +08:00
dragondriver 7e1887da12 Add start.sh and stop.sh, remove them later 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 14:31:40 +08:00
bigsheeper a7eae3a4c1 Remove consumer in query node's initialization 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-05 14:15:10 +08:00
dragondriver e6aec3fd08 Hardcode the peer id of proxy node 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 14:09:55 +08:00
dragondriver 6ef82a59e5 Fix the bug when close the message stream of InsertTask 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 13:49:51 +08:00
godchen 0f620ab149 Fix invalidate metacache error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-05 11:49:13 +08:00
dragondriver e384222803 Fix the logic of node id allocator 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 11:25:52 +08:00
neza2017 bea1535425 Add call of release collection when drop collection 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-05 11:05:37 +08:00
bigsheeper 1578c13224 Refactor collection replica 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-05 10:53:11 +08:00
dragondriver a48590a2df Add query service logic to Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 10:29:14 +08:00
bigsheeper 2ee54541aa Fix query node address 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-05 10:25:47 +08:00
sunby 9c3ea487f6 Fix close insert msg stream bug 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-05 10:19:52 +08:00
dragondriver 75e549da4e Change sdk version used in regression to 0.0.19 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 10:09:03 +08:00
XuanYang-cn a64c4efde9 Refactor collectionReplica in datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-04 20:31:23 +08:00
sunby be596123d1 Fix dataservice startup 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-04 20:03:19 +08:00
godchen c2ca2c276f Fix response check error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-04 19:34:35 +08:00
cai.zhang 4f2a1e7912 Fix bug for dataservice dockerfile 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-04 18:11:19 +08:00
dragondriver e0e8e1605e Implement the release-related task and load-related task in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-04 17:47:19 +08:00
sunby 7270d18c98 Fix timesync startup bugs 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-04 17:31:02 +08:00
xige-16 aaac839075 Joint debug query service success with other components 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-04 17:09:18 +08:00
XuanYang-cn 560d2350ae Fix datanode watchDmChannel bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-04 16:09:34 +08:00
godchen d6200a5196 Fix meta cache error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-04 15:54:16 +08:00
cai.zhang 55634dc5f3 Add interface of load and release collection and partition 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-04 15:31:02 +08:00
yukun e3fadc45d4 Fix for new msgstream interface 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-04 15:23:21 +08:00
zhenshan.cao b89e5a3240 Add GetQuerySegmentInfo 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-04 14:37:12 +08:00
sunby 71b6c88c6e Defer lock's unlock in meta::DropSegment 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-04 11:52:10 +08:00
Xiangyu Wang 8d35e76ad1 Change Msgstream interface 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-04 13:55:47 +08:00
bigsheeper c6c99ef32b Implement getSegmengInfo function 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-04 11:40:14 +08:00
XuanYang-cn 1aafe86ffe Change MsgPosition Logic 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-04 11:19:48 +08:00
dragondriver b92ff69cea Implement GetIndexState in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-04 11:08:36 +08:00
XuanYang-cn bc7ad02df4 Fix datanode init bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-04 10:25:01 +08:00
godchen 6e7e0b748a Send timetick to insert channel 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-03 20:04:29 +08:00
sunby e7588d3250 Implement getSegmentInfo interfaces 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-03 19:45:52 +08:00
neza2017 1189332786 Create default partition when create collection 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-03 18:41:20 +08:00
zhenshan.cao 1b616fb874 Add persistent_segmentinfo 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-03 18:55:00 +08:00
bigsheeper f12a0490bb Do not drop collections or partitions in flow graph 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 18:12:48 +08:00
Xiangyu Wang fd9e42d3ed Update msgstream/unmarshal interface 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-03 17:30:10 +08:00
bigsheeper abffdbff62 Remove partition tag 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 16:44:38 +08:00
dragondriver 6c3f169ecc Add collection id and partition id to insert & search workflow 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-03 16:40:23 +08:00
zhenshan.cao b853020ef9 Remove redudent ctx and openTracing logic 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-03 16:34:56 +08:00
yukun 35f6f4b7c2 Add consume in rmq_msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-03 15:18:54 +08:00
XuanYang-cn 73a8e7de84 Fix datanode insertchannel refresh bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-03 15:18:05 +08:00
sunby 208700b0e7 Set fixed insert channel number, remove channel manager 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-03 15:07:48 +08:00
bigsheeper 01e9dc8e3f Remove collection name 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 11:52:19 +08:00
sunby f3aad3a71c Change SegmentInfo 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 18:53:10 +08:00
bigsheeper 5e781b9370 Remove field name in query node and segCore 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 10:10:07 +08:00
sunby 45b99c0cf3 Save indexName and indexID in IndexCodec 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 19:56:11 +08:00
neza2017 899d31e72d Change interface: add index id and index name params 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-02 20:09:46 +08:00
zhenshan.cao bfa14d40a8 Rename indexID to indexBuildID and add IndexID and IndexName 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-02 19:56:04 +08:00
godchen 66d5189601 Add get collection statistics implemetation 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-02 19:54:31 +08:00
Xiangyu Wang 32c5a92449 Update interface in msgstream 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-02 16:32:15 +08:00
dragondriver e4e3ec88d1 Add usage count of insert channels 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-02 15:49:13 +08:00
bigsheeper 09bdbee944 Refactor load service, remove seg manager 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-02 15:13:08 +08:00
sunby ca9c2f5c12 Add GetCount and implement GetCollectionStatistics 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 14:25:58 +08:00
xige-16 4cd315408b Refactor query service’s meta interface 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-02 11:52:41 +08:00
sunby 46f14a5e69 Fix bugs in GetInsertChannels 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 11:14:17 +08:00
zhenshan.cao 3afa7dd50d Fix GetSegmentStates 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-02 10:58:39 +08:00
neza2017 5de89422ed Support multi index 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-02 10:09:10 +08:00
XuanYang-cn 90d471b133 Refactor main of queryservice and querynode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-02 09:52:42 +08:00
godchen 93fd6b70f3 Fix the error usage of ttmsg 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-01 19:22:35 +08:00
sunby a74767fe57 Fix bugs 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-01 16:20:13 +08:00
neza2017 3f45cfd6e8 Fix data node 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-01 11:44:02 +08:00
godchen 94dfab5b21 Check collection whether exists before drop 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-01 10:53:13 +08:00
godchen abcc565311 Add proxy metacache 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-31 14:55:36 +08:00
zhenshan.cao c2914dd113 Refactor segment allocator, use collectionID and partitionID 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-31 10:15:54 +08:00
neza2017 854accf95b Remove master and writenode 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-30 16:41:47 +08:00
bigsheeper 3573e01093 Check index periodically and load index, add indexing segment type 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-30 16:02:10 +08:00
dragondriver 996111bf8c Refactor insert channels used in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-30 15:30:38 +08:00
sunby 3a67dda06c Add NewDataService in main 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-30 11:49:48 +08:00
neza2017 51eac0eb84 Use GetCollectionByName instead 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-30 11:38:02 +08:00
dragondriver 454c05f2de Implement the InvalidateCollectionMetaCacheTask in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-30 10:40:22 +08:00
dragondriver 4d5a9e067c Fix the cancel logic of proxy service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-29 19:25:32 +08:00
zhenshan.cao bfb6766f7c Refactor indexservice 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-29 18:12:21 +08:00
dragondriver f660caca0c Refactor the client interface of proxy service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-29 17:29:31 +08:00
Xiangyu Wang 99cef4b5c8 Add Distributed main entrypoint 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-01-29 17:08:31 +08:00
bigsheeper 73d2b6a101 Get index param from minio and filter by vector fields 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-29 15:22:24 +08:00
neza2017 bdf84f08ab Add watcher_test and fix some bugs 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-29 14:40:19 +08:00
sunby f088467dde Add watcher_test and fix some bugs 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-28 10:14:31 +08:00
XuanYang-cn b597a637f3 Add datanode metatable tests 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-29 12:03:10 +08:00
neza2017 305c814215 Fix master 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 19:37:25 +08:00
zhenshan.cao ee0faddb07 Refactor proxy 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-29 09:27:26 +08:00
dragondriver 7957170220 Dock Proxy with other service component 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-28 20:51:44 +08:00
neza2017 3a866dab3d Update proto 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-28 17:25:43 +08:00
neza2017 2ebeade25e Fix get component states 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-28 17:13:00 +08:00
sunby 2be8cc1c4b Add index params serde to IndexCodec 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-28 16:41:24 +08:00
sunby bcd7368533 Fix get component states 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-28 15:20:24 +08:00
sunby 5ac463c9ca Add datanode client in data service 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-28 11:24:41 +08:00
XuanYang-cn 4acaa97039 Add unittests for datanode and queryservice 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-27 20:00:47 +08:00
sunby f1afd5d311 Add loop to update tsoallocator and id allocator in master service 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-27 16:38:18 +08:00
bigsheeper 68cd15af63 Get index info from master 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-27 14:41:56 +08:00
dragondriver a0226c09f0 Change the logic node id allocator in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-27 13:52:01 +08:00
sunby b4bcaaf847 Fix data service startup bug 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-27 11:40:10 +08:00
XuanYang-cn 05b013c52a Fix datanode bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-27 11:37:26 +08:00
xige-16 947976dbec Add test for query service’s load function 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-27 11:34:16 +08:00
sunby dbfe68512d Add dbID to data service proto 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-27 10:33:17 +08:00
sunby a3adb22e10 Change params 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-27 09:59:17 +08:00
bigsheeper 4f914a2c30 Add docker file and main function of query node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-27 09:50:52 +08:00
neza2017 4d37a329da Fix data service init 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 19:25:17 +08:00
neza2017 d1b23cb8c5 Add debug info 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 19:24:09 +08:00
sunby f632ed0272 Fix data service init 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-26 18:45:12 +08:00
zhenshan.cao e83ac41bca Add component for index 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-26 18:01:32 +08:00
neza2017 8c54af292e Add dd handler 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 17:55:15 +08:00
neza2017 c4f4ae8627 Fix master 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 17:47:38 +08:00
XuanYang-cn 0b1c4f0420 Add building datanode cmd in Makefile 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-26 17:40:26 +08:00
sunby 9f72633fd9 Add dd handler 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-26 17:06:13 +08:00
neza2017 2ea6a5f9ee Add make dataservice in Makefile 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 16:41:10 +08:00
sunby b4997a37a2 Add make dataservice in Makefile 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-26 16:36:26 +08:00
neza2017 b6a788373b Fix proxy server 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 16:30:36 +08:00
neza2017 fd0227b69a Add logic of get time tick 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 15:23:42 +08:00
sunby 3b08162059 Add main function 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-26 15:14:49 +08:00
xige-16 7c210310dc Add load function for query service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-26 15:13:20 +08:00
dragondriver ad244dc4e8 Add initial interface to proxy service client 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-26 15:09:31 +08:00
dragondriver d972b75aaa Add GetTimeTickChannel, GetComponentStates to proxy service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-26 14:55:57 +08:00
XuanYang-cn b445587e11 Add datanode main 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-26 14:46:54 +08:00
cai.zhang 346bb1ba49 Delete some unused code and some debug logs 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-26 14:31:31 +08:00
dragondriver c6a6b1436c Add soft time tick in proxy service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-26 13:41:41 +08:00
XuanYang-cn 6c303c67b2 Rearrange update segment statistics of start position procedure 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-26 09:44:39 +08:00
neza2017 5af23cf018 Init params and add segment start/end position 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-26 09:43:41 +08:00
cai.zhang f940cc455a Add dockerfile for index 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-26 09:38:40 +08:00
neza2017 7f3aa92d10 Add docker file for masterservice 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-25 15:58:04 +08:00
FluorineDog a8fd6e7686 Enable drop index/field_data and extract info visitor 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-01-25 18:59:44 +08:00
yukun 5600d06583 Add global_rmq 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-01-25 18:53:08 +08:00
bigsheeper 7554246ace Add segment seeking and use real client 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-25 18:33:10 +08:00
sunby 7c0c835b49 Init params and add segment start/end position 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-25 15:16:28 +08:00
XuanYang-cn 5015a1035d Add logic of get meta from MasterService for datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-25 17:10:32 +08:00
neza2017 1fe099775d Refactor data service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-25 15:17:17 +08:00
neza2017 e89e89b894 Add main for masterservice 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-25 12:03:44 +08:00
sunby 54fcd486bb Refactor data service. 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-25 13:45:56 +08:00
XuanYang-cn 5d9d35b733 Add Supplementary statistics 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-25 13:36:40 +08:00
FluorineDog 69f6963d45 Enable lock and fix row count 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-01-25 11:20:25 +08:00
XuanYang-cn 9fe672ab86 Enchance datanode interface 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-24 21:20:11 +08:00
neza2017 5aec8bc5e6 Refactor masterservice 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-24 20:26:35 +08:00
bigsheeper 445675a588 Refactor loadSegment request and load strategy 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-24 18:02:08 +08:00
zhenshan.cao aef4e41e91 Enable proxynode microbatch pulsar msg and enable parallel test 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-23 20:58:46 +08:00
sunby 82d6fb18b8 Add init params returned to data node 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-23 20:22:59 +08:00
XuanYang-cn b794921b7e Add enum Role in internal.proto 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-23 18:56:08 +08:00