milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,237 Commits
149 Branches
122 Tags
579 MiB
Commit Graph

1174 Commits (22b2b7dbbe0e34376100c7dbd008f231392ecbb0)

Author SHA1 Message Date
XuanYang-cn 6deba1faa5 Add license in datanode package 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-04-19 15:16:33 +08:00
zhenshan.cao b1f664d04f Add copyright 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-04-19 15:15:33 +08:00
cai.zhang 8a037a404e Add Copyright for indexnode and indexservice 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-04-19 13:50:12 +08:00
bigsheeper 71b8f309b2 Add licence to query node and query service 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-19 13:47:10 +08:00
Xiangyu Wang c731e2df78 Add license header in util 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-19 13:42:47 +08:00
sunby 0f2f46f39d Add copyright 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-19 11:35:38 +08:00
godchen 0dfcb90881 Add storage copyright 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-19 11:32:24 +08:00
xige-16 51b2b45e03 Add copyright for msgStream 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-04-19 11:30:19 +08:00
FluorineDog 6059558698 Add license files 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-04-19 11:16:16 +08:00
yukun 89c4bf82a0 Add copyright in kv module 
Signed-off-by: kun yu <kun.yu@zilliz.com>
 2021-04-19 11:13:52 +08:00
neza2017 83647e8503 Add copyright in master service module 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-04-19 11:12:56 +08:00
godchen a5ad70a5ab Add unittest for storage 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-19 10:36:19 +08:00
Cai Yudong a2041abab4 improve code coverage for datanode 
improve code coverage for collection_replica_test.go
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-04-19 10:34:56 +08:00
dragondriver dc7db73251 Add copyright annotations to proxynode and proxyservice 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-04-19 10:09:43 +08:00
bigsheeper 08c5134286 Fix collection cannot be found when searching 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-16 17:37:50 +08:00
sunby 94c3fc9321 Add unit tests in dataservice 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-16 16:30:55 +08:00
cai.zhang a54097708d Adjust the DropIndex logic of the indexservice 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-04-16 15:37:13 +08:00
bigsheeper 9e7559b865 Make query node asynchronously load and release collection 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-16 14:40:33 +08:00
FluorineDog ce7a5ea699 Support segcoreinit 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-04-16 14:02:49 +08:00
xige-16 78155d3959 Refactor load Release to async call in query service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-04-15 15:15:46 +08:00
XuanYang-cn e281f6a4a1 Remove useless configs in config files 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-04-13 10:04:39 +08:00
sunby a7dac818ee Refactor datanode register 
Dataservice should not be stalled during registering datanode.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-13 09:47:02 +08:00
godchen deba964590 Change name of kvBase 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-12 18:09:28 +08:00
sunby 61edc53faf Refactor data service segment allocator 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-12 16:35:51 +08:00
neza2017 e760605187 Add unit test for master service, the coverage of meta table reachs 100% 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-04-12 15:03:23 +08:00
Cai Yudong 687d48b75e Improve code readibility 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-04-12 14:11:06 +08:00
godchen 8204546bcd Add unit test for binlog module 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-12 14:07:46 +08:00
bigsheeper 39458697c7 Use asynchronous functions of load and release 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-12 09:18:43 +08:00
neza2017 15f26f653e Add unit test for master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-04-10 10:53:58 +08:00
xige-16 22eda19bc4 Fix mem leak when load index 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-04-10 10:10:53 +08:00
neza2017 56d367b1e9 Add unit test for master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-04-09 16:10:12 +08:00
Cai Yudong 952f70af2a Clean makefile 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-04-09 14:07:03 +08:00
sunby 751c9d4e81 Refactor meta.go 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-09 09:55:04 +08:00
yukun 964e7e0a28 Add unittest for mqclient 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-04-08 20:05:33 +08:00
neza2017 8dd3e1e28f Add unit test for masterservice 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-04-08 17:31:39 +08:00
dragondriver 57831b9978 Add unittest to insertChannelsMap in proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-04-08 15:41:28 +08:00
neza2017 a250eb370f Check request type and add unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-04-08 15:26:18 +08:00
sunby 6e70ce3f66 Add cluster unit tests 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-08 15:08:34 +08:00
sunby 1d7195e036 Add minio and etcd stats watcher 
Our system needs to watch the performance of etcd and minio. For now I just add the creation stats.

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-08 14:14:43 +08:00
XuanYang-cn 46d27e37d7 Skip install gtest when build with test 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-04-08 10:01:18 +08:00
dragondriver 846fd8a154 Add unittest to conf adapter in proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-04-08 09:51:04 +08:00
xige-16 332b14c915 Delete querynode’s redundant load IO 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-04-07 18:29:19 +08:00
FluorineDog 1446cd5453 Fix flat unsupported bug 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-04-07 10:39:35 +08:00
dragondriver bfb7ca0df6 Add unittest to time tick logic in proxy service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-04-07 10:06:17 +08:00
dragondriver decc80a525 Add unittest to proxy service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-04-06 14:12:57 +08:00
bigsheeper fe3fa8dd1d Fix search service unittest 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-01 15:39:23 -05:00
Xiangyu Wang 87a1e0b662 Reorganize msgstream 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-02 13:48:25 +08:00
Xiangyu Wang 8b62790017 Reorganize mqclient 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-02 10:01:11 +08:00
neza2017 d7f5c6b1ef Add benchmark for timtick 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-04-01 18:05:43 +08:00
yukun cdc96d1c32 Add unittest for msgstream_impl 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-04-01 17:29:21 +08:00
zhenshan.cao c6950eb7eb Add log for allocator 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-04-01 13:37:18 +08:00
neza2017 a532444da7 Add log when alloc time stamp failed 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-04-01 11:25:30 +08:00
dragondriver 6796c7bfe7 Add unittest to function util 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-31 18:22:17 +08:00
xige-16 c10b36f188 fix getting wrong offset when segment preinsert 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-31 16:16:58 +08:00
yukun 18e0581d5c Destroy readOptions after use 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-31 14:10:07 +08:00
zhenshan.cao 7bad0a01e9 Fix bug: unmarshal got unexpected EOF 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-31 13:55:51 +08:00
yukun 604f96ed2b Replace consumers map by sync.map 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-31 10:05:24 +08:00
xige-16 a6f1de036b Optimize search performance in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-30 22:16:58 +08:00
zhenshan.cao efbf4452c5 Add log for port changed or proto unmarshal error 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-30 20:19:30 +08:00
godchen 8f58d5641b Perfect trace and add trace init 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-30 16:36:34 +08:00
Xiangyu Wang 8d66388b8c Add logs for querynode 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-30 15:00:14 +08:00
xige-16 e823b56040 Add retry for seek function 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-30 13:53:49 +08:00
Xiangyu Wang 72917467ec Change singlenode logs output 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-30 10:52:42 +08:00
XuanYang-cn 91f5c48037 Fix datanode goroutine leak 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-30 09:47:27 +08:00
sunby f8e879b423 Fix high cpu usage in singlenode 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-29 17:48:15 +08:00
sunby ff8ebd5f42 Fix bug of estimating record size 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-29 15:14:51 +08:00
neza2017 71614313b0 Fix master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-29 09:50:52 +08:00
neza2017 536a0e33d6 Fix meta table, dead lock 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-27 19:46:00 +08:00
yukun c4f9a6f22b Fix get position wrong 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-27 17:39:12 +08:00
sunby a78cd3e2a7 Add proxy service timetick loop to expire allcoations 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-27 14:01:52 +08:00
XuanYang-cn 35656ea9ce Update pulsar client version and fix cannot close tcp connection error 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-27 13:58:52 +08:00
cai.zhang c4a58d259a Add deploy mode env variable 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-03-27 11:56:57 +08:00
Xiangyu Wang 5ccf0c3d70 Add log for segment row estimate 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-27 11:35:42 +08:00
xige-16 3ab8b9af25 Dlete unlocked mutex in seek function 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-27 11:16:20 +08:00
xige-16 a48d46cfaf Alter msgID’s type to bytes 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-27 09:46:54 +08:00
yukun 432b53149b Refactor msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-26 20:10:11 +08:00
bigsheeper 8a4c6a039f Add search log 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-25 18:09:47 -05:00
xige-16 7e60460aee close timeTickStream in flow graph when the collection is released 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-26 18:40:04 +08:00
bigsheeper 1c274de3e0 Improve mutex in queryservice 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-25 16:09:16 -05:00
bigsheeper dfb68a7e46 Remove unused segment 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-25 16:08:08 -05:00
FluorineDog f39dcdb8f3 Support error code in segcore 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-26 16:18:30 +08:00
neza2017 eca1e25470 Add msg id at master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-26 15:13:33 +08:00
dragondriver 41e1975611 Parallelize the processing of loading binlog and saving index files 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-26 11:19:02 +08:00
cai.zhang 30cc84b164 Add more log for indexservice and indexnode 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-03-26 10:01:08 +08:00
yukun 85bc2a54f0 Fix load twice hang bug 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-25 18:49:41 +08:00
bigsheeper 1e4afda824 Add mutex for query node map, search buffer and dataSyncService 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-24 15:40:59 -05:00
quicksilver 7ad9b36207 Update reviewdog/action-hadolint github action version to v1.16.1 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-25 14:41:46 +08:00
dragondriver 1b743e5c6c Parallelize the processing of SearchTask.PostExecute 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-25 10:14:09 +08:00
sunby ca3187d173 Fix data race in datanode tests 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-24 16:35:38 +08:00
sunby 972dad4d8e Add Flush segment optimization 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-24 14:43:38 +08:00
zhenshan.cao 21ffc660b0 Get rid of memory panic 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-23 19:55:00 +08:00
bigsheeper 350e0df5f7 Refactor search service 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-22 16:59:46 -05:00
XuanYang-cn 51f669f1cb Flush segments concurrently & change etcd meta 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-23 18:50:13 +08:00
dragondriver 9d062b54ee Add benchmark for indexbuilder, refactor test utils 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-23 18:44:57 +08:00
zhenshan.cao 9c574b7c7b Update sdk version to 0.0.55 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-23 18:40:08 +08:00
紫晴 25f3e520f9 Add testcase for index 
Signed-off-by: 紫晴 <ting.wang@zilliz.com>
 2021-03-23 16:57:59 +08:00
bigsheeper 1634d75980 Release flowgraph properly 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-22 12:49:50 -05:00
紫晴 3be6dde459 Add smoke tag 
Signed-off-by: 紫晴 <ting.wang@zilliz.com>
 2021-03-23 14:35:30 +08:00
dragondriver 49c6eeb052 Add useful debug log in proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-22 19:28:43 +08:00
xige-16 1a1e864709 Fix query node crash when empty search 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-22 19:20:56 +08:00
zhenshan.cao c2734fa55f Fix bug and enchance system 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-22 16:36:10 +08:00
sunby f4d055cf42 Add global mutex for segment allocate 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-20 20:41:39 +08:00
XuanYang-cn 4c98f4efbf Update binlog meta proto 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-20 09:50:42 +08:00
Xiangyu Wang 9ba9bf5e8a Release memory after rocksdb.write() 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-19 20:20:32 +08:00
groot 2280791128 Add memory message stream for search 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-03-19 20:16:04 +08:00
yukun 0823382876 Refactor rmq_msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-19 19:33:21 +08:00
bigsheeper ba65c38d40 Fix Deadline Exceeded 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-19 17:50:26 +08:00
dragondriver 7554b8afe7 Add default nbits when build ivf-pq index 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-19 16:37:13 +08:00
zhenshan.cao 636755827f Fix unittest bug: query node num 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-17 14:19:14 +08:00
FluorineDog 2cec04ed90 Fix empty schema proto hack 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-17 11:35:28 +08:00
yukun 207f6ae773 Fix rocksmq CreateConsumerGroup 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-16 21:37:25 +08:00
XuanYang-cn 32660d547b Add endPositions in datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-16 17:55:42 +08:00
neza2017 1d105ae451 Tag test case of test_connect.py::TestConnect::test_close_repeatedly 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-16 11:19:56 +08:00
xige-16 b92ad3edd6 Fix error when search on collection without partitions 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-15 19:58:52 +08:00
zhenshan.cao ec8ad89900 Delete uncessary print 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-15 19:47:13 +08:00
neza2017 9e0fc0b169 Fix ts loop 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-15 15:45:17 +08:00
neza2017 6eb45c92eb Fix flush segment 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-13 17:05:36 +08:00
groot 47d9471498 Change gracefultime to 1 second 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-03-13 14:42:53 +08:00
sunby 7bb806b5f6 Log msgID and role when log collection 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-13 11:59:24 +08:00
Xiangyu Wang fc6472eb08 Implement RocksMQ client 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-13 11:46:50 +08:00
dragondriver dd20ab44b8 Add interface of check param util 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-13 09:57:22 +08:00
cai.zhang 012f5f360e Add init log logic for index and proxy 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-03-12 19:47:37 +08:00
xige-16 8a5c039137 Fix search error when running single node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-12 19:23:06 +08:00
FluorineDog e33d0a797c Migrate knowhere to segcore 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-12 15:54:59 +08:00
godchen f3649f0419 Refactor interface and proto 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-12 14:22:09 +08:00
XuanYang-cn 7dd531d159 Fix release collection bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-11 18:42:25 +08:00
sunby 6287ffa1b4 Replace log with zap 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-11 14:15:18 +08:00
ThreadDao ca916a5c47 Add ci nightly 
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2021-03-11 14:14:29 +08:00
dragondriver 0f2a377b62 Delete redundant insert log 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-11 11:27:42 +08:00
xige-16 d35d316056 Close seek function to pass regression test 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-11 10:21:24 +08:00
xige-16 b2b39f5720 Add field position to watchDmChannelRequest’s porto 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-11 10:08:46 +08:00
xige-16 09ae985daa Fix wrong error code in master_service_test.go 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-10 22:06:22 +08:00
neza2017 aee7c74762 Fix override index name 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-10 16:21:49 +08:00
sunby 800e31ebe4 Fix grpc startup in query node 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-10 15:49:23 +08:00
del-zhenwu b1763c3690 Disable xdist in test cases 
Signed-off-by: del-zhenwu <zhenxiang.li@zilliz.com>
 2021-03-10 15:27:26 +08:00
XuanYang-cn 8567679888 Check enableIndex before load index 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-10 14:51:00 +08:00
neza2017 7b2d67242d Fix describe segment if index not exist 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-10 14:45:35 +08:00
Xiangyu Wang c875fdf913 Add logs in rocksmq 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-10 10:02:55 +08:00
cai.zhang 60b97f9b65 Replace log of indexnode and indexservice 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-03-10 09:56:09 +08:00
yukun 0d499a9f99 Fix search waiting for seconds in singlenode 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-09 17:42:13 +08:00
cai.zhang 408fe6e7ad Make system return success when partitions has been already loaded 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-03-09 16:23:41 +08:00
dragondriver af32f442bb Split big insert message into serveral smaller 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-09 16:18:58 +08:00
FluorineDog af1900b42a Remove ConcurrentBitsetPtr in segcore 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-09 16:16:43 +08:00
quicksilver 455cc59dbb Update publish-test-images.yaml 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-09 13:55:35 +08:00
cai.zhang aa5ecbdc02 Use the project log instead of standard log of proxyservice and proxynode 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-03-08 19:39:36 +08:00
sunby 2a083328ee Remove redundant mutex 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-08 18:04:09 +08:00
Cai Yudong c795cdbe2a Change minSegmentSizeToEnableIndex default to 1024 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-03-07 13:37:25 -06:00
sunby 76480fcefe Change default log output to stdout 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-08 16:24:55 +08:00
neza2017 55fc3f56aa Set grpc max recv mag as math.MaxInt32 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-08 15:49:42 +08:00
neza2017 014c4fe8ce Add config of minSegmentSizeToEnableIndex 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-08 15:46:51 +08:00
sunby ab21115d54 Init rand.Seed 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-08 15:25:55 +08:00
XuanYang-cn 403212c5fa Fix describe_index test cases 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-08 15:00:08 +08:00
FluorineDog ec0407d5f1 Support schema verifier 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-08 12:41:46 +08:00
sunby aeb3038545 Remove GetCount in data service 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-08 10:49:44 +08:00
ThreadDao 7a3223dc81 tags collection stats 
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2021-03-08 10:09:48 +08:00
neza2017 3ef18e86f0 Add 0331 tag for test_partition 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-06 17:47:11 +08:00
neza2017 efa7306ef6 Remove with prefix 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-06 16:00:41 +08:00
zhenshan.cao ce5d7c7ecc Change retry times and sleep duration 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-06 11:52:43 +08:00
xige-16 f6a4bacbeb Delete field fieldIDs from ReleaseSegmentRequest’s photo 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-05 20:47:43 +08:00
dragondriver c68a8d35bf Remove timeout controlled by server, use timeout client passed instead 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-05 20:43:19 +08:00
neza2017 2835bcdc4e Update masterservice unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-05 20:41:34 +08:00
Xiangyu Wang e5d595564a Add log for singlenode 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-05 18:16:50 +08:00
yukun 33d8e13f96 Fix load-collection in singlenode 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-05 18:14:59 +08:00
quicksilver 74154a11a4 Fix deploy error during regression stage 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-05 16:52:45 +08:00
XuanYang-cn a5c6f40c60 Enable tests in test_index.py 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-05 15:21:33 +08:00
BossZou 1efcf605e2 Enlarge Proxy node max size of reveivable grpc message 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-03-05 11:16:23 +08:00
sunby ddddd65d10 Delete internal/errors package 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-05 10:15:27 +08:00
bigsheeper 447a15207e Add zap log to query node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-05 09:21:35 +08:00
dragondriver 801396e3ef Disable register link logic, use proxy node directly 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-03-04 22:27:12 +08:00
xige-16 ebaa5d4f5b Fix error of repeated loading partition failures 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-04 19:21:44 +08:00
sunby 6dc938e6f4 Refactor code 
1. add error as return value
2. check assertion success

Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-04 18:58:16 +08:00
FluorineDog ef98dab2a9 Support segcore config 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-04 17:09:48 +08:00
sunby dec0b40a66 Add log panic in log module 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-04 16:01:30 +08:00
yukun 36bae90635 Fix singlenode search timeout caused by selectcase in rmq_msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-04 15:53:17 +08:00
neza2017 ba21b11e9e Fix masterservice :drop index 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-04 14:54:16 +08:00
xige-16 6cdd4d02fc Refactor seek function 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-04 14:49:51 +08:00
godchen 3ba0dc2f44 Change doc 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-04 10:35:28 +08:00
ophunter233 4a921ffd7e Stop listen to dd channel in query node 
Signed-off-by: ophunter233 <chengming.li@zilliz.com>
 2021-03-03 16:06:02 +08:00
sunby ae432436db Init start position when opening new segment 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-02 15:52:42 +08:00
XuanYang-cn c0edffc63f Fix datanode update segment statistics bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-02 15:47:37 +08:00
xige-16 795cb74c85 Fix message stream miss ack 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-02 13:15:49 +08:00
FluorineDog b7a00e30e2 Fix memory leak 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-01 11:14:51 +08:00
bigsheeper 456d0e8c23 Refactor time tick align strategy 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-27 18:33:29 +08:00
FluorineDog 66146223ca Support flat 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-02-27 12:46:37 +08:00
zhenshan.cao 966cf42b54 Enable indexnode to build index asynchronously 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-27 10:45:03 +08:00
bigsheeper d3db2b0d82 Add zap log in query service 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-27 10:11:52 +08:00
BossZou 7893ebdb40 Fix test insert case 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-26 17:59:13 +08:00
godchen 7b9fdd7f29 Add opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-26 17:44:24 +08:00
xige-16 50a5f97421 Delete componentStateResponse in queryService.proto 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-26 15:26:21 +08:00
cai.zhang f5977a1302 Change retry times and add retry logic 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-26 15:17:47 +08:00
neza2017 4ea27335fd Ignore index param from create_collection 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-26 14:40:53 +08:00
zhenshan.cao 007e7ea858 Add tso unittest and EnableMaxLogic 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-26 13:05:52 +08:00
zhenshan.cao 12ea873216 Fix bug: drop empty index should be considered as legal operation 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-26 11:19:10 +08:00
bigsheeper 9edbff0a6b Fix fields check 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-26 11:15:44 +08:00
XuanYang-cn d34e3e77d1 Fix typo 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-26 11:07:25 +08:00
XuanYang-cn 10374f15d7 Update log in datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-26 10:13:36 +08:00
sunby e95cb78ec8 Add LoadCollection from master 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-26 09:23:39 +08:00
neza2017 c36c532c70 Remove unused test code 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-25 19:12:40 +08:00
xige-16 650bf1d993 Refactor release collection warn message 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-25 18:26:11 +08:00
FluorineDog bf75c2fbb4 Fix bug (#1053) 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-02-25 17:56:43 +08:00
Xiangyu Wang 38266b8569 Lock channel at produce and consume 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-25 17:44:29 +08:00
Cai Yudong 84110d2684 Add tests/benchmark and tests/python_test using new python SDK 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-02-25 17:35:36 +08:00
neza2017 d5631b09e8 Describe collection by id 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-25 16:08:56 +08:00
bigsheeper 7cb28428ce Prevent consuming after loading collection 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-25 15:08:50 +08:00
neza2017 4c3486c770 Fix timestamp allocator 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-25 14:31:38 +08:00
yukun a391d4fbbf Fix stuck problem of create_index 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-25 12:03:35 +08:00
cai.zhang 1d899513dd Unify service name in config file 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-24 18:46:19 +08:00
cai.zhang 9be4a107f9 Fix bug for free memory 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-24 18:25:34 +08:00
zhenshan.cao bbc65e0b14 Remove checking of VectorFieldMetricType in create_collection 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-24 18:14:59 +08:00
sunby c149995bcd Change return value of GetCollectionStatistics. 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-24 17:56:03 +08:00
bigsheeper 08a020798e Release collection and partitions 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-24 17:24:51 +08:00
zhenshan.cao 0679954922 Refactor tso and global id allocator 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-24 17:12:06 +08:00
neza2017 e396c3f4ad Add zap log 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-24 16:25:40 +08:00
xige-16 4c491471ee Add release collection and release partition interface for query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-24 15:58:55 +08:00
BossZou b02a9d6b43 Enlarge rocksmq stream buffer size 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-24 15:16:09 +08:00
XuanYang-cn fd562f9f9c Update doc: add proxy graph 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-24 09:48:17 +08:00
zhenshan.cao 7c5e2d89f6 Read node port from config 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 18:08:17 +08:00
zhenshan.cao 6b392cbe58 Fix memory leak 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 14:35:00 +08:00
xige-16 d5379a82a8 Fix load binlog error when segment not flushed 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-23 14:13:33 +08:00
dragondriver b506754364 Add workflow of drop index in index service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-23 11:57:18 +08:00
zhenshan.cao bbfcbbdd68 Fix ci: paramstable and startup logic refactor 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 11:40:30 +08:00
xige-16 7a7a73e89c Fix high memory usage in pulsarTtStream 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-23 11:40:12 +08:00
FluorineDog 15dd17488e Support benchmark 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-02-23 10:47:21 +08:00
sunby ea1d9ea99a Replace log in dataservice 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-23 09:58:06 +08:00
dragondriver 68518fec38 Fix memory leak in indexnode 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-22 18:33:40 +08:00
neza2017 f0c16a228e Fix show segments 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-22 16:40:26 +08:00
yukun e786ad6aa8 Fix inconsistency of produce msg and consume msg with rmq 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-22 16:34:15 +08:00
sunby b07a371f25 Create a goroutine to receive msg for every consumer 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-22 09:58:34 +08:00
xige-16 861576f77a Checkout field ids when load segment in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-22 10:44:38 +08:00
BossZou 7ac1821cf1 Add RocksMQ throughout test 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-20 18:30:37 +08:00
BossZou ae0e8b1a19 Init data service param before new DataServive 
Signed-off-by: BossZou <yinghao.zou@zilliz.com>
 2021-02-20 16:45:49 +08:00
cai.zhang 5c8747cf40 Make proxy retry time more frequently 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-20 16:14:10 +08:00
yukun 346b0a9549 Add unittest for RmqTtMsgStream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-20 15:58:19 +08:00
neza2017 1d3fdebcf4 Add drop index for proxy 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-20 15:38:44 +08:00
cai.zhang 4c5a898e9a Add masterservice docker 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-20 11:15:19 +08:00
dragondriver 6ebb03791f Fix drop index interface 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-20 10:37:11 +08:00
zhenshan.cao b3891ca199 Correct docker file 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-20 10:15:37 +08:00
xige-16 f62078c027 Fix crash error when search 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-20 10:14:03 +08:00
xige-16 a6442cb600 Fix querynode memory leak 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-20 09:20:51 +08:00
sunby 5769c000a1 Find failed python tests and add skip mark "r0.3-test" 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-19 15:37:04 +08:00
bigsheeper d8f41db823 Fix search crash error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-19 13:56:01 +08:00
sunby 5b61182140 Add zap log 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-18 19:53:55 +08:00
zhenshan.cao d404d87633 Fix bug of getindexstate 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-19 09:52:06 +08:00
xige-16 450ea631bc Fix queryService assign queryChannel failure 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-18 16:26:02 +08:00
neza2017 549bda93ea Fix get index by name 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-18 15:58:59 +08:00
groot af3736aff6 Fix rocksmq bug 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-18 14:13:55 +08:00
neza2017 85c1404393 Refactor master:index ids 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-11 08:41:59 +08:00
xige-16 70fe5233cf Watch queryChannel on query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-09 17:09:26 +08:00
groot 833f1d49ab Let singlenode use rocksmq 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-09 15:57:10 +08:00
neza2017 e2d8358cb4 Refactor master:index ids 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-09 13:11:55 +08:00
neza2017 c2956dbe92 Change master meta prefix 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-09 10:36:31 +08:00
zhenshan.cao 7901d98d5e Fix bug: return wrong IndexID 0 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-08 18:42:33 +08:00
sunby c74221e4da Change ttMsgStream of dataservice to MsgStream 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-08 16:42:56 +08:00
neza2017 4035083188 Support Index name 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-08 14:49:12 +08:00
sunby 90d69658d9 Fix master meta table bugs 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-08 15:19:48 +08:00
groot dc3736281a Integrate message stream 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-08 14:30:54 +08:00
zhenshan.cao faa23fde86 Fix getindexstate failed to fetch indexstate 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-08 14:20:29 +08:00
dragondriver da94fbed0a Parse to map when the key of index params is named params 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-07 21:32:37 +08:00
bigsheeper 12b2eaf196 Rename apis 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 21:26:03 +08:00
zhenshan.cao 1ba8e2448f Add tsLoop for indexservice 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-07 18:38:58 +08:00
bigsheeper a64c831ed9 Fix segment replacement bug 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 18:37:18 +08:00
bigsheeper 7342e0758c Fix index and flush errors, and fix master crash error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 17:02:13 +08:00
zhenshan.cao 2c9e226703 Add log for segment logic in proxy 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-07 16:44:11 +08:00
xige-16 c7ac238aac Fix error when query node release collection 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-07 15:56:57 +08:00
xige-16 2ca53fa668 Fix msgstream deadlock when loadCollection 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-07 15:47:10 +08:00
yukun 563a8d66ae Add RmqTtMsgStream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-07 15:10:13 +08:00
sunby 6ee6557981 Record segment flushed state 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-07 14:51:52 +08:00
sunby eba5b9ae27 Fix queryservice client init bug 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-07 13:53:40 +08:00
bigsheeper b07b2484dc Fix error when loading 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 10:29:58 +08:00
dragondriver 8f4995dab5 Fix DropCollection task in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-07 09:56:39 +08:00
xige-16 cdcf006690 Fix loadCollection error when query service assign insert channels and query nodes 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-07 09:30:48 +08:00
zhenshan.cao 39161cac9a Add collectonScheam when calling load_collection and load_partition 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-06 21:17:18 +08:00
xige-16 a12b184e02 Add field schema to LoadCollection and LoadPartition photo 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-06 17:37:10 +08:00
bigsheeper e685f1825b Fix unittest caused by param table 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-06 15:16:10 +08:00
groot b034740c6c Make paramstable initialize once by using do_once 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-06 13:39:15 +08:00
yukun 099220394e Add rmq_msgstream unittest and fix bugs 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-06 11:39:12 +08:00
bigsheeper e9ee9a273e Refactor load service and check insertion binLog periodically 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-06 11:35:35 +08:00
sunby 0111cba8ae Fix startup bugs in some modules 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-06 10:28:53 +08:00
XuanYang-cn dd72dfcbd1 Fix queryservice start 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-05 17:57:41 +08:00
godchen c231f6da79 Fix collectionName not found error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-05 16:32:03 +08:00
xige-16 57ba236117 Add mutex for message stream’s asConsumer 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-05 15:42:53 +08:00
dragondriver 7e1887da12 Add start.sh and stop.sh, remove them later 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 14:31:40 +08:00
bigsheeper a7eae3a4c1 Remove consumer in query node's initialization 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-05 14:15:10 +08:00
dragondriver e6aec3fd08 Hardcode the peer id of proxy node 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 14:09:55 +08:00
dragondriver 6ef82a59e5 Fix the bug when close the message stream of InsertTask 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 13:49:51 +08:00
godchen 0f620ab149 Fix invalidate metacache error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-05 11:49:13 +08:00
dragondriver e384222803 Fix the logic of node id allocator 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 11:25:52 +08:00
neza2017 bea1535425 Add call of release collection when drop collection 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-05 11:05:37 +08:00
bigsheeper 1578c13224 Refactor collection replica 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-05 10:53:11 +08:00
dragondriver a48590a2df Add query service logic to Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 10:29:14 +08:00
bigsheeper 2ee54541aa Fix query node address 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-05 10:25:47 +08:00
sunby 9c3ea487f6 Fix close insert msg stream bug 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-05 10:19:52 +08:00
dragondriver 75e549da4e Change sdk version used in regression to 0.0.19 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 10:09:03 +08:00
XuanYang-cn a64c4efde9 Refactor collectionReplica in datanode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-04 20:31:23 +08:00
sunby be596123d1 Fix dataservice startup 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-04 20:03:19 +08:00
godchen c2ca2c276f Fix response check error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-04 19:34:35 +08:00
cai.zhang 4f2a1e7912 Fix bug for dataservice dockerfile 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-04 18:11:19 +08:00
dragondriver e0e8e1605e Implement the release-related task and load-related task in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-04 17:47:19 +08:00
sunby 7270d18c98 Fix timesync startup bugs 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-04 17:31:02 +08:00
xige-16 aaac839075 Joint debug query service success with other components 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-04 17:09:18 +08:00
XuanYang-cn 560d2350ae Fix datanode watchDmChannel bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-04 16:09:34 +08:00
godchen d6200a5196 Fix meta cache error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-04 15:54:16 +08:00
cai.zhang 55634dc5f3 Add interface of load and release collection and partition 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-04 15:31:02 +08:00
yukun e3fadc45d4 Fix for new msgstream interface 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-04 15:23:21 +08:00
zhenshan.cao b89e5a3240 Add GetQuerySegmentInfo 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-04 14:37:12 +08:00
sunby 71b6c88c6e Defer lock's unlock in meta::DropSegment 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-04 11:52:10 +08:00
Xiangyu Wang 8d35e76ad1 Change Msgstream interface 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-04 13:55:47 +08:00
bigsheeper c6c99ef32b Implement getSegmengInfo function 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-04 11:40:14 +08:00
XuanYang-cn 1aafe86ffe Change MsgPosition Logic 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-04 11:19:48 +08:00
dragondriver b92ff69cea Implement GetIndexState in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-04 11:08:36 +08:00
XuanYang-cn bc7ad02df4 Fix datanode init bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-04 10:25:01 +08:00
godchen 6e7e0b748a Send timetick to insert channel 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-03 20:04:29 +08:00
sunby e7588d3250 Implement getSegmentInfo interfaces 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-03 19:45:52 +08:00
neza2017 1189332786 Create default partition when create collection 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-03 18:41:20 +08:00
zhenshan.cao 1b616fb874 Add persistent_segmentinfo 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-03 18:55:00 +08:00
bigsheeper f12a0490bb Do not drop collections or partitions in flow graph 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 18:12:48 +08:00
Xiangyu Wang fd9e42d3ed Update msgstream/unmarshal interface 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-03 17:30:10 +08:00
bigsheeper abffdbff62 Remove partition tag 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 16:44:38 +08:00
dragondriver 6c3f169ecc Add collection id and partition id to insert & search workflow 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-03 16:40:23 +08:00
zhenshan.cao b853020ef9 Remove redudent ctx and openTracing logic 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-03 16:34:56 +08:00
yukun 35f6f4b7c2 Add consume in rmq_msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-03 15:18:54 +08:00
XuanYang-cn 73a8e7de84 Fix datanode insertchannel refresh bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-03 15:18:05 +08:00
sunby 208700b0e7 Set fixed insert channel number, remove channel manager 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-03 15:07:48 +08:00
bigsheeper 01e9dc8e3f Remove collection name 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 11:52:19 +08:00
sunby f3aad3a71c Change SegmentInfo 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 18:53:10 +08:00
bigsheeper 5e781b9370 Remove field name in query node and segCore 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 10:10:07 +08:00
sunby 45b99c0cf3 Save indexName and indexID in IndexCodec 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 19:56:11 +08:00
neza2017 899d31e72d Change interface: add index id and index name params 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-02 20:09:46 +08:00
zhenshan.cao bfa14d40a8 Rename indexID to indexBuildID and add IndexID and IndexName 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-02 19:56:04 +08:00
godchen 66d5189601 Add get collection statistics implemetation 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-02 19:54:31 +08:00
Xiangyu Wang 32c5a92449 Update interface in msgstream 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-02 16:32:15 +08:00
dragondriver e4e3ec88d1 Add usage count of insert channels 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-02 15:49:13 +08:00
bigsheeper 09bdbee944 Refactor load service, remove seg manager 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-02 15:13:08 +08:00
sunby ca9c2f5c12 Add GetCount and implement GetCollectionStatistics 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 14:25:58 +08:00
xige-16 4cd315408b Refactor query service’s meta interface 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-02 11:52:41 +08:00
sunby 46f14a5e69 Fix bugs in GetInsertChannels 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 11:14:17 +08:00
zhenshan.cao 3afa7dd50d Fix GetSegmentStates 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-02 10:58:39 +08:00
neza2017 5de89422ed Support multi index 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-02 10:09:10 +08:00
XuanYang-cn 90d471b133 Refactor main of queryservice and querynode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-02 09:52:42 +08:00
godchen 93fd6b70f3 Fix the error usage of ttmsg 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-01 19:22:35 +08:00
sunby a74767fe57 Fix bugs 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-01 16:20:13 +08:00
neza2017 3f45cfd6e8 Fix data node 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-02-01 11:44:02 +08:00
godchen 94dfab5b21 Check collection whether exists before drop 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-01 10:53:13 +08:00
godchen abcc565311 Add proxy metacache 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-31 14:55:36 +08:00
zhenshan.cao c2914dd113 Refactor segment allocator, use collectionID and partitionID 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-31 10:15:54 +08:00
neza2017 854accf95b Remove master and writenode 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-30 16:41:47 +08:00
bigsheeper 3573e01093 Check index periodically and load index, add indexing segment type 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-30 16:02:10 +08:00
dragondriver 996111bf8c Refactor insert channels used in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-30 15:30:38 +08:00
sunby 3a67dda06c Add NewDataService in main 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-30 11:49:48 +08:00
neza2017 51eac0eb84 Use GetCollectionByName instead 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-30 11:38:02 +08:00