milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
21,028 Commits
127 Branches
113 Tags
553 MiB
Commit Graph

392 Commits (a2f0092e39d70efcef57b5e1e0dddd032cf7bcc7)

Author SHA1 Message Date
Cai Yudong e27283abbf
Update query reduce log (#11402) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-08 16:07:34 +08:00
Cai Yudong d32c77e99b
Update search reduce log (#11400) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-08 16:03:30 +08:00
godchen 88c9067873
Add hashvalue warning (#10668) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-05 14:20:55 +08:00
bigsheeper c99d1a56bf
Refactor proxy ticker to fix guarantee_time not work (#11248) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-11-05 09:14:02 +08:00
Cai Yudong 0fac38000c
Correct empty retrieve result handling (#11219) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-04 18:08:32 +08:00
godchen a0a3a889e7
Add common endian for global use (#11092) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-02 18:16:32 +08:00
Cai Yudong 70188990dc
Remove primary key duplicated query result on proxy (#10967) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-01 10:27:56 +08:00
Cai Yudong da0cb4a702
Use primary key only to check search result duplicate (#10949) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-01 10:06:03 +08:00
Cai Yudong 0049837dc4
Add utils API AppendFieldData (#10910) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-29 18:06:43 +08:00
Cai Yudong 5ad4cdda25
Clean search result duplicates removal debug log (#10769) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-27 20:02:26 +08:00
Cai Yudong 24f9411249
Optimize queryTask PostExecute (#10739) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-27 17:34:35 +08:00
Cai Yudong c51155a542
Optimize decodeSearchResults (#10728) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-27 14:10:27 +08:00
cai.zhang 3734ff49ac
Add default value for the parameter roundDeciaml (#9626) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-26 18:24:23 +08:00
zhenshan.cao bb08df28f3
Fix golint error in proxy (#10620) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-26 10:38:41 +08:00
zhenshan.cao 4b0fd28de7
Fix golint error in proxy (#10621) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-25 23:42:29 +08:00
godchen daaa9428ea
Fix delete msg hash error (#10599) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-25 22:20:20 +08:00
godchen d2e0943f21
Fix delete node flush error (#10549) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-25 18:49:20 +08:00
cai.zhang c3cce20413
Reset the msgstream threshold to PulsarMaxMessageSize (#10186) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-25 17:57:14 +08:00
Cai Yudong 880e375b00
[skip ci] Remove comments (#10541) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-25 15:25:12 +08:00
Cai Yudong f6802589eb
Optimize proxy reduce code readability (#10537) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-25 14:29:12 +08:00
zhenshan.cao 3af852eafb
Fix golint error in proxy (#10488) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-23 23:17:11 +08:00
zhenshan.cao 51d82fc71d
Fix golint error in proxy (#10487) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-23 18:25:37 +08:00
zhenshan.cao 169359448f
Fix golint error in proxy (#10486) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-23 18:23:44 +08:00
zhenshan.cao 43ef5d5eb9
Fix golint error in proxy (#10485) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-23 10:53:12 +08:00
Cai Yudong 824005115a
Remove duplicated search results in proxy (#10355) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-21 19:52:28 +08:00
Cai Yudong b099179ac0
Optimize proxy reduce search result data (#10327) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-21 10:46:36 +08:00
Cai Yudong b40513b211
Use timerecord to calculate time span (#10287) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-20 17:56:38 +08:00
godchen 24a6fc096b
Update query node bloom filter (#10109) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-18 20:08:42 +08:00
Cai Yudong dc0269cf4c
Optimize proxy getChannels (#9964) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-15 20:37:04 +08:00
yukun 92815f4542
Change timestamp to timestamps in internalproto.DeleteRequest (#9802) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-14 14:48:34 +08:00
dragondriver 8f5bf46705
Fix golint warning of plan_parser.go (#9827) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-10-13 20:30:32 +08:00
Cai Yudong 84e568ee32
Support Delete in datanode (#8505) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-11 16:31:44 +08:00
Cai Yudong bd3a8ed3cf
Support delete in proxy (#9588) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-11 07:48:55 +08:00
Xiangyu Wang 75716e0deb
Update partition name validation in DeleteTask (#9560) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-10-09 22:50:39 +08:00
dragondriver 302ca04d6c
Describe collection by collection ID (#9471) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-10-09 16:10:56 +08:00
elfisworking d4232f88a2
For search funtion adding a round_decimal paramter to precision control (#8574) 
Signed-off-by: elfisworking <zymustb@126.com>
 2021-10-08 17:39:55 +08:00
congqixia 827a18a7c1
Fix Assigner typo (#9075) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-02 18:43:56 +08:00
godchen 4bfa85eda3
Fix searchTask result nil (#8844) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-09-29 15:48:00 +08:00
zhenshan.cao a662c62880
Fix golint error of segment.go in proxy package (#8629) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-09-28 23:06:15 +08:00
dragondriver 6065bb50c4
Fix golint warning of funcutil (#8652) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-27 23:18:02 +08:00
zhenshan.cao b71bffe26f
Replace ChannelID with ShardName in DML Request (#8539) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-09-27 10:01:59 +08:00
Cai Yudong 3938d68034
Optimize proxy query execute (#8479) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-25 00:21:57 +08:00
Cai Yudong ee3eaa2966
Rename CreateQueryExprPlan to CreateExprPlan (#8438) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-09-24 10:27:54 +08:00
XuanYang-cn 4ee8275b88
Fix master compile error (#8333) 
Resolves: #8336

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-09-22 20:31:54 +08:00
cai.zhang 729d409a32
Add trace log for search (#8302) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-09-22 19:36:00 +08:00
cai.zhang 79b7dcc75a
Add trace log for insert (#8301) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-09-22 19:33:54 +08:00
dragondriver b2e8ba7b33
Fix reduce algorithm in proxy search task (#8206) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-22 10:15:54 +08:00
congqixia bdfb8768d4
Enable gosec (#8202) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-18 14:45:50 +08:00
Yusup ccb57ba4b2
add collection alias (#7255) 
Signed-off-by: Yusup <yusup@lsgrep.com>
 2021-09-18 11:13:51 +08:00
yukun 46871fcaec
Remove unused code in queryTask (#8077) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-17 21:57:58 +08:00
congqixia 4fe22bcaaa
Fix query task panicking (#8060) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-16 20:03:50 +08:00
dragondriver 5f3b961502
Add test case for delete task (#7939) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-15 14:04:54 +08:00
Xiaofan d0667b58d7
Change Error Msg When dimension of vector field not exist (#7931) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-15 10:01:30 +08:00
dragondriver 2382be82a5
Add test case for the workflow of insert task (#7912) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-14 19:07:48 +08:00
dragondriver ec776a30dc
Add test case for the workflow of query task (#7886) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-14 16:19:08 +08:00
dragondriver f122e93383
Proxy ut cover case that search task process the fields data (#7874) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-14 14:09:05 +08:00
zhenshan.cao e5e4b24954
Fix bug: describe collection return wrong shard number (#7841) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-09-14 11:59:47 +08:00
dragondriver bcf9b4e240
Coverage error case of searchTask.PreExecute (#7826) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-14 10:24:00 +08:00
dragondriver 8d239b6473
Add test cases for search task (#7796) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-13 17:12:19 +08:00
Xiaofan 619c4393f3
Refine logs in proxy search path(#7357) (#7753) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-09-12 17:54:01 +08:00
dragondriver 5a3fc37584
Add test cases for create collection (#7759) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-11 18:06:02 +08:00
dragondriver a872a4357c
Add more test cases for proxy implementation (#7685) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-11 11:36:22 +08:00
dragondriver 596357dc85
Add more data definition test cases (#7623) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-09 19:02:08 +08:00
zhenshan.cao b0524adbd8
Support specified shard number when create collection (#7482) 
Add shards_num to CollectionInfo

Modify dmlChannelNum config

Compatible with old meta

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-09-08 15:00:00 +08:00
dragondriver 42b687bf48
Add unittest for task scheduler (#7508) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-09-06 20:49:04 +08:00
yukun 94272bba87
Support query by expression (#7386) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-03 17:12:55 +08:00
groot 8701c477e2
Add delete api into proto (#7304) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2021-08-26 12:15:52 +08:00
Cai Yudong 0ab70271a6
correct empty search result handle (#7244) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-08-24 11:13:52 +08:00
Cai Yudong 8405d90f5e
proxy add illegal check for search result (#7227) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-08-23 17:03:51 +08:00
yukun 671b2737d2
Remove RetrieveRequest and RetrieveTask (#6848) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-08-16 17:06:10 +08:00
Cai Yudong e771bda92f
optimize retrieve output vector code structure (#7102) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-08-14 11:18:10 +08:00
xige-16 87c5a49d9a
Refactor showCollection/showPartition to get load state in memory (#6935) 
* refactor showCollection/showPartition to get load state in memory

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update milvus.proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* add show result log to debug

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix wrong err message

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update pymilvus-orm version

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix repeated load error

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* update proto

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix status nil error

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-08-02 22:39:25 +08:00
Cai Yudong fe50f97726
Use id to tell search result validation in proxy reduce stage (#6905) 
* use id to tell search result validation

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable test_search_binary_hamming_flat_index

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix merge retrieve result issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-08-02 10:25:49 +08:00
cai.zhang d51fec6678
Enhance log when create index failed (#6871) 
* Enhance log when create index failed

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>

* Fix bug for repeatedly drop index

Signed-off-by: xiaocai2333 <cai.zhang@zilliz.com>
 2021-07-29 14:47:22 +08:00
zhenshan.cao 8a1a841011
FixBug: Incorrect handling of time synchronization (#6814) 
* FixBug: proxy send wrong sync timetick and DataCoord did not filter channel

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-07-27 14:34:48 +08:00
dragondriver 3cc8ee298e
Filter NaN when reducing search results (#6806) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-27 10:19:21 +08:00
zhenshan.cao 2962d31a5f
Fixbug: segment allocate but return value incorrect (#6788) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-07-24 20:49:20 +08:00
dragondriver cf8e9d52b1
Reduce the log of proxy (#6766) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-23 20:33:34 +08:00
dragondriver 67c1fdd2c0
Show/Describe collections/partitions with created timestamp (#6700) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-21 18:00:14 +08:00
Cai Yudong cbb4052fdb
Support search and query output fields using wildcard (#6671) 
* update wildcard polocy for search and query

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* mark xfail cases

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-21 14:52:12 +08:00
dragondriver 2d9361e0ba
Fix bug: remove duplicated primary field id when retrieve entities (#6577) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-17 10:53:30 +08:00
Cai Yudong a992dcf6a8
Support query return vector output field (#6570) 
* improve code readibility

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add offset in RetrieveResults

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add VectorFieldInfo into Segment struct

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add new interface for query vector

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fill in field name in query result

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add FieldId into FieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add fillVectorOutputFieldsIfNeeded

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update data_codec_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add DeserializeFieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* realize query return vector output field

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable query vector case

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-16 17:19:55 +08:00
dragondriver 5016e509be
Support wildcard match on search/query output fields (#6510) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-14 18:51:54 +08:00
Cai Yudong 1c29b42434
change output_fields to out_fields_id for RetrieveRequest (#6506) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-07-14 13:11:54 +08:00
dragondriver d896e3119e
Fix 6419, check if num_rows is greater than zero in proxy (#6439) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-13 16:33:55 +08:00
dragondriver bd317a5461
Check if field data is empty (#6404) 
* Check if field data is empty

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* FixBug: fast fail when output_fields contain fields that are not in schema

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-07-11 19:39:52 +08:00
neza2017 8b89744c6b
search with exp (#6409) 
* process float vector and binary vector field
on the search result

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add more debug info on etcd

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* add debug info when grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-07-10 10:21:52 +08:00
dragondriver 166548447b
Fix 6249, use more clear error message (#6316) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-07-06 14:40:04 +08:00
zhenshan.cao ff4c6c7ff4
Fix travel timestamp and guarantee timestamp (#6234) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-30 21:02:13 +08:00
yukun 95aa3c85b6
Add index_name check for drop_index (#6217) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-30 17:56:12 +08:00
yukun b87baa108a
Add topks in searchResult (#6168) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-28 15:28:11 +08:00
dragondriver d3c503f3aa
Calculate the real topk in proxy (#6132) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-28 14:30:14 +08:00
yukun ae66446f16
Fix issue-5992 (#6158) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-28 12:16:13 +08:00
yukun effcbc7cb3
Fix issue-6143 (#6145) 
* Fix issue-6143

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Return error when output_fields contains vector field

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Fix when output_fields contain primarykey

Signed-off-by: kun yu <kun.yu@zilliz.com>

* Add primarykey

Signed-off-by: kun yu <kun.yu@zilliz.com>
 2021-06-28 10:12:13 +08:00
xige-16 ca4cba152c
return loaded collection when ShowCollectionsType_InMemory (#6149) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-27 12:10:08 +08:00
yukun eb818d05ab
Remove vector fields from output fields (#6112) 
* Remove vector fields from output fields

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add empty_collection error_code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-25 16:52:11 +08:00
yukun 9b549e0fa4
Add primaryKey to output fields (#6094) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-24 21:08:14 +08:00
sunby 46f86a0354
Return sealed segments when calling flush (#6011) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-23 16:56:11 +08:00
dragondriver a493151f70
Fix #5951 (#6010) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-23 12:06:10 +08:00
yukun 792dba4ae3
Fix id field insert failed when auto_id is true (#5984) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-22 18:46:04 +08:00
zhenshan.cao 0f05622c09
Modify error msg when index is not created (#5978) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-06-22 18:04:07 +08:00
xige-16 2af1c27811
Rename queryservice to querycoordinator (#5931) 
* Rename queryservice to querycoordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix param table

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* fix unittest

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* service to coordinator

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* code format

Signed-off-by: xige-16 <xi.ge@zilliz.com>

* set querycoord in ci

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-06-22 16:44:09 +08:00
dragondriver fd57554d32
Add index params check in proxy (#5958) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-06-22 15:28:04 +08:00
Cai Yudong 2926a78968
Rename proxynode to proxy (#5961) 
* rename package name from proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move distributed/proxynode to distributed/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move internal/proxynode to internal/proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename proxynode to proxy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-06-22 14:40:07 +08:00
quicksilver 7fe061b846 Raneme init_devcontainer.sh to devcontainer.sh 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-01-15 14:38:36 +08:00
neza2017 56ab2bda2d Update proto:add partition id 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-15 11:53:22 +08:00
zhenshan.cao 63b21321d6 Refactor indexservice and update doc 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-15 13:50:27 +08:00
cai.zhang 3d885742ee Reopen ruleguard and format code 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-13 13:36:27 +08:00
bigsheeper 2d2af1505a Improve mutex usage in query node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-12 18:03:24 +08:00
GuoRentong f3d2afa4f4 Update doc:detailed interface definition 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2021-01-11 18:35:54 +08:00
quicksilver a6690dbc99 Fix syntax error: Bad fd number 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-01-11 15:25:19 +08:00
yukun 18cfd31d40 Fix Load and Remove with prefix 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-01-11 15:17:06 +08:00
bigsheeper 4fad3b189c Fix search without insertion, improve nil hits behavior (#622) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-09 17:58:18 +08:00
xige-16 f4566731fc Fix search error in regression test 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-09 09:47:22 +08:00
sunby 5dfe9448ae Modify test_index 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-08 16:00:16 +08:00
GuoRentong 82ad488555 Update doc: rocksmq 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2021-01-08 18:37:24 +08:00
dragondriver 018466a256 Avoid crash when index parameters are invalid 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-08 17:45:57 +08:00
XuanYang-cn 65089ea362 Fix flush sync test bug and remove unecessary lock in meta_table 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-08 17:20:59 +08:00
GuoRentong c4599e26de Update doc: node api 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2021-01-08 16:59:19 +08:00
bigsheeper e13822043d Fix search test when partition not exists, add binary vector search validation 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-08 16:41:27 +08:00
godchen fa73f17f94 Add opentracing doc 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-08 15:32:38 +08:00
dragondriver ced56c84f8 Add support to build binary vector index 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-07 14:56:17 +08:00
godchen 8f1f118b26 Add query opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-07 14:24:35 +08:00
bigsheeper a1e2ff3c26 Add gcNode to write node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-06 16:44:12 +08:00
neza2017 4015d7245d Merge operation 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-06 14:45:50 +08:00
zhenshan.cao b06e01f523 Initmate querynode collection manager 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-06 11:31:36 +08:00
godchen 1c87faf8b3 Add insert request opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-06 11:17:35 +08:00
GuoRentong e8ec0424d8 Update doc: service api 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2020-12-27 09:05:24 +08:00
XuanYang-cn e316533a3a Add Flush Sync service 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-26 16:28:55 +08:00
bigsheeper 5acc9681d6 Fix search error about metric type 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-26 15:29:48 +08:00
cai.zhang a8b78f11c3 Update proto about index 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-23 19:06:11 +08:00
cai.zhang 05918f55d2 Impl CreateIndex interface for proxy server 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-22 15:39:10 +08:00
bigsheeper 762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-10 16:31:09 +08:00
godchen 7ab5b5d80d Add MinIO kv implements 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-03 19:00:11 +08:00
neza2017 cec903da19 Add parquet payload 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-02 18:31:56 +08:00
zhenshan.cao 569057251c Add support of custom ID 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-02 16:12:21 +08:00
zhenshan.cao 23aebe7e2a Fix proxy unittest 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-01 19:53:53 +08:00
neza2017 17cc7bc337 Use MetaRootPath instead 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-01 17:03:39 +08:00
dragondriver 84b15c1f5c Fix the bug of meta cache in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-01 16:41:25 +08:00
cai.zhang c1fbc4f8fa Open the tests for insert 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-01 15:18:57 +08:00
FluorineDog 6b74d82259 Support some validation for dsl parser 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-01 02:16:53 +08:00
zhenshan.cao fda8d62b38 Fix make error 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-30 22:14:19 +08:00
neza2017 d3858500c6 Add static check 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-30 19:38:23 +08:00
dragondriver 2358fae703 Check the status of search results before reduce in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-30 19:08:32 +08:00
bigsheeper 0cd3e8d86c Use exception instead of runtime error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-30 17:58:23 +08:00
dragondriver 51a9f49d35 Fix the topk check when deal with the search result in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-30 17:46:00 +08:00
zhenshan.cao 24942c3d6d Fix insert failed to return correct id range 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-30 12:08:47 +08:00
bigsheeper d88a654db8 Fix search result 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-30 11:22:59 +08:00
quicksilver 86573d0053 Create Jenkins pipeline 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-11-30 10:07:04 +08:00
FluorineDog 77fa75b1ec Add binary insert and warper of binary search, rename vector 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-30 05:18:44 +08:00
cai.zhang 85a544b79b Add dockerfile for each component 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-28 19:06:48 +08:00
cai.zhang 115d0732af Update docker-compose file for unittest 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-28 16:43:39 +08:00
dragondriver eb6c0e0a28 Fix the bug when reduce the multiple search result from query nodes 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-28 15:57:49 +08:00
godchen 92bf09c7e6 Add ddstream and scheduler implements 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-11-28 15:26:39 +08:00
dragondriver 568cef0730 Fix InsertTask and SearchTask 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-28 10:48:29 +08:00
FluorineDog c9fb34142c Enable primary_key switch 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-28 09:16:00 +08:00
neza2017 32445dd02b Add cpu profile 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-26 16:01:31 +08:00
cai.zhang 57da66ba72 Update read config logic in docker 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-26 15:18:36 +08:00
neza2017 d9e5a8774d Add dimension validation 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-26 15:07:47 +08:00
FluorineDog 8ae573b580 Fix bool bug 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-26 14:56:58 +08:00
sunby e8e21a0308 Add dimension validation. 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-11-26 14:54:03 +08:00
zhenshan.cao 045fc3d808 Fix segment fault 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-26 14:29:01 +08:00
XuanYang-cn 0a16a9a651 Add logic of getting milvus distributed config from master by grpc 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-11-26 13:21:06 +08:00
dragondriver 8ce0deca4e Add default partition tag to InsertTask in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-26 11:38:04 +08:00
xige-16 ae78aa1ac1 Warp cpp reduce and repack searchResultMsg to specified pulsar channel 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-11-26 11:12:33 +08:00
neza2017 ae26a81550 Change to use Marshal as text 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-26 10:13:23 +08:00
sunby f4c643f1bd Add number limitation of partition and field. And Change default partition tag to "_default" 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-11-25 18:39:05 +08:00
zhenshan.cao f12366342f Fix bug and delete unused code 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-25 16:45:42 +08:00
bigsheeper 4ed11d9775 Reopen segment test assertion 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-25 16:24:57 +08:00
bigsheeper 3b24c52a8c Catch cpp runtime error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-25 16:20:07 +08:00
dragondriver a7b3efecd7 Read query node num from param table instead of hardcode 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-25 16:17:33 +08:00
FluorineDog a6f950e5b6 Fix boost::dynamic_bitset misuse 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-25 15:17:17 +08:00
sunby 24b29bec30 Add collectionName/partitionTag/fieldName validation. 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2020-11-25 14:44:30 +08:00
dragondriver c7a49c97cc Add repack function of InsertTask in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-23 16:52:17 +08:00
dragondriver aabe9afd75 Add meta cache in Proxy and use meta cache to generate id in InsertTask 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-20 17:53:31 +08:00
neza2017 429c94e18b Add grpc for proxy 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-19 17:09:22 +08:00
dragondriver b32e55d5f0 Add timeout logic to task in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-17 20:00:23 +08:00
xige-16 4e8efe85df Delete wrong useage of *TsMsg 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-11-17 14:10:07 +08:00
dragondriver aff1e79f82 Fix SearchTask in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-16 17:01:10 +08:00
dragondriver 23d9ddb4a3 Add unittest to grpc service that Proxy provide 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-14 18:18:10 +08:00
dragondriver c2382adf04 Initialize the task scheduler of Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-14 11:24:49 +08:00
cai.zhang 0b478b614a Change Id to ID in go files according to golint check 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-13 15:17:18 +08:00
cai.zhang 1cd2fd9954 Add the standard of golangci-lint 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-12 12:04:12 +08:00
dragondriver be0a2e7e5f Add other data definition related task to Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-09 17:25:53 +08:00
dragondriver b8ccbc1c97 Add InsertTask and SearchTask to Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-07 16:18:23 +08:00
zhenshan.cao 9c490a209b Delete unused code 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-05 19:11:12 +08:00
dragondriver c732c267e1 Refactor proxy framework and add CreateCollection Task 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-05 18:01:33 +08:00
zhenshan.cao e9122921b9 Use type alias 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-04 17:58:43 +08:00
neza2017 d18c17e786 Add Meta table implementation 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-04 16:28:14 +08:00
godchen bade037fb2 Add meta_table implementation 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-11-04 16:01:28 +08:00
zhenshan.cao 2e2e059f8a Add msgheader and change ReqType to MsgType 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-04 15:13:42 +08:00
zhenshan.cao e962b57fbb Add timestamp allocator 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-03 14:53:36 +08:00