Commit Graph

350 Commits (ab5a7cbf44458cf008df7d0be8f01676eac5b344)

Author SHA1 Message Date
Cai Yudong a992dcf6a8
Support query return vector output field (#6570)
* improve code readibility

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add offset in RetrieveResults

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add VectorFieldInfo into Segment struct

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add new interface for query vector

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update load vector field logic

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fill in field name in query result

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add FieldId into FieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add fillVectorOutputFieldsIfNeeded

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update data_codec_test.go

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add DeserializeFieldData

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* realize query return vector output field

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable query vector case

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-16 17:19:55 +08:00
dragondriver 7f99d9d714
Loose the limit of dimension when searching on a binary vector field (#6503)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-14 14:33:54 +08:00
Cai Yudong 1c29b42434
change output_fields to out_fields_id for RetrieveRequest (#6506)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-14 13:11:54 +08:00
Cai Yudong 724f10b9a0
Unify the usage of query and search (#6467)
Unify the usage of query and search

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-13 22:20:33 +08:00
dragondriver 25b8c37383
Fix 6421, split raw vectors to serveral smaller one (#6449)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-13 16:05:36 +08:00
xaxys e5f8a77a2d
Extend grammar of boolean expression (#6460)
* Extend grammar of boolean expression

Signed-off-by: xaxys <tpnnghd@163.com>
2021-07-13 14:37:03 +08:00
FluorineDog 99ed122d11
Remove Dead Code, use signed type (#6398)
* make type signed

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* remove dead code

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* remove code

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-09 18:25:11 +08:00
FluorineDog 2ff072ea7b
Use larger dim for binary test to avoid false positive test failed (#6416)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-09 18:12:40 +08:00
FluorineDog d972a0361d
Weaken debug message (#6324)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-08 15:50:57 +08:00
quicksilver 7d920e9c9c
Fix compile error on CentOS (#6359)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-07-08 15:43:45 +08:00
shengjun.li a1ba9e9dcd
Fix comile on CentOS (#6334)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-07-07 12:34:02 +08:00
FluorineDog 97b1426666
fix benchmark, add support for primary key in schemaHelper (#6281)
* fix benchmark

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* fix tests

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-06 09:50:01 +08:00
Cai Yudong 21a9bcd844
Create default partition when receive CreateCollectionMsg (#6305)
* create default partition when create collection

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* adjust sleep time for unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-06 09:16:03 +08:00
shengjun.li 1f42b4ec9d
fix the case that knng is not full (#6308)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-07-05 17:26:07 +08:00
Cai Yudong bc8392eb1e
Add collectionID and partitionID into SegmentIndexInfo (#6289)
* add collectionID and partitionID into SegmentIndexInfo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add partID2SegID into meta table

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update DropIndex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-03 20:00:40 +08:00
neza2017 111a24a49f
remove msgstream of segment info (#6280)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-03 14:36:18 +08:00
dragondriver 8dd2051a9a
Fix 5045, decrease the nb and nlist in unittest (#6282)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-07-02 18:20:14 +08:00
groot a364d7303b
Implement calculate vectors distance in server (#6219)
* calc distance stage 3

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc distance stage 5

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* calc distance stage 6

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* code style

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2021-07-01 18:56:17 +08:00
FluorineDog bf8b2be4a7
Deprecate num_groups to simplify search API (#6230)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-07-01 10:32:15 +08:00
zhenshan.cao ff4c6c7ff4
Fix travel timestamp and guarantee timestamp (#6234)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-30 21:02:13 +08:00
FluorineDog b90b4f2058
renaming (#6192)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-30 10:18:26 +08:00
yukun b87baa108a
Add topks in searchResult (#6168)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-28 15:28:11 +08:00
yukun eb818d05ab
Remove vector fields from output fields (#6112)
* Remove vector fields from output fields

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add empty_collection error_code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-25 16:52:11 +08:00
sunby 46f86a0354
Return sealed segments when calling flush (#6011)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-23 16:56:11 +08:00
yukun b2eb596c23
Change search result to columnBased and return target_entry (#5896)
* Change search result to columnBased and return target_entry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: fluorinedog <fluorinedog@gmail.com>

* synchronize the latest pymilvus-orm version

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change pymilvus-orm version to 40

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: fluorinedog <fluorinedog@gmail.com>
2021-06-21 20:18:13 +08:00
dragondriver 2642415bc3
Redefine the auto_id and primary_key (#5903)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-21 11:42:18 +08:00
zhenshan.cao 958280ebb8
Add support for time travel (#5897)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-21 10:42:10 +08:00
FluorineDog 255e3959af
support time travel (#5894)
* support time travel

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* lint

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-19 17:38:11 +08:00
zhenshan.cao ebe5073749 Update proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:56:09 +08:00
FluorineDog bec9f2c182
Split segcore and plan proto for future feature (#5767)
* Split segcore and plan proto for future feature

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* lint

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-15 14:43:57 +08:00
bigsheeper cdbc6d2c94
Refactor query node and query service (#5751)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 12:41:40 +08:00
FluorineDog 006dae35c3
fix retrieve bug (#5727)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-11 00:06:55 +08:00
Xiangyu Wang 68f703f983
Add Query impl in proxynode (#5644)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-07 17:21:36 +08:00
shengjun.li 8ead67ef95
IVF_SQ8 and IVF_PQ cannot be built on multiple GPUs (#5624)
* IVF_SQ8 and IVF_PQ cannot be built on multiple GPUs

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* index created by std::make_shared

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-06-05 17:19:34 +08:00
FluorineDog 1ff4764ba6
Fix wrong primary key (#5604)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-04 18:35:36 +08:00
Xiangyu Wang afa5da7df0
Add query rpc into proto (#5592)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-04 12:02:34 +08:00
FluorineDog 9a90313390
Support GetEntityByIDs in CGo, fix segcore bugs (#5563)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-04 10:38:34 +08:00
dragondriver 99699b2021
Support to show collections which are loaded into memory (#5567)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-03 19:09:33 +08:00
dragondriver e76830ab09
Support column-based insert request in proxy (#5560)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-03 15:03:34 +08:00
Xiangyu Wang ca0993c274 Update OWNERS
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-30 18:58:47 +08:00
FluorineDog b1a9aea6a6
support get entity by ids in segcore (#5456)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-05-28 10:39:30 +08:00
Xiangyu Wang 88b42304ef
Add dummy RPC (#5376)
Adding a dummy RPC makes it easier to develop new RPCs in the future.
When developing a new function, the SDK is often implemented last. 
At this time, this is helpful for debugging.

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-25 06:44:43 +00:00
shengjun.li b688cc5f51
Improve IVF query on GPU when no entity deleted (#5359)
Not to call getListIndex() if bitset is empty.

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-05-24 07:38:56 +00:00
FluorineDog 3f42a9ed12
reduce unittest time (#5363)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-05-24 03:54:06 +00:00
sunby c1efa53779
Remove redundant fields in SegmentInfo and related structure (#5280)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-05-21 06:51:25 +00:00
yukun f18dfb4ff6
Add RetrieveTask implementation (#5313)
Resolves: #5257 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-20 07:02:31 +00:00
yukun a7bb701f73
Add Retrieve method grpc definition (#5294)
See also: #5253 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-19 10:45:15 +00:00
quicksilver 3db653f2d0
Update cpp thirdparty source URL (#5295)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Resolves: #5293
2021-05-19 09:39:29 +00:00
neza2017 d68ee3fbc2
query base on timeline (#5279)
query base on timeline, add timestamp on these queries:
- ShowCollections
- HasCollection
- DescribeCollection

Resolves: #5219

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-18 09:12:17 +00:00
Cai Yudong e8ceb37f81
Add ShardsNum when create collection (#5245)
Updata PhysicalChannelNames and VirtualChannelNames based
on ShardsNum when create collection, and update collVChan2Chan
map.

Resolves: #5240 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-17 11:15:01 +00:00