Commit Graph

327 Commits (dc63d825959f73e0428bbce8c6367d1537d25573)

Author SHA1 Message Date
sunby 46f86a0354
Return sealed segments when calling flush (#6011)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-23 16:56:11 +08:00
yukun b2eb596c23
Change search result to columnBased and return target_entry (#5896)
* Change search result to columnBased and return target_entry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: fluorinedog <fluorinedog@gmail.com>

* synchronize the latest pymilvus-orm version

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change pymilvus-orm version to 40

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: fluorinedog <fluorinedog@gmail.com>
2021-06-21 20:18:13 +08:00
dragondriver 2642415bc3
Redefine the auto_id and primary_key (#5903)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-21 11:42:18 +08:00
zhenshan.cao 958280ebb8
Add support for time travel (#5897)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-21 10:42:10 +08:00
FluorineDog 255e3959af
support time travel (#5894)
* support time travel

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* lint

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-19 17:38:11 +08:00
zhenshan.cao ebe5073749 Update proto
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:56:09 +08:00
FluorineDog bec9f2c182
Split segcore and plan proto for future feature (#5767)
* Split segcore and plan proto for future feature

Signed-off-by: fluorinedog <fluorinedog@gmail.com>

* lint

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-15 14:43:57 +08:00
bigsheeper cdbc6d2c94
Refactor query node and query service (#5751)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 12:41:40 +08:00
FluorineDog 006dae35c3
fix retrieve bug (#5727)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-11 00:06:55 +08:00
Xiangyu Wang 68f703f983
Add Query impl in proxynode (#5644)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-07 17:21:36 +08:00
shengjun.li 8ead67ef95
IVF_SQ8 and IVF_PQ cannot be built on multiple GPUs (#5624)
* IVF_SQ8 and IVF_PQ cannot be built on multiple GPUs

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* index created by std::make_shared

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-06-05 17:19:34 +08:00
FluorineDog 1ff4764ba6
Fix wrong primary key (#5604)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-04 18:35:36 +08:00
Xiangyu Wang afa5da7df0
Add query rpc into proto (#5592)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-06-04 12:02:34 +08:00
FluorineDog 9a90313390
Support GetEntityByIDs in CGo, fix segcore bugs (#5563)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-06-04 10:38:34 +08:00
dragondriver 99699b2021
Support to show collections which are loaded into memory (#5567)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-03 19:09:33 +08:00
dragondriver e76830ab09
Support column-based insert request in proxy (#5560)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-03 15:03:34 +08:00
Xiangyu Wang ca0993c274 Update OWNERS
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-30 18:58:47 +08:00
FluorineDog b1a9aea6a6
support get entity by ids in segcore (#5456)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-05-28 10:39:30 +08:00
Xiangyu Wang 88b42304ef
Add dummy RPC (#5376)
Adding a dummy RPC makes it easier to develop new RPCs in the future.
When developing a new function, the SDK is often implemented last. 
At this time, this is helpful for debugging.

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-25 06:44:43 +00:00
shengjun.li b688cc5f51
Improve IVF query on GPU when no entity deleted (#5359)
Not to call getListIndex() if bitset is empty.

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-05-24 07:38:56 +00:00
FluorineDog 3f42a9ed12
reduce unittest time (#5363)
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-05-24 03:54:06 +00:00
sunby c1efa53779
Remove redundant fields in SegmentInfo and related structure (#5280)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-05-21 06:51:25 +00:00
yukun f18dfb4ff6
Add RetrieveTask implementation (#5313)
Resolves: #5257 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-20 07:02:31 +00:00
yukun a7bb701f73
Add Retrieve method grpc definition (#5294)
See also: #5253 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-19 10:45:15 +00:00
quicksilver 3db653f2d0
Update cpp thirdparty source URL (#5295)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Resolves: #5293
2021-05-19 09:39:29 +00:00
neza2017 d68ee3fbc2
query base on timeline (#5279)
query base on timeline, add timestamp on these queries:
- ShowCollections
- HasCollection
- DescribeCollection

Resolves: #5219

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-18 09:12:17 +00:00
Cai Yudong e8ceb37f81
Add ShardsNum when create collection (#5245)
Updata PhysicalChannelNames and VirtualChannelNames based
on ShardsNum when create collection, and update collVChan2Chan
map.

Resolves: #5240 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-17 11:15:01 +00:00
yukun ecaef24fea
Change should_not to must_not in dsl (#5131)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-07 11:52:24 +00:00
FluorineDog b1974aede2
Merge knowhere unittests (#5139)
Re-enable knowhere unittests and merge them all together. 
Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-05-07 11:50:13 +00:00
dragondriver b7977698a8
Change the codec style of expr plan from text to binary (#5129)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-07 11:27:17 +00:00
dragondriver e5d4963ba0
Distinguish the dsl and expr in proxy and query node (#5118)
Pass a parameter `IsExpr` from proxy to query node, so query 
node can create search plan according to the variable `IsExpr`.
It makes `segcore` unnecessary to distinguish `dsl` or `expr`
using dynamic exception. When `IsExpr` is set to true, query
node will call `CreatePlanByExpr` according to the passed
information about expression. Otherwise query node will keep
still to use `CreatePlan` according to the `dsl` information. At
the same time, this pr adds some unittests to `CreatePlanByExpr`,
these unittests translate already exist case with `dsl` to case with
`expr`.

Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-07 07:20:47 +00:00
FluorineDog 396b3f33e9
Support TermExpr, NotExpr, LogicalExpr (#5096)
1. Support Term, like `A in [1, 2, 3]`
2. Support Not, like `! A < 3`
3. Support logical combination, like `A < 3 && B > 5 or C == 0`

Type: Feature

Signed-off-by: fluorinedog <fluorinedog@gmail.com>
2021-04-30 07:19:52 +00:00
shengjun.li b46ae44087
Zero size allocation in StackDeviceMemory (#5100)
When the remaining space is empty, the zero size allocation will
get the pointer `end_`. However, `cudafree` the pointer `end_` will
cause to crash.

Fix: #5078 #4770 #4412 #4340 #3646

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-04-30 04:21:02 +00:00
FluorineDog 88f5642603
Add plan proto and support basic boolean expr parser (#5088)
**What type of PR is this?**
- [x] Feature

**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
    like `A > 3 && not B < 5 or C in [1, 2, 3]`. 
2. Defines `plan.proto`, as Intermediate Representation (IR) 
    used between go and cpp. 
3. Support expr parser, convert predicate expr to IR
    in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
2021-04-29 08:48:06 +00:00
godchen 060f7ca0d2
Add get index build progress proto (#5049)
Add get index build progress proto.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-27 07:46:45 +00:00
shengjun.li a3e4339027
improve rhnsw (#5059)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-04-27 06:17:52 +00:00
neza2017 262c484cec
the return struct of "DescribeIndex" should contain "filed_name" (#5055)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-04-27 02:30:55 +00:00
shengjun.li a2875f9d95
Update knowhere (#5006)
Import performance of ivf::train and hnsw, and fix bugs
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2021-04-23 18:03:49 +08:00
Xiangyu Wang 82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
xige-16 1165db75f6 Optimize search performance
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-04-19 19:30:36 +08:00
cai.zhang 9ccaff8005 Add copyright
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-04-19 19:28:11 +08:00
FluorineDog 43b39cc9c9 Fix memory leak, temp solution
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-04-19 16:39:23 +08:00
FluorineDog 6059558698 Add license files
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-04-19 11:16:16 +08:00
cai.zhang a54097708d Adjust the DropIndex logic of the indexservice
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-04-16 15:37:13 +08:00
FluorineDog ce7a5ea699 Support segcoreinit
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-04-16 14:02:49 +08:00
xige-16 22eda19bc4 Fix mem leak when load index
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-04-10 10:10:53 +08:00
XuanYang-cn 46d27e37d7 Skip install gtest when build with test
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-04-08 10:01:18 +08:00
FluorineDog 1446cd5453 Fix flat unsupported bug
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-04-07 10:39:35 +08:00
xige-16 a6f1de036b Optimize search performance in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-30 22:16:58 +08:00
FluorineDog f39dcdb8f3 Support error code in segcore
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-03-26 16:18:30 +08:00