milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,482 Commits
144 Branches
119 Tags
578 MiB
Commit Graph

1148 Commits (51679fd436691ec09fefb096e51a2691f8eae1ac)

Author SHA1 Message Date
JinHai-CN adc6dc38bf Fix compiling error 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-27 10:25:07 +08:00
dongwei 2c6fcacdbd delete unused file 
Signed-off-by: dongwei <d279617552@gmail.com>
 2020-04-26 23:21:06 +08:00
Cai Yudong 150cca6bbf
Caiyd 1946 cpu2gpu getres (#2124) 
* #1946 use new log format in knowhere

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1946 add more debug info for GPU device initialize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-04-26 18:29:21 +08:00
Wang XiangYu 12b3bf0693
Add instance lock (#2060) 
* Add instance lock

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update message

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update unittest CMakeLists.txt

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
 2020-04-26 08:31:18 +00:00
DanielHuang1983 7ecafbd93a
rename the variable name collection_schema (#2126) 
Signed-off-by: DanielHuang1983 <huanghe888@hotmail.com>
 2020-04-26 08:03:52 +00:00
shana0325 53133aafac
train func add catch excetpion (#2096) 
* train func add catch excetpion

Signed-off-by: shana0325 <645842705@qq.com>

* Fix lint error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-26 11:29:20 +08:00
chen lu d22f3c2c5f
[skip ci] remove useless log (#2119) 
Signed-off-by: yiuluchen 1093633856@qq.com
 2020-04-26 10:19:31 +08:00
BossZou 8edb7b6a09
add cpu instruction sets check (#2055) 
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-04-26 00:59:25 +08:00
groot 777c36a0ac
#1997 index file missed after compact (#2108) 
* #1997 index file missed after compact

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-04-25 22:06:26 +08:00
JinHai-CN 9db19a9a20 Merge remote-tracking branch 'main/master' 2020-04-25 18:58:59 +08:00
GuoRentong fed865c7c9
Values of type 'const size_t' may not fit into the receiver type 'int' (#2109) 
Signed-off-by: GuoRentong <rt.guo@outlook.com>

Co-authored-by: guo rentong <grt@guodeMacBook-Pro.local>
 2020-04-25 18:35:04 +08:00
yxm1536 4c108c7578
Rename MemMenagerFactory.cpp/h to MemManagerFactory.cpp/h (#2093) 
* Rename MemMenagerFactory.cpp to MemManagerFactory.cpp

Signed-off-by: yxm1536 <62009483+yxm1536@users.noreply.github.com>

* Rename MemMenagerFactory.h to MemManagerFactory.h

Signed-off-by: yxm1536 <62009483+yxm1536@users.noreply.github.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-25 16:57:14 +08:00
czp b454f462c1
put error when call Flush (#2097) 
Signed-off-by: beautifulpython <2426009680@qq.com>
 2020-04-25 16:13:32 +08:00
grtoverflow 6fe2cf1580
Update ConfAdapter.cpp (#2106) 
Signed-off-by: grtoverflow <grtoverflow@gmail.com>
 2020-04-25 16:12:58 +08:00
xge-zilliz 42b1983625
Delete useless boolean (#2107) 
Signed-off-by: xge-zilliz <xi.ge@zilliz.com>
 2020-04-25 15:49:43 +08:00
dd-He c33df2ee20
Use emplace_back instead of push_back (#2103) 
Signed-off-by: dd-He <1843757857@qq.com>
 2020-04-25 15:37:23 +08:00
chengpu 6d2c50ac21
Use std::copy to replace forloop assignment (#2087) 
Signed-off-by: chengpu <chengpu@live.com>
 2020-04-25 15:28:31 +08:00
binbin12580 2247d58cc4
Unused variable: ids (#2088) 
Signed-off-by:  binbin12580 <1565196967@qq.com>
 2020-04-25 15:26:16 +08:00
Cross 7deef8029f
Update GrpcRequestHandler.cpp (#2085) 
Signed-off-by: Cross <xinyuanm9@gmail.com>
 2020-04-25 14:53:49 +08:00
groot 82f4c289eb
New api HasPartition, SearchByID, GetVectorsByID (#2056) 
* modify proto

Signed-off-by: groot <yihua.mo@zilliz.com>

* modify c sdk

Signed-off-by: groot <yihua.mo@zilliz.com>

* rewrite search by id

Signed-off-by: groot <yihua.mo@zilliz.com>

* change cases

Signed-off-by: zw <zw@zilliz.com>

* fix cases

Signed-off-by: zw <zw@zilliz.com>

* fix wal case

Signed-off-by: zw <zw@zilliz.com>

* fix search by id

Signed-off-by: groot <yihua.mo@zilliz.com>

* modify sdk

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* add search_by_ids case

Signed-off-by: zw <zw@zilliz.com>

* disable test_search)by_ids

Signed-off-by: zw <zw@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
 2020-04-25 14:49:32 +08:00
lwglgy 7e52815fb6
DBImpl.cpp::ExecWalRecord need to process error status in case wal::MXLogType::Entity (#2083) 
Signed-off-by: lwglgy <1486111482@qq.com>
 2020-04-25 13:24:24 +08:00
yukun 67d8a9b54c
Add http interface for hybrid search (#2079) 
* Add http interface for hybrid search

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add unittest for http hybrid interface

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* clang format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix codacy quality

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2020-04-25 13:19:17 +08:00
Jin Hai 6d87dc3c72
Merge pull request #2084 from lee-eve/master 
Unused variable: offsets
 2020-04-25 13:18:19 +08:00
water32 420d70dcc5
unused variable, just remove it. (#2082) 
Signed-off-by:  water32 <miaochen@mail.ustc.edu.cn>

unused variable, just remove it.
 2020-04-25 11:58:23 +08:00
lieve 6fea6a17de Unused variable: offsets 
Signed-off-by: lieve <645417202@qq.com>
Signed-off-by: lizhengxie <zhengxie.li@zuolin.com>
 2020-04-25 11:48:27 +08:00
JinHai-CN fb85d8b3f0 Fix lint error 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-25 11:21:58 +08:00
PJZero fedbb697e3
Update GrpcRequestHandler.cpp (#2069) 
Signed-off-by: pengjian05 <pengjian05@baidu.com>

Co-authored-by: pengjian05 <pengjian05@baidu.com>
 2020-04-25 11:19:30 +08:00
Jin Hai 54c9386bdc
#2076 CheckMetricConfigAddress error message (#2077) 
Signed-off-by: jinhai <hai.jin@zilliz.com>
 2020-04-25 10:04:04 +08:00
Jin Hai 723b10600b
#2073 Fix CheckDBConfigBackendUrl error message (#2074) 
Signed-off-by: jinhai <hai.jin@zilliz.com>
 2020-04-25 09:31:25 +08:00
Wang XiangYu c0d5a47eba
rename optimizer to selector (#2067) 
* rename

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>
 2020-04-24 11:42:17 +00:00
格雷大大 bf4cea1d1f
add log 4 annoy if index has been built when BuildAll interface invoked (#2071) 
* add log 4 annoy if index has been built when BuildAll interface invoked

Signed-off-by: grace <yan.gu@zilliz.com>

* fix lint error

Signed-off-by: grace <yan.gu@zilliz.com>
 2020-04-24 19:38:21 +08:00
Wang XiangYu 67819be29b
warn when use sqlite as metadata management (#2065) 
Signed-off-by: wxyu <xy.wang@zilliz.com>
 2020-04-24 15:23:46 +08:00
Yhz 87778052f4 move http parser to http utils 
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-04-23 23:16:04 +08:00
JinHai-CN a41949f075 Merge remote-tracking branch 'main/master' into AllenYu1987-patch-1 2020-04-23 14:08:28 +08:00
Jin Hai 9a3ae916a2
Merge pull request #2044 from Tlincy/Tlincy-patch-1 
Update LogUtils.cpp: use switch ... case to replace if ... else
 2020-04-23 14:03:54 +08:00
Jin Hai 3267ed406f
Merge pull request #2032 from ggaaooppeenngg/code-clean 
Delete CommandDto class and remove file CmdDto.hpp
 2020-04-23 14:01:48 +08:00
Allen Zhang f741be0a79 Move http query parser 
Signed-off-by: Allen Zhang <allen.zhangyilun@gmail.com>
 2020-04-23 12:14:11 +08:00
loguo fc969f73ba
Update DBImpl.cpp: rename flush_tables to flushed_collections (#2043) 
Signed-off-by: loguo <xiangzhouguo@gmail.com>
 2020-04-23 11:27:29 +08:00
Aredcap b55f249938
Assert cmd reply (#2030) 
* Assert cmd reply

Signed-off-by: Aredcap <zhangleilei19@mails.ucas.ac.cn>

* fix unittest error

Signed-off-by: Aredcap <zhangleilei19@mails.ucas.ac.cn>
 2020-04-23 11:22:58 +08:00
JinHai-CN 58d0fa56de Fix error 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-23 11:20:30 +08:00
moe-of-faith b23beebbfd
Change LogOut("[%s][%d]" ..... ) to LogOut("[%s][%ld]" ......) (#2041) 
Signed-off-by: moe-of-faith <aeon.sym@gmail.com>
 2020-04-23 11:03:10 +08:00
Gxz ce44da04e1
unused var: s (#2042) 
Signed-off-by: guoxiangzhou <8188755@qq.com>
 2020-04-23 11:02:39 +08:00
JinHai-CN 7d63d15718 Fix lint 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-23 10:53:33 +08:00
JinHai-CN 23ad133c14 Fix lint 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-23 10:45:34 +08:00
AllenYu1987 f4629b242e
Update GrpcRequestHandler.cpp: Use std::copy to replace forloop assignment 
Signed-off-by: AllenYu1987 <myextremelove@hotmail.com>
 2020-04-23 10:42:54 +08:00
phantom ea5c9f0178
Update core/src/scheduler/JobMgr.cpp: format include (#2046) 
Signed-off-by: qingyigudu <sore_loner@sohu.com>
 2020-04-23 10:25:46 +08:00
Tlincy 92508bcc0a
Update LogUtils.cpp: use switch ... case to replace if ... else 
Signed-off-by: Tlincy <540549770@qq.com>
 2020-04-23 09:35:51 +08:00
Yoya 44e8e02065
remove unused function "preprocessor" (#2034) 
Signed-off-by: Yoya <taydycn@gmail.com>
 2020-04-22 20:16:55 +08:00
ryjiang a3e4d29af6
Remove unused function (#2029) 
* [skip ci] Remove unused function

Signed-off-by:  shanghaikid <jiangruiyi@gmail.com>
Signed-off-by: ruiyi.jiang <ruiyi.jiang@zilliz.com>

* remove unused function in utils.cpp

Signed-off-by: ruiyi.jiang <ruiyi.jiang@zilliz.com>
 2020-04-22 20:15:09 +08:00
Fierralin 72cae5221c
Move NULL to nullptr (#2027) 
* Move NULL to nullptr

Signed-off-by: Fierralin <fierralin@hotmail.com>

* Try fix

Signed-off-by: Fierralin <fierralin@hotmail.com>
 2020-04-22 18:39:04 +08:00
Ben 9bfd48cdf5
move the definiation to cpp file (#2033) 
Signed-off-by: Ben <sckim2008@gmail.com>
 2020-04-22 15:16:37 +08:00
yukun ff9697b184
Change input format for attributes (#1990) 
* Change input of attributes

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix unittest for new attrs format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix float type for new format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Use new HEntity

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add sdk_hybrid

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add metric_type assign in CreateHybridCollection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix for clang-format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Read engine_type from extra_params

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove todo annotation

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove some comments

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add DescribeHybridCollection request

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove unused code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2020-04-22 06:12:46 +00:00
HesterG ba191b263a
rename table_schema_array (#2024) 
Signed-off-by: HesterG <hestergong@gmail.com>
 2020-04-22 12:01:01 +08:00
Jin Hai aca299166c
Merge pull request #2026 from mileyzjq/code_polish 
Fix unused code and clarify params
 2020-04-22 11:59:29 +08:00
Peng Gao f7351e7482 Delete CommandDto class and remove file CmdDto.hpp 
Signed-off-by: Peng Gao <peng.gao.dut@gmail.com>
 2020-04-22 11:46:54 +08:00
sahuang 9e78cde3f2 Fix 
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-04-22 10:47:12 +08:00
Lin-gh-Saint 04db47fa66
fix warning: no return statement (#2025) 
Signed-off-by: Lin-gh-Saint <linguanhaohyq@gmail.com>
 2020-04-22 10:03:11 +08:00
HesterG 68f7a0fa77
rename table_schema to collection_schema (#2023) 
Signed-off-by: HesterG <hestergong@gmail.com>
 2020-04-22 10:02:19 +08:00
Accagain c7eb9b0fa2
add safty check 4 annoy load index (#2022) 
Signed-off-by: cms <chenmaosen0@gmail.com>
 2020-04-22 10:00:46 +08:00
nezha2017 00e76219f6
use #include "segment/SegmentWriter.h" instead (#2020) 
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2020-04-22 09:46:33 +08:00
mileyzjq 72d80aca74 fix unused code and clarify params 
Signed-off-by: mileyzjq <mileyzha@gmail.com>
 2020-04-21 22:08:50 +08:00
shiyu09 0320259976
rename table_info into collection_info (#2016) 
Signed-off-by: shiyu09 <39143280+shiyu09@users.noreply.github.com>
 2020-04-21 20:30:16 +08:00
adream307 a3d99de5de
Context-head-file (#2021) 
* Context-head-file

Signed-off-by: adream307 <adream307@163.com>

* clang-format

Signed-off-by: adream307 <adream307@163.com>

Co-authored-by: adream307 <adream307@163.com>
 2020-04-21 19:04:06 +08:00
nameczz 5bcd3034c6
remove unused function "normalize_vector" (#2015) 
Signed-off-by: nameczz <nameczz@126.com>
 2020-04-21 15:53:17 +08:00
bo huang 574ff66c01
add safty check 4 annoy load index (#2009) 
Signed-off-by: bo-huang <1013767522@qq.com>
 2020-04-21 14:23:42 +08:00
Cai Yudong f039032c8c
Caiyd 1965 unify metric cal (#2008) 
* #1965 add metric algorithm benchmark for FAISS

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for NSG

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for ANNOY

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for HNSW

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* calculate average time

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 annoy/nsg/hnsw all use faiss distance algorithm

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 support AVX512/AVX2/SSE42

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 fix hnsw ip calculation error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-04-21 14:22:33 +08:00
groot a5eec9d7b4
create index notify (#2003) 
* create index notify

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-04-21 14:22:18 +08:00
xudalin0609 9e30c468ff
remove unused var (#2006) 
Signed-off-by: xudl <isxudl@163.com>

Co-authored-by: xudl <isxudl@163.com>
 2020-04-21 12:13:23 +08:00
Morry Niu 75ca0d96b6
keep bitset naming consistent (#2004) 
Signed-off-by: Xinyao Niu <morryniu123@gmail.com>
 2020-04-21 09:40:49 +08:00
JUN a2e7a91964
Modify data type in CommonUtil.cpp (#1996) 
Signed-off-by: JUN <gujun720@foxmail.com>
 2020-04-20 20:42:16 +08:00
Jin Hai 315dac3338
Automerge1 (#1984) 
* [skip ci] remove auto-merge git-action

Signed-off-by: jinhai <hai.jin@zilliz.com>

* [skip ci] update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Stop strip the binary

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix warning

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-20 20:39:22 +08:00
Yaning Zhong 814b8f0c53
move const char* to .cpp file (#1994) 
Signed-off-by: ynzhong <zhongyn522@gmail.com>

Co-authored-by: ynzhong <zhongyn522@gmail.com>
 2020-04-20 15:41:38 +08:00
Tracy199 c50de3de9a
use TimeRecorder instead of chrono (#1989) 
Signed-off-by: Tracy199 <houkx@vip.sina.com>
 2020-04-20 01:29:11 +00:00
HuangHua 008f24a7c2
replace metaOptions with meta_options (#1979) 
* replace metaOptions with meta_options

Signed-off-by: huanghua <bruce.huang@hotmail.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: groot <yihua.mo@zilliz.com>
 2020-04-19 08:00:50 +00:00
smiling849 50ddc49332
fix hnsw code format (#1987) 
Signed-off-by: Zhonglei Song <szl_84@sina.com>
 2020-04-19 12:00:45 +08:00
bigbraver e8f759c22b
code optimize (#1986) 
Signed-off-by: bigbraver <bigbraver@qq.com>
 2020-04-19 03:54:23 +00:00
Hanyang Ke 25b6efc8e8
fix warning (#1988) 
Signed-off-by: kehycs <kehycs@foxmail.com>
 2020-04-19 11:12:01 +08:00
anchun 29acf6a594
update some variables in validation utils. (#1978) 
Signed-off-by: anchun <nkterry@163.com>
 2020-04-18 21:59:26 +08:00
zaqwsxqwer111 54fb8ef357
remove pragma once from .cpp files (#1982) 
Signed-off-by: zaqwsxqwer111 <2536173042@qq.com>
 2020-04-18 20:28:10 +08:00
zhangchen b02a37d79d not to printf the log when record type is None 
Signed-off-by: zhangchen <zhangch5n@163.com>
 2020-04-18 17:29:12 +08:00
Rayn Wei 375fd709a9
. (#1972) 
Signed-off-by: RyanWei <rainmanwy@gmail.com>
 2020-04-18 13:22:47 +08:00
groot 85de706d22
#1929 Skip mysql meta schema field width check (#1937) 
* #1929 Skip mysql meta schema field width check

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* merge master

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-04-18 10:26:22 +08:00
Jin Hai 59035d9892
#1454 Improve code quality (#1948) 
* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix issue

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-17 21:05:17 +08:00
yukun 3e1b2ab4a0
Filtering by numeric scalar fields prototype (#1919) 
* Add hybrid request handler

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add C++ sdk for createcollection and insertentities

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add RequestHandler

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add test case for hybrid insert

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix sqlite bug for createcollection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridQuery Handler DBImpl and ExecBinaryQuery

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridSearch sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridSearch test case

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix HybridSearch bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix HybridSearch crash bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change void* to vector<uint8_t> in Attr codec

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add context and new search task

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add merge for Hybrid

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add AST validation

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add unittest for hybrid

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix hybrid search nq bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix bugs after merge master

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix clang format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix unittest bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix Codacy

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix compact unittest bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove grpc request in hybridsearchcontext

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix some codacy quality issue

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix HYBRID_DB_TEST bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Annotate new search task

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add Hybrid RPC handler unittest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix logs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridSearch unittest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2020-04-16 14:54:12 +08:00
groot 93244a2a79
#1705 Limit the insert data batch size (#1936) 
* #1705 Limit the insert data batch size

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-04-16 00:18:23 +08:00
Jin Hai 0323aa1aad
Merge 080 (#1940) 
* #1910 C++ SDK GetIDsInSegment could not work for large dataset (#1911)

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1903 Fix invalid annoy result (#1912)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* #1914: Partition max size should be 4096 (#1915)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* add log (#1913)

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* partition limit 4096

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix py test

Signed-off-by: groot <yihua.mo@zilliz.com>

* update server version (#1916)

Signed-off-by: zw <zw@zilliz.com>

* Update to 0.8.0 (#1918)

* Create new branch 0.8.0 and change preload_table to preload_collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* update helm version

Signed-off-by: zw <zw@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>

* fix issue 1901 (#1920)

* fix issue 1901

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>

* #1900 (#1923)

* add log

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix #1900

Signed-off-by: groot <yihua.mo@zilliz.com>

* Upgrade mishards to 0.8.0 (#1933)

* update grpc server of milvus & rename table name to collection

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Skip CI

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Caiyd 1883 fix rw (#1926)

* #1883 use DiskIO

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix logic error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>

* #1928 Too many data and uid copies when loading files (#1931)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>

* Update mishards configure files (#1938)

* Update web readme

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] update configure files

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] rename table to collection

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update test.groovy

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update test.groovy

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

Co-authored-by: groot <yhmo@zeronedata.com>
Co-authored-by: shengjun.li <49774184+shengjun1985@users.noreply.github.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
Co-authored-by: zw <zw@zilliz.com>
Co-authored-by: op-hunter <ophunter52@gmail.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
 2020-04-15 21:32:20 +08:00
Wang XiangYu 890fe08e7c
Refactor LOG macro and improve log format (#1927) 
* update log format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add new log macro

Signed-off-by: wxyu <xy.wang@zilliz.com>

* use new log macro instead

Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com>

* add SetThreadName function

Signed-off-by: wxyu <xy.wang@zilliz.com>

* clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* set thread name

Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com>

* add changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add git ignore .swp file

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update log level in LogUtil.cpp

Signed-off-by: wxyu <xy.wang@zilliz.com>
 2020-04-15 14:44:00 +08:00
Cai Yudong a63d832772
#1921 use TimeRecorder instead of chrono (#1922) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-04-13 18:42:31 +08:00
Jin Hai f8a28699b4
Change to 0.8.0 and rename from preload_table to preload_collection (#1904) 
* Create new branch 0.8.0 and change preload_table to preload_collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* update helm version

Signed-off-by: zw <zw@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>
 2020-04-11 19:01:29 +08:00
groot ad4645fce0
check task table limit (#1902) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-04-11 13:13:32 +08:00
shengjun.li b04b3e942c
#1897 add heap_swap_top (#1898) 
* add heap_swap_top

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix wrong code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-04-11 12:56:32 +08:00
Wang XiangYu 3bc17d8cc2
Fix multi client search crash in tracing module fix #1789 fix #1832 (#1899) 
* Fix multi client search crash in tracing module fix #1789 fix #1832

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add lock for every context_map_ access

Signed-off-by: wxyu <xy.wang@zilliz.com>

* remove never used variable

Signed-off-by: wxyu <xy.wang@zilliz.com>
 2020-04-11 12:12:38 +08:00
groot c8a59b273c
#1276 SQLite throw exception when create 50000+ partitions in a table (#1879) 
* #1276 SQLite throw exception when create 50000+ partitions in a table

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1276 SQLite throw exception when create 50000+ partitions in a table

Signed-off-by: groot <yihua.mo@zilliz.com>

* remove some log

Signed-off-by: groot <yihua.mo@zilliz.com>

* remove some log

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-10 22:51:01 +08:00
groot e25c5af2d7
rename table to collection (#1888) 
* rename table to collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* rename collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* rename

Signed-off-by: groot <yihua.mo@zilliz.com>

* Update change log

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-10 19:17:33 +08:00
groot 9074366f42
fix potential hang bug (#1884) 
* test

Signed-off-by: groot <yihua.mo@zilliz.com>

* test

Signed-off-by: groot <yihua.mo@zilliz.com>

* test

Signed-off-by: groot <yihua.mo@zilliz.com>

* unittest

Signed-off-by: groot <yihua.mo@zilliz.com>

* format code

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix crash

Signed-off-by: groot <yihua.mo@zilliz.com>

* merge master

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-04-10 09:49:31 +08:00
Cai Yudong a05ec6a972
Caiyd 1885 optimize knowhere ut (#1889) 
* optimize knowhere unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code clean

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize knowhere util

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest build error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
 2020-04-09 23:37:13 +08:00
BossZou 354f29a058
Refactor log on search and insert request (#1887) 
* fix stash conflict

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Log config information and device information (fix #1893)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add index annoy into http module (fix #1882)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Refactor log on search and insert request (fix #1886)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* use vsnprintf to avoid oom

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix PR Quality problem & remove surplus comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* replace strlen with strnlen & remove some surplus comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-04-09 21:04:18 +08:00
Cai Yudong 66e38d20cd
Caiyd 1881 bad alloc (#1892) 
* #1881 update storage APIs, use int64_t for size

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-04-09 17:13:15 +08:00
Cai Yudong cc3b75b6b7
code revert (#1877) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-04-07 18:58:18 +08:00
BossZou a4b31cb02f
[skip ci] Fix incorrect information in http readme (fix #1812) (#1878) 
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-04-07 18:56:19 +08:00
Cai Yudong bf6d22e2f5
#1873 fix index file serialize to incorrect path (#1874) 
* #1873 fix index file serialize to incorrect path

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* not create sq8h index when gpu disabled

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-04-07 16:25:46 +08:00
groot c824c297c1
build index wait interval (#1872) 
* #1827 Combine request target vectors exceed max nq

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* merge master

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine meta code

Signed-off-by: groot <yihua.mo@zilliz.com>

* set build index check interval to 5 seconds

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
 2020-04-07 10:05:37 +08:00
groot 6a960cda2c
refine meta code (#1871) 
* #1827 Combine request target vectors exceed max nq

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* merge master

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine meta code

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
 2020-04-06 18:43:20 +08:00
groot 0fd4b244c0
refine dbimpl (#1869) 
* #1827 Combine request target vectors exceed max nq

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failure

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
 2020-04-04 21:14:30 +08:00
Jin Hai 35276ffcb0
Rename from table to collection (#1857) 
* Change HasTable to HasCollection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change HasTable to HasCollection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change table to collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change Table to Collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change Table to Collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change Table to Collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change Table to Collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change Table to Collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix Unit test

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-03 23:12:41 +08:00
Cai Yudong dc707bb8a3
Caiyd 1858 disable s3 build (#1860) 
* #1858 disable s3 build

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable s3 in unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-04-03 16:35:48 +08:00
op-hunter 54337e9361
refactor knowhere inner unittest for annoy and hnsw, which a… (#1855) 
* skip ci, refactor knowhere inner unittest for annoy and hnsw, which aimed to remove logs of std output

Signed-off-by: cmli <chengming.li@zilliz.com>

* skip ci, fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix annoy load bugs and add serialize test 4 annoy and hnsw

Signed-off-by: cmli <chengming.li@zilliz.com>

* update changelog

Signed-off-by: cmli <chengming.li@zilliz.com>

* update lower bound of annoy search_k param

Signed-off-by: cmli <chengming.li@zilliz.com>

* lint errorgit add src/utils/ValidationUtil.cpp

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-04-03 14:14:04 +08:00
Cai Yudong 7ccfa5b9f0
Caiyd 1627 move rw index (#1833) 
* #1627 move read/write index into codec

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update WriteVectorIndex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-04-02 22:44:45 +08:00
Xiaohai Xu f82aa4f493
#1849 Support NSG Deletion (#1839) 
* nsg delete

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix request

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix clang

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix typo

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* [skip ci] add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix test_nsg and changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix test

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix test case

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix clang-format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Polish

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-04-02 20:46:58 +08:00
groot f66b43b81b
#1827 Combine request target vectors exceed max nq (#1847) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-04-02 14:24:28 +08:00
Jin Hai 3e03850169
Merge from 0.7.1 to master (#1840) 
* update mishards images to 0.7.1 (fix #1799) (#1802)

* update mishards images to 0.7.1 (fix #1799)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Skip CI

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix lsn meta bug (#1813)

* fix lsn meta bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix mysql lsn meta bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* Update CHANGELOG.md (#1814)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1815)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1816)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1817)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Increase Resources

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Decrease the compiling thread

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #1818 Duplicate data generated after restart milvus server (#1819)

* #1818 Duplicate data generated after restart milvus server

Signed-off-by: groot <yihua.mo@zilliz.com>

* set compile thread number

Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip-ci]Remove auto flush interval from config (#1837)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compile error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: groot <yhmo@zeronedata.com>
 2020-04-02 14:20:33 +08:00
Jin Hai 0e0739ef87
1845 no UT log (#1846) 
* #1845 No UT log

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* No UT log

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-02 09:48:29 +08:00
op-hunter 6e7fb831bd
add annoy params validate check (#1829) 
* fix bug & solve compile warning from annoylib

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
 2020-04-02 09:47:45 +08:00
Xiaohai Xu 2fcf2855ce
fix sq8h deletion (#1830) 
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-04-01 20:50:48 +08:00
Jin Hai 68625cbf1c
Fix 1724 (#1821) 
* #1724 Remove unused unittests

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part2

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part3

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part4

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part5

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change table_id to collection_id

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change table to collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change table_ to collection_

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change table_id_ to collection_id_

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Change table_name to collection_name

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Reformat

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change TableFile to Segment

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change TableSchema to CollectionSchema

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix compile lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-04-01 17:37:55 +08:00
Cai Yudong 531fca8869
Caiyd 1655 gpu ivfflat delete (#1809) 
* support GPUIVF delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* calc bitset offset correctly

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support GPUSQ8 delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support GPUIVFPQ delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest for GPU delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* delete FAISS compile option -DFAISS_USE_FLOAT16

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-04-01 10:42:48 +08:00
Xiaohai Xu 88cf5e2bd8
Optimize functions (#1803) 
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-03-30 18:11:08 +08:00
Cai Yudong dbbf91db28
remove unnecessary lock (#1801) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-30 14:13:02 +08:00
Xiaohai Xu 8a731dea39
Add Unittest for IDMAP (#1774) 
* gpu idmap unittest

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix clang format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix build issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* small fix

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix case

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix back

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Compact code

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* efficient code

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix GPU search hang

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-03-30 14:08:56 +08:00
JinHai-CN 4574f9d998 Merge branch '0.7.1' 2020-03-29 18:50:10 +08:00
shengjun.li 52ac810a1f
fix share_ptr array (#1786) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-03-28 20:27:07 +08:00
Tinkerrr 3cdba5b62e
fix (#1790) 
Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>
 2020-03-28 19:27:21 +08:00
groot a3935eed71
unitest validation (#1788) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-28 19:26:39 +08:00
BossZou 4cd5d8dd1b
Add Superstructure/Substructure in http module (#1785) 
* add sub/super-structure in http module

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update http test case

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add Superstructure/Substructure in http module (fix #1784)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-03-28 14:41:24 +08:00
groot ebc6c69b95
add qps sdk example (#1779) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-28 13:57:23 +08:00
Cai Yudong a7932b0365
#1781 fix search hang with SQ8H (#1783) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-28 13:08:42 +08:00
groot af73dccb1c
fix combine result bug (#1780) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-27 23:01:02 +08:00
Cai Yudong ac443e37ac
Caiyd 1689 gpu out of memory (#1776) 
* #1689 update cache stragegy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1689 add lock when change cache capacity

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1689 update cache

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-27 19:24:10 +08:00
Jin Hai d92bcd2e54
Update CHANGLOG.md (#1771) 
* Update CHANGLOG.md

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-03-27 14:02:54 +08:00
op-hunter 4b996cd622
fix commpile error because of interface changed (#1773) 
* fix commpile error because of interface changed

Signed-off-by: lichengming <chengming.li@zilliz.com>

* fix comiple error

Signed-off-by: lichengming <chengming.li@zilliz.com>

Co-authored-by: lichengming <chengming.li@zilliz.com>
 2020-03-27 13:07:57 +08:00
groot e4264d8461
Server is not forbidden to create new partition which tag is _default (#1770) 
* HasTable with a not exist table name, return error status

Signed-off-by: groot <yihua.mo@zilliz.com>

* Server is not forbidden to create new partition which tag is _default

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-27 10:00:51 +08:00
op-hunter 310d5d70bc
Annoy support (#1746) 
* add annoy source code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* add annoy knowhere

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* annoy local gtest passed

Signed-off-by: lichengming <chengming.li@zilliz.com>

* fix lint error and update changelog

Signed-off-by: lichengming <chengming.li@zilliz.com>

* fix compile error

Signed-off-by: cmli <chengming.li@zilliz.com>

* Update connect timeout in test cases

Signed-off-by: zw <zw@milvus.io>

* fix some potential bugs

Signed-off-by: cmli <chengming.li@zilliz.com>

* retry ci

Signed-off-by: cmli <chengming.li@zilliz.com>

* rerun ci!

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix errors tested by c++ sdk

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: lichengming <chengming.li@zilliz.com>
Co-authored-by: zw <zw@milvus.io>
 2020-03-27 09:52:31 +08:00
Cai Yudong 207b854c3f
Caiyd 1655 gpu ivfflat delete (#1767) 
* update sdk

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update segment interfaces

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update some interfaces

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-26 21:58:48 +08:00
groot 0da3c712ec
HasTable with a not exist table name, return error status (#1761) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-26 19:59:49 +08:00
Xiaohai Xu f93b464172
#1654 GPU Index Flat Delete (#1736) 
* Add Flat Index Delete

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix log

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix bitset

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix reference

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix bug

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix bug

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix bug

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
 2020-03-26 18:36:46 +08:00
Tinkerrr 243e843aac
fix #1747 (#1757) 
* fix

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update.

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* lint pass

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* UPDATE CASE: search with tag not exist

Signed-off-by: zw <zw@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>
 2020-03-26 18:28:24 +08:00
shengjun.li e9f7422bc5
fix binary flat may retruns wrong distance when nq larger than 1 (#1760) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-03-26 16:45:49 +08:00
groot c4b7be8ed6
#1728 Optimize request handler to combine similar query (#1743) 
* modify changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* improve search qps

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix hang bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix combine request result bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* add unittest for combine request

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failure

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-26 14:26:02 +08:00
Cai Yudong 578d2633eb
#1756 fix memory exhausted during searching (#1758) 
* #1756 fix memory exhausted during searching

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1756 update const names

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-26 13:55:11 +08:00
Cai Yudong d34bd505ba
#1735 fix search fail with gpu out of memory using ivf_flag (#1748) 
* #1735 fix search fail with gpu out of memory using ivf_flag

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1735 add lock for each GpuCacheMgr instance

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1735 optimize cache debug info

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1735 set smaller cache threshold

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1735 update cache debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1735 fix cache issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-25 19:54:50 +08:00
Tinkerrr cda57cf77f
Update NSG (#1744) 
* enable IP and fix crash

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update.

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* lint pass

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>
 2020-03-25 16:24:18 +08:00
groot e865e9c893
modify changelog (#1739) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-24 17:48:13 +08:00
groot 0e914949e8
#1734 Opentracing for combined search request (#1737) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-24 14:44:01 +08:00
Jin Hai f4d7dd6c93
#1724 Remove unused unittests (#1725) 
* #1724 Remove unused unittests

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part2

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part3

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part4

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part5

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-03-24 10:11:10 +08:00
shengjun.li 5fc875a020
#1603 modify substructure/superstructure to perfect match (#1718) 
* modify substructure/superstructure to perfect match

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* Update cases

Signed-off-by: zw <zw@zilliz.com>

* Fix case bug

Signed-off-by: zw <zw@zilliz.com>

* set invalid distance infinite

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* Add distance cases

Signed-off-by: zw <zw@zilliz.com>

* Fix test cases

Signed-off-by: zw <zw@zilliz.com>

* Re-trigger ci

Signed-off-by: zw <zw@zilliz.com>

* fix wrong code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* Fix test case

Signed-off-by: zw <zw@zilliz.com>

* Fix case

Signed-off-by: zw <zw@zilliz.com>

* Fix cases

Signed-off-by: zw <zw@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>
 2020-03-24 10:10:56 +08:00
groot 99470b4dfa
#1702 Optimize request handler to combine similar query (#1727) 
* #1686 API search_in_files cannot work correctly when vectors is stored in certain non-default partition

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #1686 API search_in_files cannot work correctly when vectors is stored in certain non-default partition

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine request code

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* combine search request

Signed-off-by: groot <yihua.mo@zilliz.com>

* combine search request

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1728 Optimize request handler to combine similar query

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* remove some log
 2020-03-23 18:26:18 +08:00
op-hunter 4fa45dc754
#1661 support HNSW deletion on nmslib (#1729) 
* support HNSW deletion on nmslib

Signed-off-by: lichengming <chengming.li@zilliz.com>

* update changelog

Signed-off-by: lichengming <chengming.li@zilliz.com>

* fix lint error on test_hnsw.cpp

Signed-off-by: lichengming <chengming.li@zilliz.com>

Co-authored-by: lichengming <chengming.li@zilliz.com>
 2020-03-23 18:04:46 +08:00
shengjun.li e3786a24f4
IVFPQ has supported to delete vectors (#1722) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-03-23 13:50:27 +08:00
Cai Yudong 538c377e65
Caiyd 1689 fix sq8h fail (#1723) 
* #1689 fix SQ8H search fail on SIFT-1B dataset

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix bug

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix gpu search fail

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix db_test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* revert segment back

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_gpuresource

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_gpuresource

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-23 10:48:15 +08:00
Jin Hai 450573849b
#267 Improve search performance: reduce delay (#1717) 
* #267 Improve search performance: reduce delay

Signed-off-by: jinhai <haijin.chn@gmail.com>

* #267 Improve search performance: reduce delay - update CHANGELOG

Signed-off-by: jinhai <haijin.chn@gmail.com>

* #267 Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #267 Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #267 Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #267 Update format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-03-23 09:26:58 +08:00
Jin Hai 9443af3e87
#1719 Remove unused log (#1720) 
Signed-off-by: jinhai <hai.jin@zilliz.com>
 2020-03-21 12:48:41 +08:00
groot fe2595fa38
#1686 API search_in_files cannot work correctly when vectors is store… (#1701) 
* #1686 API search_in_files cannot work correctly when vectors is stored in certain non-default partition

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #1686 API search_in_files cannot work correctly when vectors is stored in certain non-default partition

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-20 12:14:07 +08:00
Cai Yudong 5950deddfc
#1678 remove CUSTOMIZATION macro (#1707) 
* #1678 remove CUSTOMIZATION macro

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable faiss_benchmark

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-20 10:24:27 +08:00
Cai Yudong 588ef95d76
Caiyd refactor knowhere (#1687) 
* add new knowhere

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* build pass

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* update

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* rename algo

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* update...

Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>

* add archive

Signed-off-by: Nicky <nicky.xj.lin@gmail.com>

* add new APIs: GetVectorById/SearchVectorById/SetBlacklist/GetBlacklist

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* all unittest pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* test_binary pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move knowhere into namespace milvus

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update hnsw

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ConfAdapterMgr

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ExecutionEngineImpl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move SetBlacklist/GetBlacklist to VecIndex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update VectorAdapter and rename SearchById to QueryById

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update interface in ExecutionEngineImpl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* milvus build pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* milvus IDMAP sdk_simple pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_schedule

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix CPU version Milvus build issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update BinaryIVF BuildAll

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update VecIndexFactory

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ConfAdapter

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix knowhere unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix SPTAG unittest

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix CPU version unittest build issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix db_test

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_engine

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix CPU version build issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* change BinarySet key back

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* change IndexType to string, and add compatible API for 0.7.0

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix IndexHNSW build warning

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_cache

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix write_index error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code clean

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: Nicky <nicky.xj.lin@gmail.com>
Co-authored-by: xiaojun.lin <xiaojun.lin@zilliz.com>
 2020-03-19 19:24:20 +08:00
shengjun.li 72ad100a90
modify bitset from deque to vector (#1696) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-03-19 16:16:14 +08:00
op-hunter fdd51400d2
#1660 support IVFPQ CPU delete (#1695) 
* support IVFPQ CPU delete

Signed-off-by: lichengming <chengming.li@zilliz.com>

* update CHANGELOG.md

Signed-off-by: lichengming <chengming.li@zilliz.com>

Co-authored-by: lichengming <chengming.li@zilliz.com>
 2020-03-19 12:26:03 +08:00
BossZou 504a9e30ab
Optimize config cpu_cache_capacity / gpu_cache_capacity setter (#1572) (#1629) 
* add gpu cache config handler

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* remove cpu/gpu cache mgr from Config class by using cache config handler (fix #1572)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* remove 0.8.0 from config version map

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* clean config header reference

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix bug in web readme

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* reduce gpu config handler to gpu resources handler

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add engine config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* modify handler hook(fix #1572)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* initalize value in handler by config default

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* code style format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix compile error in release mode

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* resolve faiss blas threshold init in DBWrapper

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* modify cache header

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* remove comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* order headers

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* convert gpu res config to lower case

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* adjust header order in cpu cache mar file

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* improve config test case

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-03-19 10:17:53 +08:00
Xiaohai Xu 1543f8dc37
#1653 IndexFlat performance improvement for NQ < thread_number (#1690) 
* optimize sse

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* optimizer BinaryDistance

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix superstructure

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: Jin Hai <hai.jin@zilliz.com>
 2020-03-18 18:27:57 +08:00
groot 55ecfd5930
#1663 PQ index parameter m validation (#1677) 
* #1663 PQ index parameter m validation

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failure

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-18 18:26:38 +08:00
Xiaohai Xu 59dab6cb84
#1653 IndexFlat performance improvement for NQ < thread_number (#1674) 
* Optimize index flat L2/IP for SSE

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* parallel optimization

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix threshold

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

Co-authored-by: sahuang <xiaohai.xu@zilliz.com>
 2020-03-16 21:32:05 +08:00
Zhiru Zhu 3de34d3831
#1619 (#1620) 
* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>

* update

Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>
 2020-03-16 19:33:28 +08:00
groot 802d62eb24
#1651 Check validity of dimension when collection metric type is binary one (#1666) 
* #1648 The cache cannot be used all when the type is binary

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1646 The cache cannot be used all when the type is binary

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1646 The cache cannot be used all when the type is binary

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1651 Check validity of dimension when collection metric type is binary one

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2020-03-16 16:26:12 +08:00
groot 6c826c1308
#1648 The cache cannot be used all when the type is binary (#1648) 
* #1648 The cache cannot be used all when the type is binary

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1646 The cache cannot be used all when the type is binary

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1646 The cache cannot be used all when the type is binary

Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-15 20:45:11 +08:00
Cai Yudong 512fe6233c
#1649 fix Milvus crash on old cpu (#1652) 
* #1649 fix Milvus crash on old cpu

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1649 update debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-14 18:03:23 +08:00
shengjun.li 4088f5e9a2
#1603 BinaryFlat add 2 Metrics: Substructure and Superstructure (#1647) 
* add substructure & superstructure

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* support c++sdk by lin.xiaojun

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* optimizer omp for BinaryFlat; add tutorial

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
 2020-03-14 10:03:00 +08:00
shengjun.li 7c3a673bfe
check table lsn (#1622) 
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
 2020-03-13 11:56:12 +08:00
BossZou dfa25f88e2
fix demo bug in HTTP doc (fix #1634) (#1640) 
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-03-13 11:16:55 +08:00
denin 055041881c fix crashes 
Signed-off-by: denin <dvzubarev@yandex.ru>
 2020-03-12 17:29:54 +03:00
yudong.cai 4cf2b2b392 Merge remote-tracking branch 'main/master' into caiyd_1548_move_store 2020-03-12 20:05:57 +08:00
Zhiru Zhu 04f00bbe91
update 
Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>
 2020-03-12 16:43:18 +08:00
yudong.cai 058c993a99 update header file 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-12 16:26:56 +08:00
yudong.cai 817ea8b9e1 #1548 add FSHandler 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-12 11:40:12 +08:00
yudong.cai 28a0f1de01 #1548 move store/Directory.cpp to storage 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2020-03-12 11:38:17 +08:00
Lutkin Wang c24e2a0111
Update README.md 
Signed-off-by: Lutkin Wang <lu.wang@zilliz.com>
 2020-03-12 09:34:25 +08:00
yudong.cai 67fad34065 Merge remote-tracking branch 'main/master' into caiyd_1547_rename_storage_file 2020-03-11 18:43:02 +08:00
Jin Hai 87214f742b
Merge pull request #1550 from cydrain/caiyd_1546_move_config 
Caiyd 1546 move config
 2020-03-11 18:41:35 +08:00
Jin Hai ad0f26f98e
Merge pull request #1538 from cydrain/caiyd_codec_opt 
Caiyd codec opt
 2020-03-11 18:41:17 +08:00
yudong.cai 539ce69305 Merge remote-tracking branch 'main/master' into caiyd_1547_rename_storage_file 2020-03-11 15:31:04 +08:00
JinHai-CN 664a1ab4e1 Fix format 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-03-11 14:06:47 +08:00
JinHai-CN a5cb8f994d Update time print log 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-03-11 13:36:48 +08:00
JinHai-CN fbe5531ba0 Test 
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
 2020-03-11 12:06:15 +08:00
yudong.cai 98da92afd2 Merge remote-tracking branch 'main/master' into caiyd_1547_rename_storage_file 2020-03-11 11:25:10 +08:00
yudong.cai 6454943aca Merge remote-tracking branch 'main/master' into caiyd_1546_move_config 2020-03-11 09:40:42 +08:00
groot 6e6d06ca4a #1598 Server down during mixed operations 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-11 09:30:39 +08:00
Jin Hai 81d767ea2f
Merge pull request #1612 from youny626/fix#1598 
Fix #1598
 2020-03-11 00:18:30 +08:00
yudong.cai 49db61ea50 Merge remote-tracking branch 'main/master' into caiyd_1546_move_config 2020-03-11 00:10:22 +08:00
Zhiru Zhu 50804ecaf5
update 
Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>
 2020-03-10 22:42:59 +08:00
yudong.cai b50f23d4d6 Merge remote-tracking branch 'main/master' into caiyd_1546_move_config 2020-03-10 22:09:27 +08:00
Zhiru Zhu 5b0b226bc4
update 
Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>
 2020-03-10 22:03:17 +08:00
Zhiru Zhu 404d607eb6
update 
Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>
 2020-03-10 21:34:47 +08:00
Zhiru Zhu 561e2f5fc8
update 
Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>
 2020-03-10 21:07:11 +08:00
groot 120ea4d0e3 #1609 Refine Compact function 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-10 19:56:06 +08:00
groot 8872054603 #1609 Refine Compact function 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-10 19:47:23 +08:00
groot 17ec3d5e21 #1609 Refine Compact function 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2020-03-10 18:58:50 +08:00
Jin Hai 2a6e70d5f0
Merge branch 'master' into fix#1598 2020-03-10 18:10:23 +08:00
Jin Hai 17d2618544
Merge pull request #1602 from BossZou/http-doc 
fix link error in http doc
 2020-03-10 17:31:24 +08:00
Zhiru Zhu 9d4b7fda62
update 
Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>
 2020-03-10 16:40:53 +08:00
Zhiru Zhu 4625bf3710
update 
Signed-off-by: Zhiru Zhu <zzhu@fandm.edu>
 2020-03-10 16:40:17 +08:00
Yhz f722119188 add options methods for system 
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
 2020-03-10 15:59:08 +08:00
Jin Hai 442bf43402
Merge pull request #1600 from yhmo/binary 
sdk typo
 2020-03-10 15:22:34 +08:00