Commit Graph

557 Commits (b548b38222de91569b430280b65dc017b20a580c)

Author SHA1 Message Date
shengjun.li b548b38222
Fix the result merging of IVF_PQ IP (#2955)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-22 09:53:08 +00:00
op-hunter 0e0a731ddd
fix a bug of annoy search parameter exisitence check (#2958)
Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-22 08:11:59 +00:00
Xiaohai Xu 7ba30c7830
#2890 Fix wrong index size (#2930)
* fix update index size

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add GetSize() interface 4 SPTAG

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix index size of index hnsw, annoy and nsg

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix clang

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add sdk 4 non-ivf index test

Signed-off-by: cmli <chengming.li@zilliz.com>

* remove std::cout and make clang-format

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix cout

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* checkout sdk

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix binary ivf

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix hnsw

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-21 06:02:44 +00:00
groot f80c592679 fix #2768 (#2809)
* fix #2752

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix #2768

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo again

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-07-18 15:24:10 +08:00
cqy123456 01437cef39
Fix warning master (#2894)
* fix clean unused variable warning in knowhere

Signed-off-by: cqy <yaya645@126.com>

* clean unused variable warning in knowhere

Signed-off-by: cqy <yaya645@126.com>

* clean unused variable warning in knowhere

Signed-off-by: cqy <yaya645@126.com>

* clean unused variable warning in knowhere

Signed-off-by: cqy <yaya645@126.com>

* fix konwhere warning

Signed-off-by: cqy <yaya645@126.com>
2020-07-18 04:47:29 +00:00
Cai Yudong 771585ba0c
use BlockingQueue in JobMgr (#2885)
* using BlockingQueue in JobMgr

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-17 09:37:00 +08:00
cqy123456 52ab18fe0e
opt faiss build (#2883)
* opt faiss build

Signed-off-by: cqy <yaya645@126.com>
2020-07-17 09:34:34 +08:00
XuanYang-cn 7a39a00498
Fix Prometheus rebuild problem (#2878)
* Solve prometheus repeatedly build problem

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* recover build.sh

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* refactor build.sh

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* altering changelog

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rm not-useful comments

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-07-16 19:45:24 +08:00
Cai Yudong 1d9ccd9bb7
add MILVUS_SUPPORT_SPTAG for some code (#2866)
* add MILVUS_SUPPORT_SPTAG for some code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use knowhere::IndexEnum in db

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* keep EngineType order

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-15 17:42:44 +08:00
Cai Yudong d473c7ad2b
snapshot integrate (#2854)
* add test_segment

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update interface GetResPath

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update SSSegmentWriter

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-15 14:32:09 +08:00
Cai Yudong 4cb8383775
move index type try2 (#2860)
* move EngineType and MetricType to MetaTypes.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move IndexType

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-15 11:37:05 +08:00
op-hunter dc31742538
Hnsw sq8 support (#2835)
* modify for SQ8

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* add hnsw-sq8-support

Signed-off-by: cmli <chengming.li@zilliz.com>

* add unittest 4 hnsw_sq8nr and fix several bugs

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-14 09:45:08 +08:00
cqy123456 777c76c87a
Fix rnsg master (#2831)
* fix rnsg ip

Signed-off-by: cqy <yaya645@126.com>

* fix rnsg search in master

Signed-off-by: cqy <yaya645@126.com>
2020-07-14 09:30:26 +08:00
Xiaohai Xu fce41e738d
#2802 Add new index: IVFSQ8NR (#2805)
* support new index ivfsq8nr

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* remove timeout

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-07-11 13:40:14 +08:00
groot f94592ecdf fix #2752 (#2789)
* fix #2752

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
(cherry picked from commit d7c47a7905)
2020-07-11 11:18:05 +08:00
op-hunter 7bf3e6e1c3
Hnsw sq8 support (#2799)
* modify for SQ8

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* add hnsw-sq8-support

Signed-off-by: cmli <chengming.li@zilliz.com>

* make clang-format and update changelog

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* hide unittest error temporary

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix ut error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-10 13:42:24 +08:00
op-hunter b221cd6898 fix a bug of getting wrong nprobe limitation in knowhere on GPU version (#2788)
* fix a bug of getting wrong nprobe limitation in knowhere on GPU version

Signed-off-by: cmli <chengming.li@zilliz.com>

* update changelog and fix build error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-10 09:33:53 +08:00
shengjun.li a8bd2cf4a6 fix too many data copies (#2784)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-09 11:18:37 +08:00
groot aa549f0fba * #2642 Create index failed caused by server crashed.
Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #2692 Milvus hangs during multi-thread concurrent search

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-04 15:02:05 +08:00
groot 045089e5f3 * #2642 Create index failed caused by server crashed.
Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-04 15:00:52 +08:00
shengjun.li 0672347775
Create index failed caused by server crashed (#2736)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: groot <yhmo@zeronedata.com>
2020-07-04 01:25:48 +00:00
Xiaohai Xu 84c7701126
#2689 Construct Knowhere Index Without Data (#2704)
* Offset for IVFFlat

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* modify segment reader

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* add index_flat_nm

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix compilation issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix segment reader

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* Fix issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix hnsw_nm

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix nsg

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix bug

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* NSG

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix NSG issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* client test

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix index size

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* remove unnecessary things

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix clang format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix issues

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Change data to shared_ptr

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix hnsw and nsg ut

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* clang-format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* not to insert cache immediately

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* Fix macro issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-03 11:27:11 +00:00
BossZou f0abfa2c6c
Web fix 2690 (#2691)
* Remove body parser in show partitions endpoints (fix #2690)

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus output and comments

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus output and comments

Signed-off-by: yhz <413554850@qq.com>

* Remove space line

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus ut

Signed-off-by: yhz <413554850@qq.com>

* laten sleep time in case test_delete_vector_collection_count_no_flush

Signed-off-by: yhz <413554850@qq.com>
2020-07-02 15:34:37 +00:00
groot d7fff5084a
merge #2659 from 0.10.1 (#2666)
* merge #2659 from 0.10.1

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* merge #2657 from 0.10.1

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* format

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-29 02:07:47 +00:00
Cai Yudong 18998a4395
print out system memory size when report invalid cpu cache size (#2679)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-28 16:18:32 +08:00
shengjun.li 403753a15b
* remove annoy search parameter validation (#2667)
Signed-off-by: cmli <chengming.li@zilliz.com>

* fix test failed

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error again

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: op-hunter <ophunter52@gmail.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-24 16:52:05 +08:00
Cai Yudong 170209453f
snapshot gc (#2662)
* add Event.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add ResourceHelper and update event handler

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ResourceHelper

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add CollectionId for segment

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add CollectionId for SegmentFile

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ResourceHelper

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* re-order Resources

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ResourceHelper

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-24 01:46:43 +00:00
shengjun.li d318b527de Suit the range of HNSW parameters (#2638)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-22 16:49:45 +08:00
shengjun.li 3cdff86852 Fix HNSW and RNSG index files size (#2618)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-20 14:19:56 +08:00
Cai Yudong b3e8856a34
move all APIs in utils into namespace milvus (#2614)
* move APIs in util into namespace milvus

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* combine BlockingQueue.h and src/utils/BlockingQueue.inl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 15:53:36 +08:00
Cai Yudong eb5564af03
fix Milvus docker image report illegal instruction (#2598)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-18 10:57:40 +00:00
shengjun.li b6b56a0852
Support IVF_PQ on GPU with using metric_type IP (#2593)
* Support IVF_PQ on GPU with using metric_type IP

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* remove useless code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-18 08:56:34 +00:00
Cai Yudong 4ce1e6b2fc
optimize unittest build (#2589)
* optimize test_storage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_metrics

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optmize test_server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_schedule

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move test_rpc and test_web to db

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-17 17:22:54 +08:00
yukun e267bd2c4b
CreateHybridIndex.cpp compile error (#2583)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-17 14:03:13 +08:00
yukun df8e64ea42
Structured index support (#2573)
* Add ValidateBooleanQuery check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add test_hybrid_collection cases

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* CreatedStructuredIndex implementation

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add attr_type in DefaultAttrIndexFormat

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add CreateHybridIndex sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix LoadAttr

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Reconstruct CreateStructuredIndex code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix CreateStructuredIndex bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix merge problems

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* add changelog

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix gpu version compile bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix RpcHandlerTest.HYBRID_TEST bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_web bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove hybrid test cases

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-17 02:00:39 +00:00
Cai Yudong a9df83766a
clean util dependency (#2576)
* move part of APIs from utils/ValidationUtil to server/ValidationUtils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move part of APIs from utils/ValidationUtil to config/Utils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move part of APIs from utils/CommonUtil to config/Utils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move EraseFromCache from utils/CommonUtil to db/Utils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove SignalHandler dependency with server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update fiu

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-16 22:13:25 +08:00
Wang XiangYu 75c7a1d0fa fix cache.cache_size range check error (#2565)
* fix cache.cache_size range check error

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update ci

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-06-15 23:37:16 +08:00
BossZou 92a7b73ea9
(fix #2487) Remove timeout when creating collection in dev test (#2541)
Signed-off-by: Yhz <yinghao.zou@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-06-15 23:25:21 +08:00
Cai Yudong b26ce06463
remove secondary path (#2560)
* config code clean

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove slave_paths_ from db options

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update version

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable all db unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update some comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-15 23:23:10 +08:00
Cai Yudong 95d970cf62
count up query statistics for debug ease (#2528)
* clean duplicated code in SearchTask

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update Search interface and add SearchJob::time_stat()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-15 07:13:25 +00:00
BossZou d1c4ab9136 Mishards 010 upgrade & Add config unit in demo config (#2550)
* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Upgrade mishards for milvus v0.10.0 (fix #2548)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Launch server fail using demo config (#2549)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Remove surplus config yaml

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-06-15 14:00:47 +08:00
Wang XiangYu 9d9a453a5b Upgrade config (#2511)
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* config cluster

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config general

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config network

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config wal

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config cache

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config gpu and metrics

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config logs

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update server_config.template

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config with unit

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* disable restart case

Signed-off-by: zw <zw@milvus.io>

* fix gpu compile failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Fix lint and comments

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix wal unittest failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix some unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update config test cases

Signed-off-by: zw <zw@milvus.io>

* Fix config error

Signed-off-by: yhz <413554850@qq.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip-ci] fix test cases

Signed-off-by: zw <zw@milvus.io>

* fix config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* make -j3 on ci

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update python test

Signed-off-by: wxyu <xy.wang@zilliz.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update status message

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix set config bug

Signed-off-by: wxyu <xy.wang@zilliz.com>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: yhz <413554850@qq.com>
2020-06-15 14:00:09 +08:00
yukun 2ee3b60e76
Fix test_hybrid_db and test_rpc error (#2552)
* Fix test_hybrid_db and test_rpc error

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* CI retry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* CI retry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-14 23:18:44 +08:00
BossZou 9fda27ff93 Improve ut coverage (#2516) (#2517)
* Improve ut coverage (#2516)

Signed-off-by: yhz <413554850@qq.com>

* Improve ut coverage

Signed-off-by: yhz <413554850@qq.com>

* CI retry

Signed-off-by: yhz <413554850@qq.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:20:11 +08:00
Jin Hai f6928f7282 Add exception throw on mysql meta error (#2490)
* Add exception throw on mysql meta error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip ci] update changelog

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix Unit test

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:12:06 +08:00
Cai Yudong 2f57d360eb fix performance (#2499)
* optimize sq_get_distance_computer

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_ref

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx512

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize Codec

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx512.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-13 14:11:37 +08:00
Jin Hai 63ae619951 Fix 2495: Add more reason of creating lock file failed (#2496)
* Fix 2495: Add more reason of creating lock file failed

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix 2495: Update changelog

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compile error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:11:18 +08:00
BossZou 2decf951e2 read_only node detect delete vector operation using mishards #2368 (#2473)
* Add new request ReLoadSegments

Signed-off-by: yhz <413554850@qq.com>

* Finish load segments functionality

Signed-off-by: yhz <413554850@qq.com>

* Add api in grpc

Signed-off-by: yhz <413554850@qq.com>

* update Reloadsegments

Signed-off-by: yhz <413554850@qq.com>

* .

Signed-off-by: yhz <413554850@qq.com>

* create new blacklist if not exists

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update api names

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Finish mishard for support api reloadsegments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add more details when failed in search task

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Fix compile issue

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update mishards requirements

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* modify docker images in mysql all_in_one

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update shards code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Move updatedeldocs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* Move reload segment del docs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* [skip ci] correct shards requirements

Signed-off-by: yhz <413554850@qq.com>
2020-06-13 14:10:07 +08:00
groot f64e4e77f0 #2394 Drop collection timeout if too many partitions created on colle… (#2477)
* #2349 Drop collection timeout if too many partitions created on collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-12 11:18:09 +08:00
op-hunter b2b787bdb0 optimize the implemention of k-selection on faiss gpu version (#2467)
* optimize the implemention of k-selection on faiss gpu version

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log, re-classified the pr from feature to improvement

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-12 11:17:50 +08:00