Commit Graph

1325 Commits (cbba262442a42cc22ae24e00842d901b58321c97)

Author SHA1 Message Date
shengjun.li 3cdff86852 Fix HNSW and RNSG index files size (#2618)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-20 14:19:56 +08:00
Jin Hai f1a1ee8678
Snapshot improve (#2608)
* Format code style

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix bug && Format code style

* Format code style

* Format code style

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update merge config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-19 23:09:30 +08:00
Cai Yudong b3e8856a34
move all APIs in utils into namespace milvus (#2614)
* move APIs in util into namespace milvus

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* combine BlockingQueue.h and src/utils/BlockingQueue.inl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 15:53:36 +08:00
Cai Yudong 2bc9c0cf25
snapshot code opt (#2609)
* opt test_snapshot

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename BaseHolders to ResourceHolder

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename DBBaseResource to BaseResource

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove ResourceHolder.inl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 11:35:25 +08:00
XuPeng-SH ecc52d365c
(db/snapshot): fix ut errors (#2605)
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-19 10:37:41 +08:00
quicksilver 523ed18c6e
Update version of Clang tools to v7.0 in ubuntu18.04 build environment (#2610)
* Update clang tools version to v7.0 in ubuntu18.04 build environment

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update clang tools version to v7.0 in ubuntu18.04 build environment

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update clang tools version to v7.0 in ubuntu18.04 build environment

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-06-19 00:11:01 +08:00
Cai Yudong eb5564af03
fix Milvus docker image report illegal instruction (#2598)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-18 10:57:40 +00:00
shengjun.li b6b56a0852
Support IVF_PQ on GPU with using metric_type IP (#2593)
* Support IVF_PQ on GPU with using metric_type IP

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* remove useless code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-18 08:56:34 +00:00
XuPeng-SH a12ce5bf6b
(db/snapshot): update ut (#2588)
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-18 01:55:46 +00:00
groot 19c7d5f71b
fix #2578 (#2591)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-18 01:36:04 +00:00
Cai Yudong 4ce1e6b2fc
optimize unittest build (#2589)
* optimize test_storage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_metrics

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optmize test_server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_schedule

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move test_rpc and test_web to db

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-17 17:22:54 +08:00
yukun e267bd2c4b
CreateHybridIndex.cpp compile error (#2583)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-17 14:03:13 +08:00
groot c131fbe3ae
fix random crash of INSERT_DUPLICATE_ID case (#2557)
* fix random crash of INSERT_DUPLICATE_ID case

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* cahnge ver

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* sdk readme

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut failure

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-17 03:50:19 +00:00
yukun df8e64ea42
Structured index support (#2573)
* Add ValidateBooleanQuery check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add test_hybrid_collection cases

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* CreatedStructuredIndex implementation

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add attr_type in DefaultAttrIndexFormat

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add CreateHybridIndex sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix LoadAttr

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Reconstruct CreateStructuredIndex code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix CreateStructuredIndex bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix merge problems

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* add changelog

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix gpu version compile bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix RpcHandlerTest.HYBRID_TEST bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_web bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove hybrid test cases

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-17 02:00:39 +00:00
Cai Yudong a9df83766a
clean util dependency (#2576)
* move part of APIs from utils/ValidationUtil to server/ValidationUtils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move part of APIs from utils/ValidationUtil to config/Utils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move part of APIs from utils/CommonUtil to config/Utils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move EraseFromCache from utils/CommonUtil to db/Utils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove SignalHandler dependency with server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update fiu

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-16 22:13:25 +08:00
XuPeng-SH aa31b0c251
(db/snapshot): fix bugs and enhance unit test (#2562)
* (db/snapshot): add ToString for Snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bugs and enhance CompoundTest1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bug in operations

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): some code refactor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): small change

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-16 15:05:55 +08:00
Wang XiangYu 75c7a1d0fa fix cache.cache_size range check error (#2565)
* fix cache.cache_size range check error

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update ci

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-06-15 23:37:16 +08:00
Cai Yudong b26ce06463
remove secondary path (#2560)
* config code clean

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove slave_paths_ from db options

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update version

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable all db unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update some comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-15 23:23:10 +08:00
groot 7351b97850
change ver (#2559)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-15 17:59:40 +08:00
Cai Yudong 95d970cf62
count up query statistics for debug ease (#2528)
* clean duplicated code in SearchTask

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update Search interface and add SearchJob::time_stat()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-15 07:13:25 +00:00
BossZou d1c4ab9136 Mishards 010 upgrade & Add config unit in demo config (#2550)
* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Upgrade mishards for milvus v0.10.0 (fix #2548)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Launch server fail using demo config (#2549)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Remove surplus config yaml

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-06-15 14:00:47 +08:00
Wang XiangYu 18539c50a3 [skip ci]update comments in config file (#2542)
* update comments in config file

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-15 14:00:39 +08:00
Wang XiangYu 9d9a453a5b Upgrade config (#2511)
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* config cluster

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config general

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config network

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config wal

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config cache

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config gpu and metrics

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config logs

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update server_config.template

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config with unit

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* disable restart case

Signed-off-by: zw <zw@milvus.io>

* fix gpu compile failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Fix lint and comments

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix wal unittest failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix some unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update config test cases

Signed-off-by: zw <zw@milvus.io>

* Fix config error

Signed-off-by: yhz <413554850@qq.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip-ci] fix test cases

Signed-off-by: zw <zw@milvus.io>

* fix config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* make -j3 on ci

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update python test

Signed-off-by: wxyu <xy.wang@zilliz.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update status message

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix set config bug

Signed-off-by: wxyu <xy.wang@zilliz.com>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: yhz <413554850@qq.com>
2020-06-15 14:00:09 +08:00
yukun 2ee3b60e76
Fix test_hybrid_db and test_rpc error (#2552)
* Fix test_hybrid_db and test_rpc error

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* CI retry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* CI retry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-14 23:18:44 +08:00
XuPeng-SH 5727cd139e
(db/snapshot): Fix bugs and more unit test (#2538)
* (db/snapshot): refactor CompoundBaseOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): some refactor for CompoundOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): remove one of Operations constructor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add DropPartition API and related test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bugs for multi threads

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add some print

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): some bug fix

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): some bug fix 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bugs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add build worker for unittest

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

(db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

(db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

(db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bug in ReferenceProxy

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-13 15:46:17 +08:00
yukun c079804b81 Improve ut coverage (#2516) (#2522)
* Improve ut coverage

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Delete unused code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add fiu in HybridSearchRequest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change BinaryQuery validation check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 15:40:45 +08:00
BossZou 9fda27ff93 Improve ut coverage (#2516) (#2517)
* Improve ut coverage (#2516)

Signed-off-by: yhz <413554850@qq.com>

* Improve ut coverage

Signed-off-by: yhz <413554850@qq.com>

* CI retry

Signed-off-by: yhz <413554850@qq.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:20:11 +08:00
Jin Hai f6928f7282 Add exception throw on mysql meta error (#2490)
* Add exception throw on mysql meta error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip ci] update changelog

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix Unit test

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:12:06 +08:00
groot e77a428885 return hnsw name (#2501)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-13 14:11:50 +08:00
Cai Yudong 2f57d360eb fix performance (#2499)
* optimize sq_get_distance_computer

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_ref

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx512

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize Codec

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx512.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-13 14:11:37 +08:00
groot 5953f05015 default nlist 2048 (#2498)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-13 14:11:29 +08:00
Jin Hai 63ae619951 Fix 2495: Add more reason of creating lock file failed (#2496)
* Fix 2495: Add more reason of creating lock file failed

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix 2495: Update changelog

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compile error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:11:18 +08:00
BossZou 2decf951e2 read_only node detect delete vector operation using mishards #2368 (#2473)
* Add new request ReLoadSegments

Signed-off-by: yhz <413554850@qq.com>

* Finish load segments functionality

Signed-off-by: yhz <413554850@qq.com>

* Add api in grpc

Signed-off-by: yhz <413554850@qq.com>

* update Reloadsegments

Signed-off-by: yhz <413554850@qq.com>

* .

Signed-off-by: yhz <413554850@qq.com>

* create new blacklist if not exists

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update api names

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Finish mishard for support api reloadsegments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add more details when failed in search task

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Fix compile issue

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update mishards requirements

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* modify docker images in mysql all_in_one

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update shards code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Move updatedeldocs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* Move reload segment del docs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* [skip ci] correct shards requirements

Signed-off-by: yhz <413554850@qq.com>
2020-06-13 14:10:07 +08:00
shengjun.li 4f8c4cacda fix merge result (#2463)
* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix tests

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-12 11:18:18 +08:00
groot f64e4e77f0 #2394 Drop collection timeout if too many partitions created on colle… (#2477)
* #2349 Drop collection timeout if too many partitions created on collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-12 11:18:09 +08:00
op-hunter b2b787bdb0 optimize the implemention of k-selection on faiss gpu version (#2467)
* optimize the implemention of k-selection on faiss gpu version

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log, re-classified the pr from feature to improvement

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-12 11:17:50 +08:00
groot 08c919156c merge before index
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-11 17:03:35 +08:00
op-hunter 30d6f986bf fix bug of openblas library install fail (#2457)
* fix bug of openblas library install fail

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix openblas install cmakelist error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-11 17:03:25 +08:00
BossZou 6e8050996b Fix inconsistent reading and writing when using mishards (# 2367) (#2442)
* Fix inconsistent reading and writing when using mishards (# 2367)

Signed-off-by: yhz <413554850@qq.com>

* CI retry

Signed-off-by: yhz <413554850@qq.com>
2020-06-11 11:54:26 +08:00
shengjun.li 96c68edcb6 fix search by GPU (#2453)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-11 11:54:11 +08:00
Cai Yudong 8b627cacfd improve knowhere coverage (#2444)
* increase nb for NSG code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* test more APIs in test_annoy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable get_vector_by_id and search_by_id

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* improve code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* improve code coverage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* install test_instructionset

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-11 11:53:52 +08:00
groot e2b23e9cf2 performance issues (#2438)
* performance issues

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix bugs

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* preload collection issue

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* uncomment boring log

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* reduce unittest time

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* reduce metric test time cost

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-11 11:53:36 +08:00
Cai Yudong 0067883c30 add debug APIs and update GPU temp memory size (#2420)
* add debug APIs and update GPU temp memory size

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update thread name

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update DESIGN.md

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-10 18:34:54 +08:00
groot 0ea259c340 optimize merge process (#2419)
* optimize merge process

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* drop collecion issue

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-10 15:07:51 +08:00
shengjun.li 5eb5f6b0a0 midify Binary IVF in faiss 1.6.3 (#2416)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-10 15:07:43 +08:00
Xiaohai Xu 3b15160a92 #2395 Fix large nq cudaMalloc (#2409)
* fix large nq cuda malloc issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix clang format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Change log

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-06-10 15:07:36 +08:00
groot 166b83a098 Wal lsn issue (#2412)
* set wal lsn after all mem files serialized

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* return status

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-10 15:07:29 +08:00
groot 459ab0502d #2366 Reduce SQL execution times (#2383)
* #2366

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix python test

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-10 15:07:20 +08:00
groot 47f99f3fb5 mysql meta connection pool issue (#2413)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-10 15:06:28 +08:00
Wang XiangYu c007e446b9 logging build index progress (#2411)
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-06-10 15:06:06 +08:00
Cai Yudong d67157235f * remove log_config from code and script
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update script

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-10 15:05:35 +08:00
Jin Hai 078100926f #2403 MySQL max_idle_time is 10 by default (#2405)
* #2403 MySQL max_idle_time is 10 by default

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #2403 MySQL max_idle_time is 10 by default

Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-06-10 15:04:43 +08:00
shengjun.li 42806de462 fix MatchNlist (#2401)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-10 15:04:17 +08:00
Cai Yudong 2be4005d83 upgrade faiss 1.6.3 (#2400)
* roll back to original faiss 1.6.0

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update to faiss_1.6.3

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* patch all change to faiss 1.6.3

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* faiss CPU version build pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* faiss GPU version build pass

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-10 15:03:12 +08:00
groot 01cab6f365 #2378 (#2388)
* return partition lsn

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix wal lsn

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix wal issue

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* all collection include partition

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix build error

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix flush

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-10 15:01:01 +08:00
Cai Yudong 2f9873987a clean compile warning (#2380)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-10 14:55:45 +08:00
XuPeng-SH 295105ff40
Snapshot code update (#2497)
* (db/snapshot): Add LSN in Resource

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add AddStepWithLsn

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add Lsn related logic

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add operator << for Operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint errors

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add more test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): operation rollback api placeholder

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): remove hard code status code

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): support more printable dump for some operations

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add ToString for Context

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): more friendly context print

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): rename some apis

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-06 05:03:02 +00:00
XuPeng-SH 530ee76a76
(db/snapshot): Add new features for Snapshot Impl (#2483)
* (db/snapshot): Add new features for Snapshot Impl

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update changelog

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Small changes to remove some dummy code

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Small changes to remove some dummy code

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-03 03:55:40 +00:00
XuPeng-SH 9b2fd331fc
Add CreateCollection and DropCollection for Snapshot (#2454)
* (db/snapshot): add merge opration test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add UpdatedOnField

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add Utils.h

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor Snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor snapshot 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor snapshot 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor snapshot 4

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor snapshot 5

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor snapshot 6

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add CreateCollectionOperation and related test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): changes for code style and clang format

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): improve ut coverage 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): improve ut coverage 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): improve ut coverage 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add soft delete collection operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update drop collection 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update drop collection 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update drop collection 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update drop collection 4

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint error

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-01 06:36:56 +00:00
groot 47303d2551 QPS decrease (#2372)
* optimize merge strategy

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #2365

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* optimize search

Signed-off-by: groot <yihua.mo@zilliz.com>

* code format

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-05-29 20:25:42 +08:00
Cai Yudong 1b8a06daef * #2370 clean compile warning
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-29 20:24:52 +08:00
XuPeng-SH 2fe4cc10de
Snapshot code optimization 1 (#2440)
* (db/snapshot): add merge opration test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add UpdatedOnField

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add Utils.h

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor Snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor snapshot 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor snapshot 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor snapshot 4

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor snapshot 5

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor snapshot 6

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-05-28 02:25:14 +00:00
yukun 2264aab084
Add new hybrid search api (#2445)
* Add json-string-dsl hybrid search api

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add C++ sdk for json-string-dsl hybrid search

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add C++ examples for new hybrid search api

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add unittest for new hybrid search api

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-05-28 02:01:28 +00:00
Cai Yudong 8908cba476
optimize snapshot (#2435)
* optimize ResourceHolder

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize OperationExecutor

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* roll back ResourceHolders

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* roll back BaseHolders

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-27 05:37:21 +00:00
op-hunter 098f5a534d
Structured index (#2418)
* Support Structured Index Based on Sort

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint errors and make clang-format

Signed-off-by: cmli <chengming.li@zilliz.com>

* pick some advices from laojin

Signed-off-by: cmli <chengming.li@zilliz.com>

* make clang-format

Signed-off-by: cmli <chengming.li@zilliz.com>

* rename n_ to size_, replace self-implemented lower_bound and upper_bound by std implemention as laojin requires

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* update implementation by shengjun's advice

Signed-off-by: cmli <chengming.li@zilliz.com>

* remove index member size_

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-05-26 14:41:21 +08:00
XuPeng-SH 96166e1a5b
Metadata Snapshot Implementation Phase I (#2414)
* (db/snapshot): migrate code from milvus_metatdata_design repo

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add license info

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add namespace snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add test_snapshot unit test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add ScopedResourceTest

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add ResourceHodlersTest

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update ResourceHodlersTest

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add Operation Test and fix some bug

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Update Operation Test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Update Operation Test to test build

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add Reset API for static instance

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix snapshot bug

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix dead lock bug

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* fix bug in concurrent mode

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* fix lint error in ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* fix lint error in snapshot 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* fix lint error in snapshot 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* fix lint error in snapshot 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* fix lint error in snapshot 4

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* fix lint error in snapshot 5

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-05-25 10:50:34 +08:00
yukun 110d4d066a
TargetEntry implementation (#2391)
* Add GetEntitiesByID in DBImpl

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add operator overload functions in ConcurrentBitset

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add GetEntityByID interface

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change format of Attributes

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change hybrid search for new rules

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix C++ sdk for new format of hybrid interfaces

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix compile bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix GetEntityByID

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix unittest bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix WebControllerTest:test_hybrid bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix field names bug in HybridSearch

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix HYBRID_SEARCH_TEST caused by const auto&

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add ConvertRowToColumnJson in WebRequestHandler

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Return target entry in WebServer

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change ValidateBinaryQuery

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add GetEntityByID in WebServer

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* [skip ci]Removed unused code in C++ sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-05-23 14:25:24 +08:00
Cai Yudong c902b0fdb5 Caiyd 090 rename config (#2340)
* rename config items

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update config unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update config unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update log rotate default value

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update parameter usage in LogUtil.cpp (#3)

Signed-off-by: wxyu <xy.wang@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix ut

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: Wang XiangYu <scsven@qq.com>
2020-05-15 20:54:34 +08:00
BossZou f6c03f47dc [skip ci] Update http document for v0.9.0 (#2347) (#2348)
Signed-off-by: yhz <413554850@qq.com>
2020-05-15 20:52:49 +08:00
BossZou 3561014f5b Change url for behavior 'get_entities_by_id' (#2330) (#2336)
Signed-off-by: yhz <413554850@qq.com>
2020-05-15 20:52:15 +08:00
groot d5f7526cd5 #2338 (#2345)
* fix #2264

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* Remove unnecessary memcpy

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* remove SearchByID from C++SDK

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #2338

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-15 20:51:49 +08:00
groot 299cd9cd94 Remove unnecessary memcpy (#2337)
* fix #2264

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* Remove unnecessary memcpy

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-15 20:51:24 +08:00
Cai Yudong b2967574ef
enable config unittest (#2329)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-05-14 10:12:19 +08:00
groot 5e777450b3 fix #2264 (#2323)
* fix #2264

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-13 21:01:50 +08:00
Cai Yudong 5293607359
Caiyd support disable sptag (#2308)
* remove KNOWHERE_GPU_VERSION

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add MILVUS_SUPPORT_SPTAG

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix SPTAG build issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-13 16:52:48 +08:00
BossZou 1b7aeaffdf
Fix server start failed when mode is 'read_only' (#2294)
* fix deploy bug

Signed-off-by: yhz <413554850@qq.com>

* fix unittest fail

Signed-off-by: yhz <413554850@qq.com>

* remove unused yml

Signed-off-by: yhz <413554850@qq.com>

* change to readonly

Signed-off-by: yhz <413554850@qq.com>
2020-05-12 09:28:10 +08:00
groot fb49e580c5
rewrite HasPartition (#2289)
* rewrite HasPartition

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* miss file

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* use layered merge

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* sdk HasPartition

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* rename sdk api

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-05-12 09:27:58 +08:00
shengjun.li bc74d16376
#2283 Suspend the building tasks when any query command arrives (#2285)
* add builder suspend

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* IndexIVF builder check wait

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Build suspend for all IVF

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* HNSW suspend

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* HNSW suspend

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* HNSW suspend

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* HNSW suspend

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* HNSW suspend

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* cpubuilder

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add suspend check during annoy build index

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix hnsw

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix changelog

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix clang format

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* suspend nsg

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: sahuang <xiaohai.xu@zilliz.com>
Co-authored-by: wxyu <xy.wang@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-05-12 09:27:45 +08:00
op-hunter b6e16af8f8
update unittest config version (#2299)
Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-05-11 22:00:21 +08:00
groot f32cba6111
Merge strategy (#2277)
* merge manager

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix build error

Signed-off-by: groot <yihua.mo@zilliz.com>

* layered merge strategy

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* use simple

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix parallel multi collections query crashissue

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-05-11 01:38:50 +00:00
Wang XiangYu 369743c10d
Config upgrade (#2281)
* upgrade config to 0.4

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update milvus version in cmakelists.txt

Signed-off-by: wxyu <xy.wang@zilliz.com>

* disable make VERBOSE

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-05-10 22:55:46 +08:00
BossZou 8d732656a7
Fix show partitions failed in http module (fix #2228) (#2234)
* Fix show partitions failed in http module (fix #2228)

Signed-off-by: yhz <413554850@qq.com>

* catch json string convert error

Signed-off-by: yhz <413554850@qq.com>
2020-05-10 12:49:26 +00:00
Cai Yudong e63904867b
update debug log (#2275)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-10 00:19:33 +08:00
op-hunter a37f9b0764
Openblas macro (#2269)
* save local work 4 add openblas macro to chose compile openblas lib or not

Signed-off-by: cmli <chengming.li@zilliz.com>

* compile openblas by auto detect

Signed-off-by: cmli <chengming.li@zilliz.com>

* Update build environment docker images

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* update changelog

Signed-off-by: cmli <chengming.li@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update CMakeLists.txt

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* set OpenBLAS_SOURCE auto in build.sh

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-05-09 22:47:03 +08:00
groot aeb9edf5c6
#2261 (#2270)
* has collection bug

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #2261

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix typo

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-05-09 18:32:20 +08:00
Wang XiangYu 2c04cc0afe
upgrade config to 0.4 (#2273)
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-05-09 18:28:03 +08:00
Cai Yudong cf6be092ab
code clean (#2265)
* use macro STATUS_CHECK

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* comment S3 related config APIs

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-09 01:51:04 +00:00
shengjun.li 4cea320943
k-means L2 (#2258)
* k-means L2

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix change log

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-05-08 14:03:50 +00:00
groot cf68c9918e
#1925 (#2257)
* write error

Signed-off-by: groot <yihua.mo@zilliz.com>

* out of storage

Signed-off-by: groot <yihua.mo@zilliz.com>

* clang format

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix #1955

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

* compact threashold

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* search by id for hnsw/pq/annoy

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* file cleanup timeout config

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #1925\

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-08 11:40:41 +00:00
BossZou 1811254de6
Obtain running rpc requests information (#2240) (#2241)
* add cmd to get requests

Signed-off-by: yhz <413554850@qq.com>

* forward class declaration

Signed-off-by: yhz <413554850@qq.com>

* Obtain running rpc requests information (fix #2240)

Signed-off-by: yhz <413554850@qq.com>

* Fix unittest compile failed issue

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Log error with msg

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-05-08 11:17:48 +00:00
caosiyang 7b0998885c
Fix #2248: use hostname and port as instance label of metrics (#2249)
Signed-off-by: caosiyang <csy3228@gmail.com>
2020-05-08 19:08:04 +08:00
Wang XiangYu 104cda1a76
print warning on stdout (#2243)
* print warning on stdout

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix compile failed

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-05-08 08:57:08 +00:00
groot da55622042
file cleanup timeout config (#2250)
* write error

Signed-off-by: groot <yihua.mo@zilliz.com>

* out of storage

Signed-off-by: groot <yihua.mo@zilliz.com>

* clang format

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix #1955

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

* compact threashold

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* search by id for hnsw/pq/annoy

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* file cleanup timeout config

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-08 06:24:29 +00:00
Wang XiangYu ac69f1ab8d
add_dependencies in cmake (#2239)
* add_dependencies in cmake

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-05-07 14:51:38 +00:00
Wang XiangYu 7e3ba4ddd2
Catch exception and print in stderr (#2238)
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-05-07 10:04:35 +00:00
yukun 1516d71a55
Add attributes to mysql and wal (#2229)
* Add hybrid operation into mysql

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add Hybrid into wal

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add unittest for hybrid wal

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix wal unittest bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix hybrid_insert unittest bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix codacy/PR quality

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change test_rpc port to 19531

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-05-07 06:14:58 +00:00
zhoubo0317 6de141b2af
rm libopenblsa-dev (#2235)
Signed-off-by: zhoubo <zhoubo@hotmail.com>

Co-authored-by: zhoubo <zhoubo@hotmail.com>
2020-05-06 21:46:33 +08:00
dyhyfu 882ed76cda
[skip ci] remove 'openblas-devel' (#2208)
Signed-of-by: yaohong<dyhyfu@126.com>
2020-05-06 21:39:14 +08:00
Cai Yudong f218217064
optimize similarity template (#2227)
* optimize similarity template

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-06 08:53:16 +00:00
groot ac8e9ff020
write error (#2184)
* write error

Signed-off-by: groot <yihua.mo@zilliz.com>

* out of storage

Signed-off-by: groot <yihua.mo@zilliz.com>

* clang format

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix #1955

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

* compact threashold

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* search by id for hnsw/pq/annoy

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix python test

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-06 16:52:37 +08:00
BossZou 82ab21aec3
Http update endpoints (#2187)
* replace table with collection in http module

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add search_by_ids in http module

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add API

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add new API and unittest

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update endpoint in http module(#2186); change vector id to string format(2185)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* allow filter when show partition

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* move constant variables defination to .cpp

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-05-06 14:07:35 +08:00
jackyu2020 86510aa295
check the return value of fscanf (#2218)
Signed-off-by: e <ych05zh@163.com>
2020-05-03 01:49:57 +00:00
Cai Yudong ed2ee8873b
fix SingleIndexTest.IVFSQHybrid unittest (#2215)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-02 10:24:35 +00:00
Wang XiangYu ff48427911
Log file rotating (#2207)
* log file rotating

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix compile failed

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-30 23:31:14 +08:00
Cai Yudong 0032c13445
Error out when index SQ8H run in CPU mode (#2204)
* Error out when index SQ8H run in CPU mode

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update cmake

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-30 15:22:29 +00:00
BossZou b458034ad5
Fix Fail to access WAL storage path (#2196) (#2198)
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-30 09:47:12 +00:00
zxf2017 14b3e95960
add head file to fix two build warnings (#2195)
* add head file to fix two build warnings

Signed-off-by: zxf <zhuxiaofeng07936@hellobike.com>

* fstream.h => fstream

Signed-off-by: zxf <zhuxiaofeng07936@hellobike.com>

Co-authored-by: zxf <zhuxiaofeng07936@hellobike.com>
2020-04-30 06:10:44 +00:00
Wang XiangYu be2feb1531
Log with request id (#2192)
* add function to get request id

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update log in GrpcRequestHandler

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update request id format

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-30 13:39:55 +08:00
feisiyicl e77fd12286
Feisiyicl 2190 memory usage (#2191)
* rename MILVUS_ENABLE_PROFILING to ENABLE_CPU_PROFILING, and add ENABLE_MEM_PROFILING

Signed-off-by: feisiyicl <7764126@qq.com>

* optimize ReadOnlyArrayInvertedLists

Signed-off-by: feisiyicl <7764126@qq.com>

* update changelog

Signed-off-by: feisiyicl <7764126@qq.com>

* retry ci

Signed-off-by: feisiyicl <7764126@qq.com>

* fix build issue

Signed-off-by: feisiyicl <7764126@qq.com>
2020-04-30 02:03:54 +00:00
BossZou 2c889636ab
Check Storage permission (#2174)
* add class storage checker

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add cpu instruction sets check (#2055)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add more details for storage

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Check storage path permission (fix #2173)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Fix logs path access fail issue

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-29 17:23:32 +00:00
feisiyicl 9b2368016c
#1946 fix load index file CPU2GPU fail during searching (#2182)
Signed-off-by: feisiyicl <7764126@qq.com>
2020-04-29 20:01:52 +08:00
shengjun.li a76e705a92
Using elkan K-Means to improve IVF (#2180)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-04-29 19:58:50 +08:00
feisiyicl b3aa06fce1
add config simd_type (#2179)
* add config simd_type

Signed-off-by: feisiyicl <7764126@qq.com>

* update changelog

Signed-off-by: feisiyicl <7764126@qq.com>
2020-04-29 17:46:36 +08:00
BossZou 0a2b77c363
Upgrade http thirdparty oatpp to v1.0.0 (fix #2153) (#2157)
* Upgrade thirdparty oatpp to v1.0.0 (fix #2153)

Signed-off-by: yhz <413554850@qq.com>

* adjust header order

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-28 15:57:51 +00:00
groot ebb01aa9b2
Fix failed to open file (#2138)
* file reference

Signed-off-by: groot <yihua.mo@zilliz.com>

* print info

Signed-off-by: groot <yihua.mo@zilliz.com>

* avoid metric crash

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* apply delete bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* has partition check

Signed-off-by: groot <yihua.mo@zilliz.com>

* duplicate id search

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-28 21:10:01 +08:00
Wang XiangYu 3be8aad1af
Merge config file (#2168)
* Using el::Configurations Class init easylog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add logs config constant

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add config check function

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add config get function

Signed-off-by: wxyu <xy.wang@zilliz.com>

* logs config set function

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update InitLog function

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update server_config.template

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-28 20:21:21 +08:00
BossZou 4ab92f3032
Add config to switch off web server (#2137)
* Add a config parameter to switch off http server (fix #2057)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* modify web enable config name

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix cmake bug & output exanption msg

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* rename web enbale config as web_enable

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* optimize code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* log out errno

Signed-off-by: yhz <413554850@qq.com>
2020-04-28 17:04:04 +08:00
op-hunter 1b25bacee2
build openblas by source (#2154)
* build openblas by source

Signed-off-by: cmli <chengming.li@zilliz.com>

* remove openblas md5

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-27 22:48:14 +08:00
Wang XiangYu b65f5072f3
fix server start failed if wal directory exist (#2144)
* fix server start failed if wal directory exist

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add wal_enable check

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix compile error

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-27 20:05:04 +08:00
BossZou bf42dbd5b6
Gpu check (#2129)
* add GpuCheck class

Signed-off-by: yhz <413554850@qq.com>

* Check gpu environment

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* add server checker unittest

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-27 16:56:12 +08:00
Wang XiangYu 1f0466708f
merge server_cpu_config.template and server_gpu_config.template (#2150)
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-27 16:27:44 +08:00
Jin Hai b6d681f458
Merge pull request #2139 from longer-is-better/patch-1
delete unused file
2020-04-27 16:17:15 +08:00
AzAz 2a7e116a67
remove uint64_t j from parameters (#2147)
Signed-off-by: talentAN <adam_an02@163.com>
2020-04-27 16:15:49 +08:00
AzAz b3028baed8
Add log for error status (#2148)
Signed-off-by: talentAN <adam_an02@163.com>
2020-04-27 16:13:11 +08:00
Cai Yudong 1384484b7d
Caiyd 2039 avx sse (#2136)
* #2039 clean redundant compile options

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #2039 add distances_avx.h and distances_avx.cpp

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #2039 add ScalarQuantizerDC_avx.cpp, ScalarQuantizerDC_avx.h and ScalarQuantizerCodec_avx.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #2039 remove -mavx2 from default compile option

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #2039 clean code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #2039 clean code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-27 15:01:47 +08:00
Gracieeea 509b87f103
replace int with size_t (#2140)
Signed-off-by: Gracieeea <sly_freesoul@icloud.com>
2020-04-27 04:52:11 +00:00
JinHai-CN 7b1b75a90d Fix lint error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-27 10:26:05 +08:00
JinHai-CN adc6dc38bf Fix compiling error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-27 10:25:07 +08:00
dongwei 2c6fcacdbd delete unused file
Signed-off-by: dongwei <d279617552@gmail.com>
2020-04-26 23:21:06 +08:00
Cai Yudong 150cca6bbf
Caiyd 1946 cpu2gpu getres (#2124)
* #1946 use new log format in knowhere

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1946 add more debug info for GPU device initialize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-26 18:29:21 +08:00
Wang XiangYu 12b3bf0693
Add instance lock (#2060)
* Add instance lock

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update message

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update unittest CMakeLists.txt

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-26 08:31:18 +00:00
DanielHuang1983 7ecafbd93a
rename the variable name collection_schema (#2126)
Signed-off-by: DanielHuang1983 <huanghe888@hotmail.com>
2020-04-26 08:03:52 +00:00
shana0325 53133aafac
train func add catch excetpion (#2096)
* train func add catch excetpion

Signed-off-by: shana0325 <645842705@qq.com>

* Fix lint error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-26 11:29:20 +08:00
chen lu d22f3c2c5f
[skip ci] remove useless log (#2119)
Signed-off-by: yiuluchen 1093633856@qq.com
2020-04-26 10:19:31 +08:00
BossZou 8edb7b6a09
add cpu instruction sets check (#2055)
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-26 00:59:25 +08:00
groot 777c36a0ac
#1997 index file missed after compact (#2108)
* #1997 index file missed after compact

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-25 22:06:26 +08:00
JinHai-CN 9db19a9a20 Merge remote-tracking branch 'main/master' 2020-04-25 18:58:59 +08:00
GuoRentong fed865c7c9
Values of type 'const size_t' may not fit into the receiver type 'int' (#2109)
Signed-off-by: GuoRentong <rt.guo@outlook.com>

Co-authored-by: guo rentong <grt@guodeMacBook-Pro.local>
2020-04-25 18:35:04 +08:00
yxm1536 4c108c7578
Rename MemMenagerFactory.cpp/h to MemManagerFactory.cpp/h (#2093)
* Rename MemMenagerFactory.cpp to MemManagerFactory.cpp

Signed-off-by: yxm1536 <62009483+yxm1536@users.noreply.github.com>

* Rename MemMenagerFactory.h to MemManagerFactory.h

Signed-off-by: yxm1536 <62009483+yxm1536@users.noreply.github.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-25 16:57:14 +08:00
czp b454f462c1
put error when call Flush (#2097)
Signed-off-by: beautifulpython <2426009680@qq.com>
2020-04-25 16:13:32 +08:00
grtoverflow 6fe2cf1580
Update ConfAdapter.cpp (#2106)
Signed-off-by: grtoverflow <grtoverflow@gmail.com>
2020-04-25 16:12:58 +08:00
xge-zilliz 42b1983625
Delete useless boolean (#2107)
Signed-off-by: xge-zilliz <xi.ge@zilliz.com>
2020-04-25 15:49:43 +08:00
dd-He c33df2ee20
Use emplace_back instead of push_back (#2103)
Signed-off-by: dd-He <1843757857@qq.com>
2020-04-25 15:37:23 +08:00
chengpu 6d2c50ac21
Use std::copy to replace forloop assignment (#2087)
Signed-off-by: chengpu <chengpu@live.com>
2020-04-25 15:28:31 +08:00
binbin12580 2247d58cc4
Unused variable: ids (#2088)
Signed-off-by:  binbin12580 <1565196967@qq.com>
2020-04-25 15:26:16 +08:00
Cross 7deef8029f
Update GrpcRequestHandler.cpp (#2085)
Signed-off-by: Cross <xinyuanm9@gmail.com>
2020-04-25 14:53:49 +08:00
groot 82f4c289eb
New api HasPartition, SearchByID, GetVectorsByID (#2056)
* modify proto

Signed-off-by: groot <yihua.mo@zilliz.com>

* modify c sdk

Signed-off-by: groot <yihua.mo@zilliz.com>

* rewrite search by id

Signed-off-by: groot <yihua.mo@zilliz.com>

* change cases

Signed-off-by: zw <zw@zilliz.com>

* fix cases

Signed-off-by: zw <zw@zilliz.com>

* fix wal case

Signed-off-by: zw <zw@zilliz.com>

* fix search by id

Signed-off-by: groot <yihua.mo@zilliz.com>

* modify sdk

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine code

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: groot <yihua.mo@zilliz.com>

* add search_by_ids case

Signed-off-by: zw <zw@zilliz.com>

* disable test_search)by_ids

Signed-off-by: zw <zw@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
2020-04-25 14:49:32 +08:00
lwglgy 7e52815fb6
DBImpl.cpp::ExecWalRecord need to process error status in case wal::MXLogType::Entity (#2083)
Signed-off-by: lwglgy <1486111482@qq.com>
2020-04-25 13:24:24 +08:00
yukun 67d8a9b54c
Add http interface for hybrid search (#2079)
* Add http interface for hybrid search

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add unittest for http hybrid interface

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* clang format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix codacy quality

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-04-25 13:19:17 +08:00
Jin Hai 6d87dc3c72
Merge pull request #2084 from lee-eve/master
Unused variable: offsets
2020-04-25 13:18:19 +08:00
water32 420d70dcc5
unused variable, just remove it. (#2082)
Signed-off-by:  water32 <miaochen@mail.ustc.edu.cn>

unused variable, just remove it.
2020-04-25 11:58:23 +08:00
lieve 6fea6a17de Unused variable: offsets
Signed-off-by: lieve <645417202@qq.com>
Signed-off-by: lizhengxie <zhengxie.li@zuolin.com>
2020-04-25 11:48:27 +08:00
JinHai-CN fb85d8b3f0 Fix lint error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-25 11:21:58 +08:00
PJZero fedbb697e3
Update GrpcRequestHandler.cpp (#2069)
Signed-off-by: pengjian05 <pengjian05@baidu.com>

Co-authored-by: pengjian05 <pengjian05@baidu.com>
2020-04-25 11:19:30 +08:00
Jin Hai 54c9386bdc
#2076 CheckMetricConfigAddress error message (#2077)
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-25 10:04:04 +08:00
Jin Hai 723b10600b
#2073 Fix CheckDBConfigBackendUrl error message (#2074)
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-04-25 09:31:25 +08:00
Wang XiangYu c0d5a47eba
rename optimizer to selector (#2067)
* rename

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-24 11:42:17 +00:00
格雷大大 bf4cea1d1f
add log 4 annoy if index has been built when BuildAll interface invoked (#2071)
* add log 4 annoy if index has been built when BuildAll interface invoked

Signed-off-by: grace <yan.gu@zilliz.com>

* fix lint error

Signed-off-by: grace <yan.gu@zilliz.com>
2020-04-24 19:38:21 +08:00
Wang XiangYu 67819be29b
warn when use sqlite as metadata management (#2065)
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-24 15:23:46 +08:00
Yhz 87778052f4 move http parser to http utils
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-23 23:16:04 +08:00
JinHai-CN a41949f075 Merge remote-tracking branch 'main/master' into AllenYu1987-patch-1 2020-04-23 14:08:28 +08:00
Jin Hai 9a3ae916a2
Merge pull request #2044 from Tlincy/Tlincy-patch-1
Update LogUtils.cpp: use switch ... case to replace if ... else
2020-04-23 14:03:54 +08:00
Jin Hai 3267ed406f
Merge pull request #2032 from ggaaooppeenngg/code-clean
Delete CommandDto class and remove file CmdDto.hpp
2020-04-23 14:01:48 +08:00
Allen Zhang f741be0a79 Move http query parser
Signed-off-by: Allen Zhang <allen.zhangyilun@gmail.com>
2020-04-23 12:14:11 +08:00
loguo fc969f73ba
Update DBImpl.cpp: rename flush_tables to flushed_collections (#2043)
Signed-off-by: loguo <xiangzhouguo@gmail.com>
2020-04-23 11:27:29 +08:00
Aredcap b55f249938
Assert cmd reply (#2030)
* Assert cmd reply

Signed-off-by: Aredcap <zhangleilei19@mails.ucas.ac.cn>

* fix unittest error

Signed-off-by: Aredcap <zhangleilei19@mails.ucas.ac.cn>
2020-04-23 11:22:58 +08:00
JinHai-CN 58d0fa56de Fix error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-23 11:20:30 +08:00
moe-of-faith b23beebbfd
Change LogOut("[%s][%d]" ..... ) to LogOut("[%s][%ld]" ......) (#2041)
Signed-off-by: moe-of-faith <aeon.sym@gmail.com>
2020-04-23 11:03:10 +08:00
Gxz ce44da04e1
unused var: s (#2042)
Signed-off-by: guoxiangzhou <8188755@qq.com>
2020-04-23 11:02:39 +08:00
JinHai-CN 7d63d15718 Fix lint
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-23 10:53:33 +08:00
JinHai-CN 23ad133c14 Fix lint
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-23 10:45:34 +08:00
AllenYu1987 f4629b242e
Update GrpcRequestHandler.cpp: Use std::copy to replace forloop assignment
Signed-off-by: AllenYu1987 <myextremelove@hotmail.com>
2020-04-23 10:42:54 +08:00
phantom ea5c9f0178
Update core/src/scheduler/JobMgr.cpp: format include (#2046)
Signed-off-by: qingyigudu <sore_loner@sohu.com>
2020-04-23 10:25:46 +08:00
Tlincy 92508bcc0a
Update LogUtils.cpp: use switch ... case to replace if ... else
Signed-off-by: Tlincy <540549770@qq.com>
2020-04-23 09:35:51 +08:00
Yoya 44e8e02065
remove unused function "preprocessor" (#2034)
Signed-off-by: Yoya <taydycn@gmail.com>
2020-04-22 20:16:55 +08:00
ryjiang a3e4d29af6
Remove unused function (#2029)
* [skip ci] Remove unused function

Signed-off-by:  shanghaikid <jiangruiyi@gmail.com>
Signed-off-by: ruiyi.jiang <ruiyi.jiang@zilliz.com>

* remove unused function in utils.cpp

Signed-off-by: ruiyi.jiang <ruiyi.jiang@zilliz.com>
2020-04-22 20:15:09 +08:00
Fierralin 72cae5221c
Move NULL to nullptr (#2027)
* Move NULL to nullptr

Signed-off-by: Fierralin <fierralin@hotmail.com>

* Try fix

Signed-off-by: Fierralin <fierralin@hotmail.com>
2020-04-22 18:39:04 +08:00
Ben 9bfd48cdf5
move the definiation to cpp file (#2033)
Signed-off-by: Ben <sckim2008@gmail.com>
2020-04-22 15:16:37 +08:00
yukun ff9697b184
Change input format for attributes (#1990)
* Change input of attributes

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix unittest for new attrs format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix float type for new format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Use new HEntity

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add sdk_hybrid

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add metric_type assign in CreateHybridCollection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix for clang-format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Read engine_type from extra_params

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove todo annotation

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove some comments

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add DescribeHybridCollection request

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove unused code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-04-22 06:12:46 +00:00
HesterG ba191b263a
rename table_schema_array (#2024)
Signed-off-by: HesterG <hestergong@gmail.com>
2020-04-22 12:01:01 +08:00
Jin Hai aca299166c
Merge pull request #2026 from mileyzjq/code_polish
Fix unused code and clarify params
2020-04-22 11:59:29 +08:00
Peng Gao f7351e7482 Delete CommandDto class and remove file CmdDto.hpp
Signed-off-by: Peng Gao <peng.gao.dut@gmail.com>
2020-04-22 11:46:54 +08:00
sahuang 9e78cde3f2 Fix
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-04-22 10:47:12 +08:00
Lin-gh-Saint 04db47fa66
fix warning: no return statement (#2025)
Signed-off-by: Lin-gh-Saint <linguanhaohyq@gmail.com>
2020-04-22 10:03:11 +08:00
HesterG 68f7a0fa77
rename table_schema to collection_schema (#2023)
Signed-off-by: HesterG <hestergong@gmail.com>
2020-04-22 10:02:19 +08:00
Accagain c7eb9b0fa2
add safty check 4 annoy load index (#2022)
Signed-off-by: cms <chenmaosen0@gmail.com>
2020-04-22 10:00:46 +08:00
nezha2017 00e76219f6
use #include "segment/SegmentWriter.h" instead (#2020)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2020-04-22 09:46:33 +08:00
mileyzjq 72d80aca74 fix unused code and clarify params
Signed-off-by: mileyzjq <mileyzha@gmail.com>
2020-04-21 22:08:50 +08:00
shiyu09 0320259976
rename table_info into collection_info (#2016)
Signed-off-by: shiyu09 <39143280+shiyu09@users.noreply.github.com>
2020-04-21 20:30:16 +08:00
adream307 a3d99de5de
Context-head-file (#2021)
* Context-head-file

Signed-off-by: adream307 <adream307@163.com>

* clang-format

Signed-off-by: adream307 <adream307@163.com>

Co-authored-by: adream307 <adream307@163.com>
2020-04-21 19:04:06 +08:00
nameczz 5bcd3034c6
remove unused function "normalize_vector" (#2015)
Signed-off-by: nameczz <nameczz@126.com>
2020-04-21 15:53:17 +08:00
bo huang 574ff66c01
add safty check 4 annoy load index (#2009)
Signed-off-by: bo-huang <1013767522@qq.com>
2020-04-21 14:23:42 +08:00
Cai Yudong f039032c8c
Caiyd 1965 unify metric cal (#2008)
* #1965 add metric algorithm benchmark for FAISS

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for NSG

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for ANNOY

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 add metric algorithm benchmark for HNSW

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* calculate average time

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 annoy/nsg/hnsw all use faiss distance algorithm

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 support AVX512/AVX2/SSE42

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* #1965 fix hnsw ip calculation error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-21 14:22:33 +08:00
groot a5eec9d7b4
create index notify (#2003)
* create index notify

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-21 14:22:18 +08:00
xudalin0609 9e30c468ff
remove unused var (#2006)
Signed-off-by: xudl <isxudl@163.com>

Co-authored-by: xudl <isxudl@163.com>
2020-04-21 12:13:23 +08:00
Morry Niu 75ca0d96b6
keep bitset naming consistent (#2004)
Signed-off-by: Xinyao Niu <morryniu123@gmail.com>
2020-04-21 09:40:49 +08:00
JUN a2e7a91964
Modify data type in CommonUtil.cpp (#1996)
Signed-off-by: JUN <gujun720@foxmail.com>
2020-04-20 20:42:16 +08:00
Jin Hai 315dac3338
Automerge1 (#1984)
* [skip ci] remove auto-merge git-action

Signed-off-by: jinhai <hai.jin@zilliz.com>

* [skip ci] update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Stop strip the binary

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix warning

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-20 20:39:22 +08:00
Yaning Zhong 814b8f0c53
move const char* to .cpp file (#1994)
Signed-off-by: ynzhong <zhongyn522@gmail.com>

Co-authored-by: ynzhong <zhongyn522@gmail.com>
2020-04-20 15:41:38 +08:00
Tracy199 c50de3de9a
use TimeRecorder instead of chrono (#1989)
Signed-off-by: Tracy199 <houkx@vip.sina.com>
2020-04-20 01:29:11 +00:00
HuangHua 008f24a7c2
replace metaOptions with meta_options (#1979)
* replace metaOptions with meta_options

Signed-off-by: huanghua <bruce.huang@hotmail.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: groot <yihua.mo@zilliz.com>
2020-04-19 08:00:50 +00:00
smiling849 50ddc49332
fix hnsw code format (#1987)
Signed-off-by: Zhonglei Song <szl_84@sina.com>
2020-04-19 12:00:45 +08:00
bigbraver e8f759c22b
code optimize (#1986)
Signed-off-by: bigbraver <bigbraver@qq.com>
2020-04-19 03:54:23 +00:00
Hanyang Ke 25b6efc8e8
fix warning (#1988)
Signed-off-by: kehycs <kehycs@foxmail.com>
2020-04-19 11:12:01 +08:00
anchun 29acf6a594
update some variables in validation utils. (#1978)
Signed-off-by: anchun <nkterry@163.com>
2020-04-18 21:59:26 +08:00
zaqwsxqwer111 54fb8ef357
remove pragma once from .cpp files (#1982)
Signed-off-by: zaqwsxqwer111 <2536173042@qq.com>
2020-04-18 20:28:10 +08:00
zhangchen b02a37d79d not to printf the log when record type is None
Signed-off-by: zhangchen <zhangch5n@163.com>
2020-04-18 17:29:12 +08:00
Rayn Wei 375fd709a9
. (#1972)
Signed-off-by: RyanWei <rainmanwy@gmail.com>
2020-04-18 13:22:47 +08:00
groot 85de706d22
#1929 Skip mysql meta schema field width check (#1937)
* #1929 Skip mysql meta schema field width check

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* merge master

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-18 10:26:22 +08:00
Jin Hai 59035d9892
#1454 Improve code quality (#1948)
* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Improve code quality

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix issue

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-17 21:05:17 +08:00
yukun 3e1b2ab4a0
Filtering by numeric scalar fields prototype (#1919)
* Add hybrid request handler

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add C++ sdk for createcollection and insertentities

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add RequestHandler

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add test case for hybrid insert

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix sqlite bug for createcollection

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridQuery Handler DBImpl and ExecBinaryQuery

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridSearch sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridSearch test case

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix HybridSearch bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix HybridSearch crash bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change void* to vector<uint8_t> in Attr codec

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add context and new search task

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add merge for Hybrid

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add AST validation

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add unittest for hybrid

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix hybrid search nq bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix bugs after merge master

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix clang format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix unittest bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix Codacy

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix compact unittest bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove grpc request in hybridsearchcontext

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix some codacy quality issue

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix HYBRID_DB_TEST bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Annotate new search task

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add Hybrid RPC handler unittest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix logs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add HybridSearch unittest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-04-16 14:54:12 +08:00
groot 93244a2a79
#1705 Limit the insert data batch size (#1936)
* #1705 Limit the insert data batch size

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-16 00:18:23 +08:00
Jin Hai 0323aa1aad
Merge 080 (#1940)
* #1910 C++ SDK GetIDsInSegment could not work for large dataset (#1911)

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1903 Fix invalid annoy result (#1912)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* #1914: Partition max size should be 4096 (#1915)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* add log (#1913)

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* add log

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: groot <yihua.mo@zilliz.com>

* partition limit 4096

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix py test

Signed-off-by: groot <yihua.mo@zilliz.com>

* update server version (#1916)

Signed-off-by: zw <zw@zilliz.com>

* Update to 0.8.0 (#1918)

* Create new branch 0.8.0 and change preload_table to preload_collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* update helm version

Signed-off-by: zw <zw@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>

* fix issue 1901 (#1920)

* fix issue 1901

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>

* #1900 (#1923)

* add log

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix #1900

Signed-off-by: groot <yihua.mo@zilliz.com>

* Upgrade mishards to 0.8.0 (#1933)

* update grpc server of milvus & rename table name to collection

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Skip CI

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Caiyd 1883 fix rw (#1926)

* #1883 use DiskIO

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix logic error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>

* #1928 Too many data and uid copies when loading files (#1931)

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>

* Update mishards configure files (#1938)

* Update web readme

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] update configure files

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] rename table to collection

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update test.groovy

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update test.groovy

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

Co-authored-by: groot <yhmo@zeronedata.com>
Co-authored-by: shengjun.li <49774184+shengjun1985@users.noreply.github.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
Co-authored-by: zw <zw@zilliz.com>
Co-authored-by: op-hunter <ophunter52@gmail.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
2020-04-15 21:32:20 +08:00
Wang XiangYu 890fe08e7c
Refactor LOG macro and improve log format (#1927)
* update log format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add new log macro

Signed-off-by: wxyu <xy.wang@zilliz.com>

* use new log macro instead

Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com>

* add SetThreadName function

Signed-off-by: wxyu <xy.wang@zilliz.com>

* clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* set thread name

Signed-off-by: Xiangyu Wang <xy.wang@zilliz.com>

* add changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add git ignore .swp file

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update log level in LogUtil.cpp

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-15 14:44:00 +08:00
Cai Yudong a63d832772
#1921 use TimeRecorder instead of chrono (#1922)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-13 18:42:31 +08:00
Jin Hai f8a28699b4
Change to 0.8.0 and rename from preload_table to preload_collection (#1904)
* Create new branch 0.8.0 and change preload_table to preload_collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update CHANGELOG

Signed-off-by: jinhai <hai.jin@zilliz.com>

* update helm version

Signed-off-by: zw <zw@zilliz.com>

Co-authored-by: zw <zw@zilliz.com>
2020-04-11 19:01:29 +08:00
groot ad4645fce0
check task table limit (#1902)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-11 13:13:32 +08:00
shengjun.li b04b3e942c
#1897 add heap_swap_top (#1898)
* add heap_swap_top

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix wrong code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-04-11 12:56:32 +08:00
Wang XiangYu 3bc17d8cc2
Fix multi client search crash in tracing module fix #1789 fix #1832 (#1899)
* Fix multi client search crash in tracing module fix #1789 fix #1832

Signed-off-by: wxyu <xy.wang@zilliz.com>

* add lock for every context_map_ access

Signed-off-by: wxyu <xy.wang@zilliz.com>

* remove never used variable

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-11 12:12:38 +08:00
groot c8a59b273c
#1276 SQLite throw exception when create 50000+ partitions in a table (#1879)
* #1276 SQLite throw exception when create 50000+ partitions in a table

Signed-off-by: groot <yihua.mo@zilliz.com>

* #1276 SQLite throw exception when create 50000+ partitions in a table

Signed-off-by: groot <yihua.mo@zilliz.com>

* remove some log

Signed-off-by: groot <yihua.mo@zilliz.com>

* remove some log

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* typo

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-10 22:51:01 +08:00
groot e25c5af2d7
rename table to collection (#1888)
* rename table to collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* rename collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* rename

Signed-off-by: groot <yihua.mo@zilliz.com>

* Update change log

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-10 19:17:33 +08:00
groot 9074366f42
fix potential hang bug (#1884)
* test

Signed-off-by: groot <yihua.mo@zilliz.com>

* test

Signed-off-by: groot <yihua.mo@zilliz.com>

* test

Signed-off-by: groot <yihua.mo@zilliz.com>

* unittest

Signed-off-by: groot <yihua.mo@zilliz.com>

* format code

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix crash

Signed-off-by: groot <yihua.mo@zilliz.com>

* merge master

Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-10 09:49:31 +08:00
Cai Yudong a05ec6a972
Caiyd 1885 optimize knowhere ut (#1889)
* optimize knowhere unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code clean

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize knowhere util

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest build error

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry CI

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-09 23:37:13 +08:00
BossZou 354f29a058
Refactor log on search and insert request (#1887)
* fix stash conflict

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Log config information and device information (fix #1893)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add index annoy into http module (fix #1882)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Refactor log on search and insert request (fix #1886)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* use vsnprintf to avoid oom

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* fix PR Quality problem & remove surplus comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* replace strlen with strnlen & remove some surplus comments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-09 21:04:18 +08:00
Cai Yudong 66e38d20cd
Caiyd 1881 bad alloc (#1892)
* #1881 update storage APIs, use int64_t for size

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-09 17:13:15 +08:00
Cai Yudong cc3b75b6b7
code revert (#1877)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-07 18:58:18 +08:00
BossZou a4b31cb02f
[skip ci] Fix incorrect information in http readme (fix #1812) (#1878)
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-07 18:56:19 +08:00
Cai Yudong bf6d22e2f5
#1873 fix index file serialize to incorrect path (#1874)
* #1873 fix index file serialize to incorrect path

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* not create sq8h index when gpu disabled

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-07 16:25:46 +08:00
groot c824c297c1
build index wait interval (#1872)
* #1827 Combine request target vectors exceed max nq

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* merge master

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine meta code

Signed-off-by: groot <yihua.mo@zilliz.com>

* set build index check interval to 5 seconds

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-07 10:05:37 +08:00
groot 6a960cda2c
refine meta code (#1871)
* #1827 Combine request target vectors exceed max nq

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failure

Signed-off-by: groot <yihua.mo@zilliz.com>

* merge master

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine meta code

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-06 18:43:20 +08:00
groot 0fd4b244c0
refine dbimpl (#1869)
* #1827 Combine request target vectors exceed max nq

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* refine dbimpl

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix unittest failure

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-04 21:14:30 +08:00
Jin Hai 35276ffcb0
Rename from table to collection (#1857)
* Change HasTable to HasCollection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change HasTable to HasCollection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change table to collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change Table to Collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change Table to Collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change Table to Collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change Table to Collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change Table to Collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix Unit test

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-03 23:12:41 +08:00
Cai Yudong dc707bb8a3
Caiyd 1858 disable s3 build (#1860)
* #1858 disable s3 build

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* disable s3 in unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-03 16:35:48 +08:00
op-hunter 54337e9361
refactor knowhere inner unittest for annoy and hnsw, which a… (#1855)
* skip ci, refactor knowhere inner unittest for annoy and hnsw, which aimed to remove logs of std output

Signed-off-by: cmli <chengming.li@zilliz.com>

* skip ci, fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix annoy load bugs and add serialize test 4 annoy and hnsw

Signed-off-by: cmli <chengming.li@zilliz.com>

* update changelog

Signed-off-by: cmli <chengming.li@zilliz.com>

* update lower bound of annoy search_k param

Signed-off-by: cmli <chengming.li@zilliz.com>

* lint errorgit add src/utils/ValidationUtil.cpp

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-04-03 14:14:04 +08:00
Cai Yudong 7ccfa5b9f0
Caiyd 1627 move rw index (#1833)
* #1627 move read/write index into codec

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update WriteVectorIndex

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-02 22:44:45 +08:00
Xiaohai Xu f82aa4f493
#1849 Support NSG Deletion (#1839)
* nsg delete

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix request

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix clang

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix typo

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* [skip ci] add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix test_nsg and changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix test

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix test case

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix clang-format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Polish

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-04-02 20:46:58 +08:00
groot f66b43b81b
#1827 Combine request target vectors exceed max nq (#1847)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-02 14:24:28 +08:00
Jin Hai 3e03850169
Merge from 0.7.1 to master (#1840)
* update mishards images to 0.7.1 (fix #1799) (#1802)

* update mishards images to 0.7.1 (fix #1799)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Skip CI

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix lsn meta bug (#1813)

* fix lsn meta bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* fix mysql lsn meta bug

Signed-off-by: groot <yihua.mo@zilliz.com>

* Update CHANGELOG.md (#1814)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1815)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1816)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix compile (#1817)

* Update CHANGELOG.md

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Increase Resources

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Decrease the compiling thread

Signed-off-by: jinhai <hai.jin@zilliz.com>

* #1818 Duplicate data generated after restart milvus server (#1819)

* #1818 Duplicate data generated after restart milvus server

Signed-off-by: groot <yihua.mo@zilliz.com>

* set compile thread number

Signed-off-by: groot <yihua.mo@zilliz.com>

* [skip-ci]Remove auto flush interval from config (#1837)

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compile error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com>
Co-authored-by: groot <yhmo@zeronedata.com>
2020-04-02 14:20:33 +08:00
Jin Hai 0e0739ef87
1845 no UT log (#1846)
* #1845 No UT log

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* No UT log

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-02 09:48:29 +08:00
op-hunter 6e7fb831bd
add annoy params validate check (#1829)
* fix bug & solve compile warning from annoylib

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-04-02 09:47:45 +08:00
Xiaohai Xu 2fcf2855ce
fix sq8h deletion (#1830)
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-04-01 20:50:48 +08:00
Jin Hai 68625cbf1c
Fix 1724 (#1821)
* #1724 Remove unused unittests

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part2

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part3

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part4

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Remove unused unittests - part5

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change table_id to collection_id

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change table to collection

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change table_ to collection_

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Change table_id_ to collection_id_

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix format error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* #1724 Change table_name to collection_name

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Reformat

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change TableFile to Segment

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change TableSchema to CollectionSchema

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix compile lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-01 17:37:55 +08:00
Cai Yudong 531fca8869
Caiyd 1655 gpu ivfflat delete (#1809)
* support GPUIVF delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* calc bitset offset correctly

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support GPUSQ8 delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support GPUIVFPQ delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update unittest for GPU delete

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* delete FAISS compile option -DFAISS_USE_FLOAT16

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-01 10:42:48 +08:00
Xiaohai Xu 88cf5e2bd8
Optimize functions (#1803)
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-03-30 18:11:08 +08:00
Cai Yudong dbbf91db28
remove unnecessary lock (#1801)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-03-30 14:13:02 +08:00
Xiaohai Xu 8a731dea39
Add Unittest for IDMAP (#1774)
* gpu idmap unittest

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix clang format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix build issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* small fix

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix case

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix back

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Compact code

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* efficient code

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix GPU search hang

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-03-30 14:08:56 +08:00
JinHai-CN 4574f9d998 Merge branch '0.7.1' 2020-03-29 18:50:10 +08:00
shengjun.li 52ac810a1f
fix share_ptr array (#1786)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-03-28 20:27:07 +08:00
Tinkerrr 3cdba5b62e
fix (#1790)
Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com>
2020-03-28 19:27:21 +08:00
groot a3935eed71
unitest validation (#1788)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-03-28 19:26:39 +08:00