Commit Graph

5530 Commits (d58f6ff817245917a1bfcb223f9d168743cc43e2)

Author SHA1 Message Date
quicksilver 359a458bff
Update nightly cron (#2681)
* Change Milvus Helm Version to v0.10.1

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* [skip ci] update nightly cron

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* [skip ci] update nightly cron

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-06-28 16:25:18 +08:00
quicksilver 8e09c29f2a
[skip ci] update nightly cron (#2680)
* Change Milvus Helm Version to v0.10.1

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* [skip ci] update nightly cron

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-06-28 16:18:05 +08:00
quicksilver f0fc403270
Change Milvus Helm Version to v0.10.1 (#2678)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-06-28 13:57:20 +08:00
JinHai-CN 0f6e36a23e [skip ci] Update cron
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-26 14:29:05 +08:00
Jin Hai 0b16c92ccc
Increase GRPC concurrency (#2671)
* Increase GRPC concurrency

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Add limitation

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-26 12:16:44 +08:00
shengjun.li 0fb88f733a
fix too many copies (#2660)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-24 09:46:48 +08:00
groot ed012f2980
Flush data when insert buffer is full (#2659)
* fix potential problem

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* insert buffer full trigger flush

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* cleanup files if no collection flushed

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* format

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-06-24 09:23:20 +08:00
op-hunter 3b38f7f198
#2649 Remove annoy search parameter validation (#2650)
* remove annoy search parameter validation

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix test failed

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error again

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-23 23:41:04 +08:00
Xiaohai Xu d4fc48bfa7
Fix naming inconsistency (#2654)
* fix naming

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix clang-format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-06-23 20:23:14 +08:00
groot d15487656c
fix potential problem (#2657)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-23 15:30:21 +08:00
shengjun.li cd04f3fc9f
Suit the range of HNSW parameters (#2638)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-22 16:42:15 +08:00
groot 0500d26ce2
Merge pull request #2644 from yhmo/v101
Add internal switch to enable/disable search combine
2020-06-22 14:36:53 +08:00
groot cb09380c6c typo
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-21 21:33:05 +08:00
groot ac7403f00b search combine configable
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-21 19:03:21 +08:00
groot 6e8fc976f3
#2640 (#2641)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-20 20:31:35 +08:00
groot 69eb1d254e #2640
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-20 17:47:10 +08:00
shengjun.li 48e8cd36c2
Fix HNSW and RNSG index files size (#2618)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-20 14:06:05 +08:00
quicksilver 583458836f
Ignore the helm dependency update (#2633)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-06-20 10:32:56 +08:00
BossZou 6ce179b170
[skip ci] Enlarge timeout value for creating collection (#2630)
Signed-off-by: yhz <413554850@qq.com>
2020-06-19 19:46:25 +08:00
del-zhenwu a7b7ca68da
update cron on 0.10.1 (#2615)
Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
2020-06-19 11:35:46 +08:00
Cai Yudong 6ac3d2bd08 fix Milvus docker image report illegal instruction (#2598)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 10:44:23 +08:00
shengjun.li 290ff2e439 Support IVF_PQ on GPU with using metric_type IP (#2593)
* Support IVF_PQ on GPU with using metric_type IP

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* remove useless code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-19 10:42:23 +08:00
groot 508da260ca fix #2578 (#2591)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-19 10:35:07 +08:00
groot 36d2214fc9 fix random crash of INSERT_DUPLICATE_ID case (#2557)
* fix random crash of INSERT_DUPLICATE_ID case

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* cahnge ver

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* sdk readme

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut failure

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-19 10:33:08 +08:00
JinHai-CN bd7e0ee5c9 Update code due to 0.10.1
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-19 10:07:55 +08:00
BossZou dae3beab34
[skip ci] Update milvus images (#2567)
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-06-16 00:32:13 +08:00
del-zhenwu 8aa4a37af6
[skip ci] disable cron on 0.10.0 (#2566)
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* disable restart case

Signed-off-by: zw <zw@milvus.io>

* [skip ci] Disable cron on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* enable cron

Signed-off-by: zw <zw@milvus.io>

* [skip ci] disable cron on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* [skip ci] remove repeat

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-16 00:01:22 +08:00
Wang XiangYu 5f3c005247
fix cache.cache_size range check error (#2565)
* fix cache.cache_size range check error

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update ci

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-06-15 22:44:58 +08:00
del-zhenwu d1311a7112
[skip ci] Disable cron on 0.10.0 (#2555)
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* disable restart case

Signed-off-by: zw <zw@milvus.io>

* [skip ci] Disable cron on 0.10.0

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-15 15:13:24 +08:00
BossZou 1f443acb3b
Mishards 010 upgrade & Add config unit in demo config (#2550)
* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Upgrade mishards for milvus v0.10.0 (fix #2548)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Launch server fail using demo config (#2549)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Remove surplus config yaml

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-06-13 19:45:31 +08:00
Wang XiangYu bcd3de2b41
[skip ci]update comments in config file (#2542)
* update comments in config file

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:06:10 +08:00
Wang XiangYu 83cc812935
Upgrade config (#2511)
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* config cluster

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config general

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config network

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config wal

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config cache

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config gpu and metrics

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config logs

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update server_config.template

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config with unit

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* disable restart case

Signed-off-by: zw <zw@milvus.io>

* fix gpu compile failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Fix lint and comments

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix wal unittest failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix some unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update config test cases

Signed-off-by: zw <zw@milvus.io>

* Fix config error

Signed-off-by: yhz <413554850@qq.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip-ci] fix test cases

Signed-off-by: zw <zw@milvus.io>

* fix config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* make -j3 on ci

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update python test

Signed-off-by: wxyu <xy.wang@zilliz.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update status message

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix set config bug

Signed-off-by: wxyu <xy.wang@zilliz.com>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: yhz <413554850@qq.com>
2020-06-12 09:22:26 +08:00
yukun a9629951e4
Improve ut coverage (#2516) (#2522)
* Improve ut coverage

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Delete unused code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add fiu in HybridSearchRequest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change BinaryQuery validation check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-11 18:45:14 +08:00
BossZou 551be85d83
Improve ut coverage (#2516) (#2517)
* Improve ut coverage (#2516)

Signed-off-by: yhz <413554850@qq.com>

* Improve ut coverage

Signed-off-by: yhz <413554850@qq.com>

* CI retry

Signed-off-by: yhz <413554850@qq.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-11 01:25:55 +00:00
del-zhenwu 0543c59471
Update helm version to 0.10.0 (#2527)
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* disable restart case

Signed-off-by: zw <zw@milvus.io>

* Merge with helm upgrade to 0.10.0

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-10 20:06:58 +08:00
groot 19a9d0d1f3
revert code (#2525)
* revert code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-10 19:27:36 +08:00
del-zhenwu 7092c85ba6
[skip ci] disable restart case. (#2515)
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* disable restart case

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-09 15:08:12 +00:00
Jin Hai 34b4d72517
Add exception throw on mysql meta error (#2490)
* Add exception throw on mysql meta error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip ci] update changelog

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix Unit test

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-08 21:16:22 +08:00
del-zhenwu a6a9021974
Add http (#2505)
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* add http test stage

Signed-off-by: zw <zw@milvus.io>

* remove

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-08 11:36:55 +00:00
groot bd0049ff14
return hnsw name (#2501)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-07 12:03:01 +00:00
Cai Yudong 8254e9ed67
fix performance (#2499)
* optimize sq_get_distance_computer

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_ref

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx512

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize Codec

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx512.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-06 07:41:57 +00:00
groot 9f7995cbba
default nlist 2048 (#2498)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-06 05:03:06 +00:00
Jin Hai 4b5eff3672
Fix 2495: Add more reason of creating lock file failed (#2496)
* Fix 2495: Add more reason of creating lock file failed

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix 2495: Update changelog

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compile error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-05 01:58:17 +00:00
BossZou 167743c993
read_only node detect delete vector operation using mishards #2368 (#2473)
* Add new request ReLoadSegments

Signed-off-by: yhz <413554850@qq.com>

* Finish load segments functionality

Signed-off-by: yhz <413554850@qq.com>

* Add api in grpc

Signed-off-by: yhz <413554850@qq.com>

* update Reloadsegments

Signed-off-by: yhz <413554850@qq.com>

* .

Signed-off-by: yhz <413554850@qq.com>

* create new blacklist if not exists

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update api names

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Finish mishard for support api reloadsegments

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update changlog

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Add more details when failed in search task

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Fix compile issue

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update mishards requirements

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Code format

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* modify docker images in mysql all_in_one

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* update shards code

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Move updatedeldocs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* Move reload segment del docs function to dbimpl

Signed-off-by: yhz <413554850@qq.com>

* [skip ci] correct shards requirements

Signed-off-by: yhz <413554850@qq.com>
2020-06-02 14:34:19 +08:00
quicksilver 5ed7f40a49
Update helm charts dependencies (#2481)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-06-02 10:17:51 +08:00
groot 0710ececae
#2373 Build index for small segment waste time on waiting background index thread finish (#2482)
* #2349 Drop collection timeout if too many partitions created on collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #2373 Build index for small segment waste time on waiting background index thread finish

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-01 10:16:37 +00:00
shengjun.li 257ea7782f
fix merge result (#2463)
* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix tests

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-01 07:31:58 +00:00
groot 1daf00dcf7
#2394 Drop collection timeout if too many partitions created on colle… (#2477)
* #2349 Drop collection timeout if too many partitions created on collection

Signed-off-by: groot <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-01 11:19:50 +08:00
del-zhenwu fdfb6526e0
fix test_during_creating_index_restart (#2474)
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
2020-05-30 09:03:28 +00:00
op-hunter 9de7f710e6
optimize the implemention of k-selection on faiss gpu version (#2467)
* optimize the implemention of k-selection on faiss gpu version

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log, re-classified the pr from feature to improvement

Signed-off-by: cmli <chengming.li@zilliz.com>

* update change log

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-05-30 04:27:59 +00:00