bf5fdc3131
Fix memory leak in IVF indexes ( #4325 )
...
* fix memory lead in IVF indexes
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* [skip ci] modify change log
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-12-01 09:36:04 +08:00
99245c0c63
#4174 Search out of memory: CPU2GPU1 with index flat ( #4178 )
...
* Fix gpu resource issue
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix_ut
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-11-05 13:57:50 +08:00
773deab62e
#4075 Improve performance for create large amount of partitions ( #4083 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-10-24 10:35:58 +08:00
8d5c1a87d5
#3773 milvus support IVF_PQ run on fpga device ( #3996 )
...
* This is my commit message Signed-off-by jack hou Developer <364427912@qq.com>
Signed-off-by: root <364427912@qq.com>
* fix build with -u -a
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix run on GPU
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix ThirdPartyPackages.cmake
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-10-23 09:41:28 +08:00
235c51c385
Fix server crashed during search with index pq ( #3954 )
...
* fix pq distances cal bug
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* fix pq distance calculation outofrange according to faiss suggests
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* fix change log
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-10-17 09:42:19 +08:00
19bda5cf97
The search of SQ8H index is slower than the previous version ( #3957 )
...
Signed-off-by: cqy <yaya645@126.com>
2020-10-12 19:55:26 +08:00
f46e2a621d
fix the bug of DeleteTask state not change ( #3935 )
...
* fix the bug of DeleteTask state not change
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2020-10-10 14:16:31 +08:00
7692a6d437
set blacklist a NULL pointer if no item deleted ( #3827 )
...
* set blacklist a nullptr is no item deleted
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix multi thread access bitset
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-22 10:01:04 +08:00
50d5b9f179
[skip ci] Upgrade mishards up to v0.10.3 ( #3785 )
...
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-09-17 19:35:03 +08:00
f962e84822
[skip ci] modify the change log ( #3761 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-16 10:32:35 +08:00
18695db507
If the GPU cache is too small, IVF_SQ8H using multiple GPUs will cause to crash ( #3744 )
...
* fix ivf_sq8h
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix cache release
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-15 20:21:10 +08:00
2737c3fbdd
[skip ci] Remove check for topk in mishards ( fix #3709 ) ( #3710 )
...
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-09-12 22:27:41 +08:00
49a5cdf682
When the topk of GPU search larger than 2048,turning into the CPU search ( #3647 )
...
* topk of GPU searching
Signed-off-by: cqy <yaya645@126.com>
* change the changelog
Signed-off-by: cqy <yaya645@126.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-09 13:32:10 +08:00
74639a3ba7
fix to check search param of BIN_IVF_FLAT ( #3660 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-09 09:14:40 +08:00
18c630a4be
release search task in time ( #3609 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-05 11:36:55 +08:00
d5ecdf2dda
GPU search turn into CPU search when nprobe >2048 ( #3594 )
...
* gpu nprobe
Signed-off-by: cqy <yaya645@126.com>
* [skip ci] changelog
Signed-off-by: cqy <yaya645@126.com>
2020-09-04 18:49:52 +08:00
a75c0a572c
Allow more choices for the parameter ‘m’ of CPU IVF_PQ ( #3469 )
...
* fix the IVFPQ fixed M problem
Signed-off-by: cqy <yaya645@126.com>
* fix compile
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-27 11:37:25 +08:00
1962b6e78b
Specifying distance type at runtime for Flat index ( #3273 )
...
* modfied IndexIDMAP.cpp && SearchTask.cpp
Signed-off-by: Zhengda Wu <wuzhengda55@gmail.com>
* Specifying distance type at runtime for Flat index(CPU)
Signed-off-by: Zhengda Wu <wuzhengda55@gmail.com>
* Specifying distance type at runtime for Flat index(GPU)
Signed-off-by: Zhengda Wu <wuzhengda55@gmail.com>
Co-authored-by: Zhengda Wu <wuzhengda55@gmail.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-17 17:05:59 +08:00
f59c240457
Update to 0.10.3
...
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-08-15 16:21:43 +08:00
f9f597b257
[v0.10.2] (mishards) Fix mishards search bug ( #3169 )
...
* [skip ci]Reverse query result if metric is IP
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
* [skip ci] Update version check
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-08-07 19:33:49 +08:00
09cf1b323e
fix crash when GPU search ( #3074 )
...
* fix crash when GPU search
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-31 09:46:55 +08:00
b1b1165c56
Opt faiss builder ( #3067 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-31 09:35:31 +08:00
91d3fe5cbc
Improve IVF search performance when NQ and nProbe are both large ( #2984 )
...
* fix indexflat search
Signed-off-by: cqy <yaya645@126.com>
* add parallel_policy_threshold
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-24 15:56:47 +08:00
dd938878ea
fix ( #2981 )
...
Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>
2020-07-23 17:02:28 +08:00
3908d10498
Fix the result merging of IVF_PQ IP ( #2953 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-22 10:08:05 +08:00
a88f5b6cbb
Fix index size ( #2941 )
...
* add virtual IF UpdateIndexSize
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* update ivf index size
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix PQ logic
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix index size of index hnsw, annoy and nsg
Signed-off-by: cmli <chengming.li@zilliz.com>
* add GetSize() interface 4 SPTAG
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix binary ivf
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: sahuang <xiaohai.xu@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-21 14:26:08 +08:00
1806e7e4b0
Update CHANGELOG.md
...
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-07-21 09:35:31 +08:00
7a594470cb
[skip ci] Upgrade mishards for milvus 0.10.1 ( #2933 )
...
Signed-off-by: yhz <413554850@qq.com>
2020-07-20 16:28:05 +08:00
8046f2946e
Hotfix 2487 ( #2892 )
...
* update.
Signed-off-by: Tinkerrr <linxiaojun.cn@outlook.com>
* update.
Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>
* add fd check
Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>
* fix
Signed-off-by: Tinkerrr <xiaojun.lin@zilliz.com>
2020-07-17 21:00:42 +08:00
a474ff269a
Server down during loading data ( #2807 )
...
* Server down during loading data
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix test_config.cpp
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* ci retry
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Change limit_in_bytes init value to max
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-07-14 00:40:20 +08:00
585c8ea018
fix rnsg ip ( #2827 )
...
Signed-off-by: cqy <yaya645@126.com>
2020-07-13 14:05:28 +08:00
5447abec5b
fix #2768 ( #2809 )
...
* fix #2752
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix #2768
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo again
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-07-12 15:41:45 +08:00
d7c47a7905
fix #2752 ( #2789 )
...
* fix #2752
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-10 16:49:47 +08:00
4922c9532f
fix a bug of getting wrong nprobe limitation in knowhere on GPU version ( #2788 )
...
* fix a bug of getting wrong nprobe limitation in knowhere on GPU version
Signed-off-by: cmli <chengming.li@zilliz.com>
* update changelog and fix build error
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-09 23:15:07 +08:00
4038b0653e
fix too many data copies ( #2784 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-09 09:23:49 +08:00
8fcdf73bc8
fix 2739 ( #2746 )
...
Signed-off-by: Tinkerrr <linxiaojun.cn@outlook.com>
2020-07-07 14:45:37 +08:00
5bd75f9381
Web fix 2690 ( #2691 )
...
* Remove body parser in show partitions endpoints (fix #2690 )
Signed-off-by: yhz <413554850@qq.com>
* Remove surplus output and comments
Signed-off-by: yhz <413554850@qq.com>
* Remove surplus output and comments
Signed-off-by: yhz <413554850@qq.com>
* Remove space line
Signed-off-by: yhz <413554850@qq.com>
* Remove surplus ut
Signed-off-by: yhz <413554850@qq.com>
* laten sleep time in case test_delete_vector_collection_count_no_flush
Signed-off-by: yhz <413554850@qq.com>
2020-07-04 16:31:03 +08:00
367c72675c
#2692 Milvus hangs during multi-thread concurrent search ( #2715 )
...
* #2642 Create index failed caused by server crashed.
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* #2692 Milvus hangs during multi-thread concurrent search
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-02 19:03:24 +08:00
714b777d05
#2642 Create index failed caused by server crashed. ( #2701 )
...
* #2642 Create index failed caused by server crashed.
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-02 09:37:15 +08:00
3b38f7f198
#2649 Remove annoy search parameter validation ( #2650 )
...
* remove annoy search parameter validation
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix test failed
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error again
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-23 23:41:04 +08:00
cd04f3fc9f
Suit the range of HNSW parameters ( #2638 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-22 16:42:15 +08:00
48e8cd36c2
Fix HNSW and RNSG index files size ( #2618 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-20 14:06:05 +08:00
6ce179b170
[skip ci] Enlarge timeout value for creating collection ( #2630 )
...
Signed-off-by: yhz <413554850@qq.com>
2020-06-19 19:46:25 +08:00
6ac3d2bd08
fix Milvus docker image report illegal instruction ( #2598 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 10:44:23 +08:00
290ff2e439
Support IVF_PQ on GPU with using metric_type IP ( #2593 )
...
* Support IVF_PQ on GPU with using metric_type IP
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* remove useless code
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-19 10:42:23 +08:00
bd7e0ee5c9
Update code due to 0.10.1
...
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-19 10:07:55 +08:00
5f3c005247
fix cache.cache_size range check error ( #2565 )
...
* fix cache.cache_size range check error
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update ci
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-06-15 22:44:58 +08:00
1f443acb3b
Mishards 010 upgrade & Add config unit in demo config ( #2550 )
...
* Update mishards config
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Update mishards config
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Upgrade mishards for milvus v0.10.0 (fix #2548 )
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Launch server fail using demo config (#2549 )
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* [skip ci] Remove surplus config yaml
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-06-13 19:45:31 +08:00
83cc812935
Upgrade config ( #2511 )
...
* enable cron test on 0.10.0
Signed-off-by: zw <zw@milvus.io>
* fix merge result
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* import test_during_creating_index_restart
Signed-off-by: zw <zw@milvus.io>
* config cluster
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config general
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config network
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config storage
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config wal
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config cache
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config gpu and metrics
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config logs
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update server_config.template
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update changelog
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config with unit
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix clang-format
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update unittests
Signed-off-by: wxyu <xy.wang@zilliz.com>
* disable restart case
Signed-off-by: zw <zw@milvus.io>
* fix gpu compile failed
Signed-off-by: wxyu <xy.wang@zilliz.com>
* Fix lint and comments
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* fix wal unittest failed
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix some unittests
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update config test cases
Signed-off-by: zw <zw@milvus.io>
* Fix config error
Signed-off-by: yhz <413554850@qq.com>
* remove unused code
Signed-off-by: wxyu <xy.wang@zilliz.com>
* Update CHANGELOG
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* [skip-ci] fix test cases
Signed-off-by: zw <zw@milvus.io>
* fix config storage
Signed-off-by: wxyu <xy.wang@zilliz.com>
* make -j3 on ci
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update python test
Signed-off-by: wxyu <xy.wang@zilliz.com>
* remove unused code
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix ut
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix ut
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update status message
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix set config bug
Signed-off-by: wxyu <xy.wang@zilliz.com>
Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: yhz <413554850@qq.com>
2020-06-12 09:22:26 +08:00
551be85d83
Improve ut coverage ( #2516 ) ( #2517 )
...
* Improve ut coverage (#2516 )
Signed-off-by: yhz <413554850@qq.com>
* Improve ut coverage
Signed-off-by: yhz <413554850@qq.com>
* CI retry
Signed-off-by: yhz <413554850@qq.com>
* Update helm config
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* CI retry
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* CI retry
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-11 01:25:55 +00:00
shengjun.li
Xiaohai Xu
groot
jackhouchina
cqy123456
chen qingxiang
BossZou
MXDA
jinhai
Tinkerrr
yukun
groot
op-hunter
shengjun.li
Cai Yudong
Wang XiangYu