40c37cb94d
Limit the number of insert requests processing concurrently ( #3830 )
...
* Limit the number of insert requests processing concurrently
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* code format
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* change the variable name
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* change comment
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-23 13:08:11 +08:00
ca60fa2cc8
update oatpp md5 ( #3832 )
...
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-23 13:08:11 +08:00
9baf7bac14
fix wal recovery issue ( #3818 )
...
* fix wal recovery issue
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-23 13:08:11 +08:00
e5f763f30c
add hidden config engine.max_partition_num ( #3817 )
...
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-23 13:08:11 +08:00
962d0828b7
use multi-segments operation to flush ( #3804 )
...
* use multi-segments operation to flush
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-23 13:08:11 +08:00
0571fe18b9
Fix web start fail for IP family ( #3778 )
...
* Fix web start fail for ip family
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
* update oatpp md5
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
* Add more detail case
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-23 13:08:11 +08:00
bbc46b7d55
when the topk of gpu search >2048 ,turning into the cpu search ( #3640 )
...
* search task
Signed-off-by: cqy <yaya645@126.com>
code refactoring
Signed-off-by: cqy <yaya645@126.com>
* fix sq8h pass
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* modify the range of param 'ef' in HNSW
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-19 16:42:04 +08:00
836a247b6a
Fix the crash caused by empty index param ( #3803 )
...
* fix crash caused by empty index param
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix C++ SDK examples
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-18 18:59:24 +08:00
d1ab5bb2c7
fix the limit of nprobe and nlist ( #3797 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-18 18:59:24 +08:00
2f4207acb2
Improve MergeTopkToResultSet performance ( #3788 )
...
* Fix WebServer README.md
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add changelog
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix cpplint
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix Readme.md
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix MergeTopkToResultSet performance problem
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-18 18:59:24 +08:00
ee4789cc5f
avoid snapshot warning during deleting entity ( #3798 )
...
* avoid snapshot warning
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-18 18:59:24 +08:00
b383748870
Fix the bug of file not close when check sum failed. ( #3783 )
...
* Fix the bug of file not close when check sum failed.
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* change the log level to error
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* fix cpp lint problem
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-18 18:59:24 +08:00
b29954cd9e
avoid dead circle when merge failed ( #3787 )
...
* avoid dead circle when merge failed
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-18 18:59:24 +08:00
35a5631698
add adaptive merge strategy ( #3780 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-18 18:59:24 +08:00
6c314348e9
Fix webserver readme for 0.11.0 ( #3765 )
...
* Fix WebServer README.md
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add changelog
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix cpplint
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-18 18:59:24 +08:00
caa251f30f
hold FaissIVFQuantizerPtr in IVFSQHybrid ( #3770 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
7a4e87fcce
modify merge strategy ( #3757 )
...
* modify merge strategy
Signed-off-by: groot <yihua.mo@zilliz.com>
* clean code
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
1f5850a4a7
close lockfile when server stop ( #3737 )
...
* close lockfile when server stop
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
* fix something error
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
3b8a69a3e2
#3756 ignore deleted docs file if it doesnt exist ( #3766 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
b5ffb46d21
fix trace level log files missing ( #3769 )
...
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
2a0bf43445
New operation named MultiSegmentsOperation support multi-segments during flush ( #3733 )
...
* Add multiSegmentsOpera
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
* rish test
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
* Add new operation MultiSegmentsOperation
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
* Recoveray ut itls
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
* fix judeg bug
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
ade8bb3f23
fix config usage ( #3747 )
...
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
41dbcf7986
(db/snapshot): fix GC PartitionCommit ( #3750 )
...
Signed-off-by: peng.xu <peng.xu@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
b62a05cb9b
Improve code of dabloom ( #3694 )
...
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
52b0181e89
Apply more stale check rules for Operations ( #3721 )
...
* (db/snapshot): Update Operations to handle stale errors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add some print in ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
82711d6065
rename segment_row_count to segment_row_limit ( #3724 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
dd31d87c4c
GET_PAGE_ENTITY sometimes failed in test_web ( #3735 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
5f9a7405f6
[skip ci] disable two compact cases ( #3731 )
...
Signed-off-by: zw <zw@milvus.io>
Co-authored-by: zw <zw@milvus.io>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
8a4d8c4d51
add some check to avoid server crash ( #3715 )
...
* add some check to avoid server crash
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* rm useless code
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
3b29ae776f
fix delete entity performance issue ( #3667 )
...
* fix delete entity performance issue
Signed-off-by: groot <yihua.mo@zilliz.com>
* force flush for delete
Signed-off-by: groot <yihua.mo@zilliz.com>
* improve delete performance
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix delete performce issue
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix hang
Signed-off-by: groot <yihua.mo@zilliz.com>
* segment row count min limit
Signed-off-by: groot <yihua.mo@zilliz.com>
* merge 0.11.0
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix python test
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix python test failed
Signed-off-by: groot <yihua.mo@zilliz.com>
* add log
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
be67ef32f9
(db/snapshot): Implement FlushableMappingField for better metadata snapshot ( #3655 )
...
* Update metadata for mappings
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* Update metadata for mappings 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* Update metadata for mappings 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* Update metadata for mappings 4
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* Update metadata for mappings 5
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* Update metadata for mappings 6
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* Update metadata for mappings 7
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): optimize PartitionCommit metadata implementation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): optimize PartitionCommit metadata implementation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update mock logic
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): small change
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): small change
Signed-off-by: peng.xu <peng.xu@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
2e564700d7
Fix#3689 Client get stuck if drop collection during creating index ( #3714 )
...
* fix jaccard distance compute
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* edit changelog
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
b2377762d1
fix snapshot not release bug ( #3708 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-17 14:15:56 +08:00
36da3045f6
Set a timeout for a whole job on Github action pipeline ( #3745 )
...
* Archive dev test logs in Jenkins CI
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-09-15 19:24:28 +08:00
eb6954d506
remove set config case ( #3587 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
13103a6dd4
add log ( #3680 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
aba62ad25f
fix3626 ( #3699 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
d03a8e4354
(db/snapshot): update LSN logic ( #3682 )
...
Signed-off-by: peng.xu <peng.xu@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
76fa7aa4c6
fix #3672 0.11.0 docker image is 200M larger due to the un-expected installation of openblas ( #3686 )
...
* fix #3672
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* fix #3672
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
b79f9f0775
avoid dabloom warning ( #3674 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
23b9081923
add memory limit check ( #3669 )
...
* add memory limit check
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* format code
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* add changelog
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* change the implementation of memory check
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* revert code
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
* revert code
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
1dadc4e4f9
Fix loadcollection memory usage is one times larger than expect ( #3664 )
...
* rm new rearanged_data to avoid additional memory usage
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* fix cpu version
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* fix data_ deconstruct twice error
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
8b92021a27
fix nprobe error ( #3658 )
...
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
4882113eb0
set wal lsn to meta ( #3638 )
...
* wal lsn
Signed-off-by: groot <yihua.mo@zilliz.com>
* temp comment
Signed-off-by: groot <yihua.mo@zilliz.com>
* recovery from meta lsn
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
6e3b32e2f6
Clear cache after merge ( #3645 )
...
* rewrite merge code
Signed-off-by: groot <yihua.mo@zilliz.com>
* clear cache after merge
Signed-off-by: groot <yihua.mo@zilliz.com>
* clear cache after drop
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix bug
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix unittest
Signed-off-by: groot <yihua.mo@zilliz.com>
* clear index cache after drop
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
a6ccffc35a
Proto of C++ sdk is different from milvus server ( #3654 )
...
* Fix getEntity when no query_params and payload is provided in webserver
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Change entities object to json::array
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Proto of C++ sdk is different from milvus server
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
b2bbf7b847
add build option -rdynamic in CMakeList.txt ( #3632 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
11c68b20c2
rewrite dabloom ( #3629 )
...
* rewrite dabloom
Signed-off-by: groot <yihua.mo@zilliz.com>
* refine code
Signed-off-by: groot <yihua.mo@zilliz.com>
* unittest
Signed-off-by: groot <yihua.mo@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
08d5c145f0
fix building index check ( #3628 )
...
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
4305cfc65f
Fix crash where getting octets information ( #3625 )
...
* Fix crash where getting octets information
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
* Code format
Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-09-12 15:54:55 +08:00
chen qingxiang
BossZou
groot
Wang Xiangyu
cqy123456
shengjun.li
yukun
XuPeng-SH
Jin Hai
del-zhenwu
XuanYang-cn
quicksilver
Xiaohai Xu
Cai Yudong