f8cb9e3d43
(db/snapshot): add AddSegmentFileOperation ( #2924 )
...
* (db/snapshot): update for row count
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bug in NewSegmentOperation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove dummy print
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add some test for row count
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update size logic
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update size logic related ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): rollback if operation is not done
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): clean store
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove some dependency
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update for store
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update Store.h
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update store related code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add field element modification operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): change new operation name
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add Segment File Operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-19 16:11:02 +08:00
482bc53950
(db/snapshot): Update Store ( #2897 )
...
* (db/snapshot): update for row count
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bug in NewSegmentOperation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove dummy print
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add some test for row count
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update size logic
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update size logic related ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): rollback if operation is not done
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): clean store
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove some dependency
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update for store
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update Store.h
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update store related code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add field element modification operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): change new operation name
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-18 19:30:45 +08:00
d0018307b8
Add python functional tests about timeout ( #2793 )
...
* Add python tests about param timeout
Signed-off-by: yangxuan <jumpthepig@gmail.com>
* add faiss configure args msg print
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* change assertion conditions
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-07-18 15:30:27 +08:00
f80c592679
fix #2768 ( #2809 )
...
* fix #2752
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix #2768
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo again
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-07-18 15:24:10 +08:00
01437cef39
Fix warning master ( #2894 )
...
* fix clean unused variable warning in knowhere
Signed-off-by: cqy <yaya645@126.com>
* clean unused variable warning in knowhere
Signed-off-by: cqy <yaya645@126.com>
* clean unused variable warning in knowhere
Signed-off-by: cqy <yaya645@126.com>
* clean unused variable warning in knowhere
Signed-off-by: cqy <yaya645@126.com>
* fix konwhere warning
Signed-off-by: cqy <yaya645@126.com>
2020-07-18 04:47:29 +00:00
708bad368a
segment writer ( #2873 )
...
* redefine SegmentWriter
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* redefine SegmentWriter
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* refine code
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* mem insert
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* unittest
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix some issues
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* segment reader
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* segment writer
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* remove files
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* remove code
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* merge
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix merge
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* get entities
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-17 20:53:45 +08:00
771585ba0c
use BlockingQueue in JobMgr ( #2885 )
...
* using BlockingQueue in JobMgr
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-17 09:37:00 +08:00
52ab18fe0e
opt faiss build ( #2883 )
...
* opt faiss build
Signed-off-by: cqy <yaya645@126.com>
2020-07-17 09:34:34 +08:00
96e9ef235d
Add some cpp unittests about scheduler algorithm ( #2888 )
...
* add some testcases
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* reformat code
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
(cherry picked from commit 8135cee414
2020-07-16 19:49:08 +08:00
7a39a00498
Fix Prometheus rebuild problem ( #2878 )
...
* Solve prometheus repeatedly build problem
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* recover build.sh
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* refactor build.sh
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* altering changelog
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* rm not-useful comments
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-07-16 19:45:24 +08:00
285f849137
(db/snapshot) Add snapshot DB unittest ( #2856 )
...
* (snapshot) Add snapshot DB unittest
Signed-off-by: yhz <413554850@qq.com>
* Optimize mock db
Signed-off-by: yhz <413554850@qq.com>
* correct server context file name
Signed-off-by: yhz <413554850@qq.com>
2020-07-16 08:06:34 +00:00
3dddbb72f7
(db/snapshot): update size related logic ( #2877 )
...
* (db/snapshot): update for row count
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bug in NewSegmentOperation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove dummy print
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add some test for row count
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update size logic
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update size logic related ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): rollback if operation is not done
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): clean store
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove some dependency
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-16 15:33:56 +08:00
38dfdf4034
# 2828 opt faiss build ( #2855 )
...
* add FAISS_USE_FLOAT16
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix faiss build
Signed-off-by: cqy <yaya645@126.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-16 09:03:26 +08:00
1d9ccd9bb7
add MILVUS_SUPPORT_SPTAG for some code ( #2866 )
...
* add MILVUS_SUPPORT_SPTAG for some code
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* use knowhere::IndexEnum in db
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* keep EngineType order
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-15 17:42:44 +08:00
5bf9b8f030
(db/snapshot): Update row count related logic ( #2862 )
...
* (db/snapshot): update for row count
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bug in NewSegmentOperation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove dummy print
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add some test for row count
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-15 17:31:00 +08:00
77cf783d4c
#2802 Add new index: IVFSQ8NR_NM ( #2865 )
...
* add ivfsq8nr_nm
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* refactor
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix error
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* [skip ci] remove unused param
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-07-15 16:46:39 +08:00
d473c7ad2b
snapshot integrate ( #2854 )
...
* add test_segment
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update interface GetResPath
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SSSegmentWriter
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-15 14:32:09 +08:00
4cb8383775
move index type try2 ( #2860 )
...
* move EngineType and MetricType to MetaTypes.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* move IndexType
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-15 11:37:05 +08:00
7bd60398a2
redefine SegmentWriter ( #2853 )
...
* redefine SegmentWriter
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* redefine SegmentWriter
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* refine code
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-15 02:02:31 +00:00
a8033a5954
(db/snapshot) Add snapshot DB module ( #2851 )
...
* New db impl
Signed-off-by: yhz <413554850@qq.com>
* Add db impl for snapshot
Signed-off-by: yhz <413554850@qq.com>
* Update operations step holders
Signed-off-by: yhz <413554850@qq.com>
* Update snapshot dbimpl & create collection test pass
Signed-off-by: yhz <413554850@qq.com>
* Delete operation pass
Signed-off-by: yhz <413554850@qq.com>
* Add constraints for db table
Signed-off-by: yhz <413554850@qq.com>
* Update index operation steps behavior
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Add lock in mysql engine and remove lock in store
Signed-off-by: yhz <413554850@qq.com>
* Index unittest case pass
Signed-off-by: yhz <413554850@qq.com>
* Add merge operation step context
Signed-off-by: yhz <413554850@qq.com>
* Remove constraints in DB and snapshot ut pass
Signed-off-by: yhz <413554850@qq.com>
* update DBImple
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Add DBHelper to generate sql & add mock Mem db & snaoshot ut pass
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Add mock db engine
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Update DB select API using template for resource
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Update snapshot db name and move them under folder meta
Signed-off-by: yhz <413554850@qq.com>
* clang-format
Signed-off-by: yhz <413554850@qq.com>
2020-07-14 19:11:24 +08:00
6779ee29da
Update sqlite_6_to_4.sql ( #2832 )
...
Simplify SQL statements
Signed-off-by: dddddai <853238012@qq.com>
2020-07-14 15:36:20 +08:00
66add7b9d1
remove old api ( #2833 )
...
* remove old api
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* refine insert code
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* refine code
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-14 09:48:53 +08:00
dc31742538
Hnsw sq8 support ( #2835 )
...
* modify for SQ8
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* add hnsw-sq8-support
Signed-off-by: cmli <chengming.li@zilliz.com>
* add unittest 4 hnsw_sq8nr and fix several bugs
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-14 09:45:08 +08:00
777c76c87a
Fix rnsg master ( #2831 )
...
* fix rnsg ip
Signed-off-by: cqy <yaya645@126.com>
* fix rnsg search in master
Signed-off-by: cqy <yaya645@126.com>
2020-07-14 09:30:26 +08:00
e79cbf098e
(db/snapshot): update visitors ( #2837 )
...
* (db/snapshot): add more visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add some Iterators
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add event handler factory
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): process lsn
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): small change
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): precheck duplicate collection in create collection operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitor 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitor 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-13 11:08:30 +00:00
0d20f8a9ca
snapshot integrate ( #2830 )
...
* update HybridQuery
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add IterateHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* IterateHandler opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add SSSegmentReader
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add src/codecs/snapshot
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update latest code
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SSSegmentReader interface
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add SSSegmentWriter
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SSSegmentWriter interface
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest/db build fail
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* use SSSegmentReader and SSSegmentWriter in SSDBImpl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix build issue
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add test_segment
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-13 06:08:15 +00:00
6c5f42f9f1
(db/snapshot): LSN logic change and design EventHandlerFactory ( #2815 )
...
* (db/snapshot): add more visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add some Iterators
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add event handler factory
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): process lsn
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): small change
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): precheck duplicate collection in create collection operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-12 14:18:12 +00:00
141835a7c7
Add SDK support for IVFSQ8NR ( #2821 )
...
* add c++ sdk
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix clang
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-07-12 20:23:25 +08:00
2c3b3a5cfa
snapshot integrate ( #2814 )
...
* update HybridQuery
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add IterateHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* IterateHandler opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add SSSegmentReader
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add src/codecs/snapshot
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update latest code
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SSSegmentReader interface
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add SSSegmentWriter
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SSSegmentWriter interface
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest/db build fail
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* use SSSegmentReader and SSSegmentWriter in SSDBImpl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix build issue
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-11 13:26:31 +00:00
d004d273ad
add unittest 4 hnsw_sq8nr and fix some bug in passing ( #2819 )
...
* modify for SQ8
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* add hnsw-sq8-support
Signed-off-by: cmli <chengming.li@zilliz.com>
* make clang-format and update changelog
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error
Signed-off-by: cmli <chengming.li@zilliz.com>
* hide unittest error temporary
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix ut error
Signed-off-by: cmli <chengming.li@zilliz.com>
* add unittest 4 hnsw_sq8nr and fix several bugs
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-11 10:24:34 +00:00
6e16967b83
fix task_search ( #2808 )
...
* fix task_search
Signed-off-by: cqy <yaya645@126.com>
* fix unit test
Signed-off-by: cqy <yaya645@126.com>
2020-07-11 15:44:51 +08:00
fce41e738d
#2802 Add new index: IVFSQ8NR ( #2805 )
...
* support new index ivfsq8nr
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* remove timeout
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-07-11 13:40:14 +08:00
f94592ecdf
fix #2752 ( #2789 )
...
* fix #2752
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
(cherry picked from commit d7c47a7905
2020-07-11 11:18:05 +08:00
9d4febde7f
(db/snapshot): Add some visitors ( #2800 )
...
* (db/snapshot): add more visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add some Iterators
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-10 16:49:00 +08:00
5857f6401e
rewrite merge manager 1 ( #2797 )
...
* rewrite merge manager 1
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* rewrite merge manager 2
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-10 06:07:09 +00:00
7bf3e6e1c3
Hnsw sq8 support ( #2799 )
...
* modify for SQ8
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* add hnsw-sq8-support
Signed-off-by: cmli <chengming.li@zilliz.com>
* make clang-format and update changelog
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error
Signed-off-by: cmli <chengming.li@zilliz.com>
* hide unittest error temporary
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix ut error
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-10 13:42:24 +08:00
b221cd6898
fix a bug of getting wrong nprobe limitation in knowhere on GPU version ( #2788 )
...
* fix a bug of getting wrong nprobe limitation in knowhere on GPU version
Signed-off-by: cmli <chengming.li@zilliz.com>
* update changelog and fix build error
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-10 09:33:53 +08:00
a8bd2cf4a6
fix too many data copies ( #2784 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-09 11:18:37 +08:00
e587c28644
rewrite mem manager 1 ( #2762 )
...
* rewrite mem manager
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* rewrite mem manager 2
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* refine
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* rewrite mem manager 3
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* merge manager 1
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-09 03:12:44 +00:00
523d97af6d
snapshot integrate ( #2782 )
...
* update HybridQuery
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add IterateHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* IterateHandler opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-08 15:08:08 +00:00
af573f93c9
(db/snapshot): Add some helper visitors ( #2777 )
...
* (db/snapshot): add Segment related visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-08 08:56:56 +00:00
b7d9c2a4db
(db/snapshot) snapshot integrate ( #2770 )
...
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add some APIs for SSDBImpl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* partially add GetVectorById
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* snapshot opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix typo
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update GetVectorByID framework
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename GetResFiles to GetResPath
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update GetVectorByIdSegmentHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add GetEntityByID
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update DataType
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update ParamField
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update GetEntityByIDSegmentHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* support INT8 and INT16 in GetEntityById
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add GetIDsInSegment
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update FieldType
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add InsertEntities and DeleteEntities
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add HybridQuery
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-07 15:36:38 +00:00
5964adef49
(db/snapshot): Implement operation rollback ( #2760 )
...
* (db/snapshot): refactor Operations
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor Operations
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add rollback for operations
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-07 11:38:30 +00:00
1b4e49a3f4
snapshot integrate GetEntityByID ( #2753 )
...
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add some APIs for SSDBImpl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* partially add GetVectorById
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* snapshot opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix typo
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update GetVectorByID framework
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename GetResFiles to GetResPath
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update GetVectorByIdSegmentHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add GetEntityByID
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update DataType
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update ParamField
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-07 02:23:04 +00:00
43496d5c0f
add background threads to SSDBImpl ( #2751 )
...
* add background threads to SSDBImpl
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* code format
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-07 01:45:58 +00:00
a49665fe85
(db/snapshot): add more APIs for Snapshot ( #2747 )
...
* (db/snapshot): Add more APIs for Snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Refactor field element related code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove some prints
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-06 11:08:07 +00:00
ef8171b90c
(db/snapshot): Bug fix in operations and unittest ( #2744 )
...
* (db/snapshot): fix bugs and update UT
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update Snapshot ToString
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add DropIndex API
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add DropIndex API UT
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-04 16:02:12 +00:00
36cb2479ba
Let codec singleton ( #2742 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-04 15:25:27 +00:00
62e437dc85
Fix compiling error
...
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-07-04 16:18:22 +08:00
aa549f0fba
* #2642 Create index failed caused by server crashed.
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* #2692 Milvus hangs during multi-thread concurrent search
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-04 15:02:05 +08:00
c594cf11c1
Increase GRPC concurrency ( #2671 )
...
* Increase GRPC concurrency
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix lint
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Add limitation
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix compilation error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix compilation error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix compilation error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-07-04 14:55:07 +08:00
3cbb5ceee9
typo
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-07-04 14:46:06 +08:00
5015557999
search combine configable
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-07-04 14:45:55 +08:00
3b898420d2
(db/snapshot): Add DropAllIndexOperation and related UT ( #2737 )
...
* (db/snapshot): add drop all index operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add ut for drop all index operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-04 04:09:21 +00:00
0672347775
Create index failed caused by server crashed ( #2736 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: groot <yhmo@zeronedata.com>
2020-07-04 01:25:48 +00:00
e8853b0681
(db/snapshot): Code Integration ( #2718 )
...
* (db/snapshot): integrate stage 1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 4
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 5
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 6
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 7
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add params for some resources
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add field type
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): implement iterate framework for Snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint errors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SnapshotHandlers
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SnapshotVistor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SizeField for all resource commit
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SegmentsToSearch collector
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Iterate handler update
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Update IterateHandler
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add some hooks for IterateHandler
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Small code refactor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add const for some snapshot APIs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add const for some snapshots APIs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Update create collection
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): merge to new ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint errors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add test_db in ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update test_ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add PartitionTest
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add DropIndexOperation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add IndexTest
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Fix some bugs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add RowCount for some resources
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): code refactor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix compile error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-03 20:35:58 +08:00
84c7701126
#2689 Construct Knowhere Index Without Data ( #2704 )
...
* Offset for IVFFlat
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* modify segment reader
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* add index_flat_nm
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix compilation issue
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix segment reader
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* Fix issue
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix hnsw_nm
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix nsg
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix bug
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* NSG
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix NSG issue
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* client test
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix index size
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* remove unnecessary things
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* add changelog
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix clang format
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* add changelog
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix compile error
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix compile error
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix compile error
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix compile error
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix issues
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Change data to shared_ptr
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix hnsw and nsg ut
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* clang-format
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* not to insert cache immediately
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* Fix macro issue
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-03 11:27:11 +00:00
f0abfa2c6c
Web fix 2690 ( #2691 )
...
* Remove body parser in show partitions endpoints (fix #2690 )
Signed-off-by: yhz <413554850@qq.com>
* Remove surplus output and comments
Signed-off-by: yhz <413554850@qq.com>
* Remove surplus output and comments
Signed-off-by: yhz <413554850@qq.com>
* Remove space line
Signed-off-by: yhz <413554850@qq.com>
* Remove surplus ut
Signed-off-by: yhz <413554850@qq.com>
* laten sleep time in case test_delete_vector_collection_count_no_flush
Signed-off-by: yhz <413554850@qq.com>
2020-07-02 15:34:37 +00:00
f7c1d0ad0d
snapshot integration ( #2716 )
...
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add some APIs for SSDBImpl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-02 13:09:49 +00:00
a5f6a7735c
fix a typo ( #2712 )
...
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-02 17:33:44 +08:00
a4b2c2b829
(db/snapshots): Integration and Enhancement ( #2673 )
...
* (db/snapshot): integrate stage 1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 4
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 5
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 6
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 7
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add params for some resources
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add field type
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): implement iterate framework for Snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint errors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SnapshotHandlers
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SnapshotVistor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SizeField for all resource commit
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SegmentsToSearch collector
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Iterate handler update
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Update IterateHandler
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add some hooks for IterateHandler
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Small code refactor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add const for some snapshot APIs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add const for some snapshots APIs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Update create collection
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): merge to new ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint errors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add test_db in ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update test_ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-01 11:45:34 +00:00
cfa5b14e73
add SSDBImpl and ssdb for ut ( #2683 )
...
* rename StatusField to StateField
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix typo
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* enable GC in OnNoRefCallBack
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add SSDBimpl and unittest/ssdb
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-29 10:08:25 +08:00
d7fff5084a
merge #2659 from 0.10.1 ( #2666 )
...
* merge #2659 from 0.10.1
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* merge #2657 from 0.10.1
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* format
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-29 02:07:47 +00:00
18998a4395
print out system memory size when report invalid cpu cache size ( #2679 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-28 16:18:32 +08:00
ba26b677da
snapshot gc ( #2668 )
...
* rename StatusField to StateField
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix typo
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* enable GC in OnNoRefCallBack
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-24 12:39:57 +00:00
403753a15b
* remove annoy search parameter validation ( #2667 )
...
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix test failed
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error again
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: op-hunter <ophunter52@gmail.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-24 16:52:05 +08:00
99fe8d96d7
Fix naming inconsistency ( #2664 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: Xiaohai Xu <xiaohai.xu@zilliz.com>
2020-06-24 04:10:30 +00:00
170209453f
snapshot gc ( #2662 )
...
* add Event.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add ResourceHelper and update event handler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update ResourceHelper
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add CollectionId for segment
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add CollectionId for SegmentFile
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update ResourceHelper
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* re-order Resources
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update ResourceHelper
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-24 01:46:43 +00:00
887b7f7f89
fix too many copies ( #2661 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-23 15:35:42 +00:00
d318b527de
Suit the range of HNSW parameters ( #2638 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-22 16:49:45 +08:00
39c7e8a4e8
(db/snapshot): Add SoftDeleteOperation and related test ( #2647 )
...
* (db/snapshot): add soft delete operation and related test
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): more readable error msg in snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix clang format
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix clang format
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-22 08:35:01 +00:00
5c833ca688
snapshot gc ( #2634 )
...
* opt test_snapshot
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename BaseHolders to ResourceHolder
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename DBBaseResource to BaseResource
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* remove ResourceHolder.inl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add GarbageCollector.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* combine ReferenceProxy.cpp into ReferenceProxy.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* reorg Resources.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename GarbageCollector to EventExecutor
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix test_snapshot
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt executor
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-21 02:40:08 +00:00
bcf7913414
Cherry-pick to master
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-20 20:35:54 +08:00
3cdff86852
Fix HNSW and RNSG index files size ( #2618 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-20 14:19:56 +08:00
f1a1ee8678
Snapshot improve ( #2608 )
...
* Format code style
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix bug && Format code style
* Format code style
* Format code style
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Update lint
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Update merge config
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix compiling error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-19 23:09:30 +08:00
b3e8856a34
move all APIs in utils into namespace milvus ( #2614 )
...
* move APIs in util into namespace milvus
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* combine BlockingQueue.h and src/utils/BlockingQueue.inl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 15:53:36 +08:00
2bc9c0cf25
snapshot code opt ( #2609 )
...
* opt test_snapshot
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename BaseHolders to ResourceHolder
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename DBBaseResource to BaseResource
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* remove ResourceHolder.inl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 11:35:25 +08:00
ecc52d365c
(db/snapshot): fix ut errors ( #2605 )
...
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-19 10:37:41 +08:00
523ed18c6e
Update version of Clang tools to v7.0 in ubuntu18.04 build environment ( #2610 )
...
* Update clang tools version to v7.0 in ubuntu18.04 build environment
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update clang tools version to v7.0 in ubuntu18.04 build environment
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update clang tools version to v7.0 in ubuntu18.04 build environment
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-06-19 00:11:01 +08:00
eb5564af03
fix Milvus docker image report illegal instruction ( #2598 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-18 10:57:40 +00:00
b6b56a0852
Support IVF_PQ on GPU with using metric_type IP ( #2593 )
...
* Support IVF_PQ on GPU with using metric_type IP
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* remove useless code
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-18 08:56:34 +00:00
a12ce5bf6b
(db/snapshot): update ut ( #2588 )
...
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-18 01:55:46 +00:00
19c7d5f71b
fix #2578 ( #2591 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-18 01:36:04 +00:00
4ce1e6b2fc
optimize unittest build ( #2589 )
...
* optimize test_storage
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize test_metrics
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize test_server
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optmize test_server
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize test_schedule
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* move test_rpc and test_web to db
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-17 17:22:54 +08:00
e267bd2c4b
CreateHybridIndex.cpp compile error ( #2583 )
...
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-17 14:03:13 +08:00
c131fbe3ae
fix random crash of INSERT_DUPLICATE_ID case ( #2557 )
...
* fix random crash of INSERT_DUPLICATE_ID case
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* cahnge ver
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* sdk readme
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix ut failure
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-17 03:50:19 +00:00
df8e64ea42
Structured index support ( #2573 )
...
* Add ValidateBooleanQuery check
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add test_hybrid_collection cases
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* CreatedStructuredIndex implementation
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add attr_type in DefaultAttrIndexFormat
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add CreateHybridIndex sdk
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix LoadAttr
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Reconstruct CreateStructuredIndex code
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix CreateStructuredIndex bugs
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix merge problems
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* add changelog
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix gpu version compile bug
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix RpcHandlerTest.HYBRID_TEST bug
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix test_web bug
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Remove hybrid test cases
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-17 02:00:39 +00:00
a9df83766a
clean util dependency ( #2576 )
...
* move part of APIs from utils/ValidationUtil to server/ValidationUtils
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* move part of APIs from utils/ValidationUtil to config/Utils
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* move part of APIs from utils/CommonUtil to config/Utils
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* move EraseFromCache from utils/CommonUtil to db/Utils
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* remove SignalHandler dependency with server
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update fiu
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-16 22:13:25 +08:00
aa31b0c251
(db/snapshot): fix bugs and enhance unit test ( #2562 )
...
* (db/snapshot): add ToString for Snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bugs and enhance CompoundTest1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bug in operations
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): some code refactor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): small change
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-16 15:05:55 +08:00
75c7a1d0fa
fix cache.cache_size range check error ( #2565 )
...
* fix cache.cache_size range check error
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update ci
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-06-15 23:37:16 +08:00
b26ce06463
remove secondary path ( #2560 )
...
* config code clean
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* remove slave_paths_ from db options
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update version
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* enable all db unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update some comments
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update comments
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-15 23:23:10 +08:00
7351b97850
change ver ( #2559 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-15 17:59:40 +08:00
95d970cf62
count up query statistics for debug ease ( #2528 )
...
* clean duplicated code in SearchTask
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update Search interface and add SearchJob::time_stat()
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-15 07:13:25 +00:00
d1c4ab9136
Mishards 010 upgrade & Add config unit in demo config ( #2550 )
...
* Update mishards config
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Update mishards config
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Upgrade mishards for milvus v0.10.0 (fix #2548 )
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Launch server fail using demo config (#2549 )
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* [skip ci] Remove surplus config yaml
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-06-15 14:00:47 +08:00
18539c50a3
[skip ci]update comments in config file ( #2542 )
...
* update comments in config file
Signed-off-by: wxyu <xy.wang@zilliz.com>
* Update config
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-15 14:00:39 +08:00
9d9a453a5b
Upgrade config ( #2511 )
...
* enable cron test on 0.10.0
Signed-off-by: zw <zw@milvus.io>
* fix merge result
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* import test_during_creating_index_restart
Signed-off-by: zw <zw@milvus.io>
* config cluster
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config general
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config network
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config storage
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config wal
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config cache
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config gpu and metrics
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config logs
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update server_config.template
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update changelog
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config with unit
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix clang-format
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update unittests
Signed-off-by: wxyu <xy.wang@zilliz.com>
* disable restart case
Signed-off-by: zw <zw@milvus.io>
* fix gpu compile failed
Signed-off-by: wxyu <xy.wang@zilliz.com>
* Fix lint and comments
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* fix wal unittest failed
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix some unittests
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update config test cases
Signed-off-by: zw <zw@milvus.io>
* Fix config error
Signed-off-by: yhz <413554850@qq.com>
* remove unused code
Signed-off-by: wxyu <xy.wang@zilliz.com>
* Update CHANGELOG
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* [skip-ci] fix test cases
Signed-off-by: zw <zw@milvus.io>
* fix config storage
Signed-off-by: wxyu <xy.wang@zilliz.com>
* make -j3 on ci
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update python test
Signed-off-by: wxyu <xy.wang@zilliz.com>
* remove unused code
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix ut
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix ut
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update status message
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix set config bug
Signed-off-by: wxyu <xy.wang@zilliz.com>
Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: yhz <413554850@qq.com>
2020-06-15 14:00:09 +08:00
2ee3b60e76
Fix test_hybrid_db and test_rpc error ( #2552 )
...
* Fix test_hybrid_db and test_rpc error
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* CI retry
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* CI retry
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-14 23:18:44 +08:00
5727cd139e
(db/snapshot): Fix bugs and more unit test ( #2538 )
...
* (db/snapshot): refactor CompoundBaseOperation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): some refactor for CompoundOperation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove one of Operations constructor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add DropPartition API and related test
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bugs for multi threads
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add some print
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): some bug fix
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): some bug fix 1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bugs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add build worker for unittest
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
(db/snapshot): update ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
(db/snapshot): update ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
(db/snapshot): update ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bug in ReferenceProxy
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-13 15:46:17 +08:00
c079804b81
Improve ut coverage ( #2516 ) ( #2522 )
...
* Improve ut coverage
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Delete unused code
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add fiu in HybridSearchRequest
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Update helm config
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Change BinaryQuery validation check
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* code format
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* code format
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* code format
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 15:40:45 +08:00
9fda27ff93
Improve ut coverage ( #2516 ) ( #2517 )
...
* Improve ut coverage (#2516 )
Signed-off-by: yhz <413554850@qq.com>
* Improve ut coverage
Signed-off-by: yhz <413554850@qq.com>
* CI retry
Signed-off-by: yhz <413554850@qq.com>
* Update helm config
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* CI retry
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* CI retry
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:20:11 +08:00
XuPeng-SH
XuanYang-cn
groot
cqy123456
Cai Yudong
BossZou
Xiaohai Xu
dwq
op-hunter
shengjun.li
JinHai-CN
groot
shengjun.li
quicksilver
yukun
Wang XiangYu