777c76c87a
Fix rnsg master ( #2831 )
...
* fix rnsg ip
Signed-off-by: cqy <yaya645@126.com>
* fix rnsg search in master
Signed-off-by: cqy <yaya645@126.com>
2020-07-14 09:30:26 +08:00
e79cbf098e
(db/snapshot): update visitors ( #2837 )
...
* (db/snapshot): add more visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add some Iterators
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add event handler factory
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): process lsn
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): small change
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): precheck duplicate collection in create collection operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitor 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitor 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-13 11:08:30 +00:00
0d20f8a9ca
snapshot integrate ( #2830 )
...
* update HybridQuery
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add IterateHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* IterateHandler opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add SSSegmentReader
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add src/codecs/snapshot
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update latest code
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SSSegmentReader interface
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add SSSegmentWriter
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SSSegmentWriter interface
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest/db build fail
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* use SSSegmentReader and SSSegmentWriter in SSDBImpl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix build issue
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add test_segment
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-13 06:08:15 +00:00
6c5f42f9f1
(db/snapshot): LSN logic change and design EventHandlerFactory ( #2815 )
...
* (db/snapshot): add more visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add some Iterators
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add event handler factory
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): process lsn
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): small change
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): precheck duplicate collection in create collection operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-12 14:18:12 +00:00
141835a7c7
Add SDK support for IVFSQ8NR ( #2821 )
...
* add c++ sdk
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix clang
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-07-12 20:23:25 +08:00
2c3b3a5cfa
snapshot integrate ( #2814 )
...
* update HybridQuery
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add IterateHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* IterateHandler opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add SSSegmentReader
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add src/codecs/snapshot
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update latest code
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SSSegmentReader interface
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add SSSegmentWriter
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SSSegmentWriter interface
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest/db build fail
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* use SSSegmentReader and SSSegmentWriter in SSDBImpl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix build issue
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-11 13:26:31 +00:00
d004d273ad
add unittest 4 hnsw_sq8nr and fix some bug in passing ( #2819 )
...
* modify for SQ8
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* add hnsw-sq8-support
Signed-off-by: cmli <chengming.li@zilliz.com>
* make clang-format and update changelog
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error
Signed-off-by: cmli <chengming.li@zilliz.com>
* hide unittest error temporary
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix ut error
Signed-off-by: cmli <chengming.li@zilliz.com>
* add unittest 4 hnsw_sq8nr and fix several bugs
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-11 10:24:34 +00:00
6e16967b83
fix task_search ( #2808 )
...
* fix task_search
Signed-off-by: cqy <yaya645@126.com>
* fix unit test
Signed-off-by: cqy <yaya645@126.com>
2020-07-11 15:44:51 +08:00
fce41e738d
#2802 Add new index: IVFSQ8NR ( #2805 )
...
* support new index ivfsq8nr
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* remove timeout
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-07-11 13:40:14 +08:00
f94592ecdf
fix #2752 ( #2789 )
...
* fix #2752
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
(cherry picked from commit d7c47a7905
2020-07-11 11:18:05 +08:00
9d4febde7f
(db/snapshot): Add some visitors ( #2800 )
...
* (db/snapshot): add more visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add some Iterators
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-10 16:49:00 +08:00
5857f6401e
rewrite merge manager 1 ( #2797 )
...
* rewrite merge manager 1
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* rewrite merge manager 2
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-10 06:07:09 +00:00
7bf3e6e1c3
Hnsw sq8 support ( #2799 )
...
* modify for SQ8
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* add hnsw-sq8-support
Signed-off-by: cmli <chengming.li@zilliz.com>
* make clang-format and update changelog
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error
Signed-off-by: cmli <chengming.li@zilliz.com>
* hide unittest error temporary
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix ut error
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-10 13:42:24 +08:00
b221cd6898
fix a bug of getting wrong nprobe limitation in knowhere on GPU version ( #2788 )
...
* fix a bug of getting wrong nprobe limitation in knowhere on GPU version
Signed-off-by: cmli <chengming.li@zilliz.com>
* update changelog and fix build error
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-10 09:33:53 +08:00
a8bd2cf4a6
fix too many data copies ( #2784 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-09 11:18:37 +08:00
e587c28644
rewrite mem manager 1 ( #2762 )
...
* rewrite mem manager
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* rewrite mem manager 2
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* refine
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* rewrite mem manager 3
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* merge manager 1
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-09 03:12:44 +00:00
523d97af6d
snapshot integrate ( #2782 )
...
* update HybridQuery
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add IterateHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* IterateHandler opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-08 15:08:08 +00:00
af573f93c9
(db/snapshot): Add some helper visitors ( #2777 )
...
* (db/snapshot): add Segment related visitors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-08 08:56:56 +00:00
b7d9c2a4db
(db/snapshot) snapshot integrate ( #2770 )
...
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add some APIs for SSDBImpl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* partially add GetVectorById
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* snapshot opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix typo
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update GetVectorByID framework
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename GetResFiles to GetResPath
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update GetVectorByIdSegmentHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add GetEntityByID
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update DataType
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update ParamField
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update GetEntityByIDSegmentHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* support INT8 and INT16 in GetEntityById
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add GetIDsInSegment
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update FieldType
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add InsertEntities and DeleteEntities
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add HybridQuery
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-07 15:36:38 +00:00
5964adef49
(db/snapshot): Implement operation rollback ( #2760 )
...
* (db/snapshot): refactor Operations
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor Operations
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add rollback for operations
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-07 11:38:30 +00:00
1b4e49a3f4
snapshot integrate GetEntityByID ( #2753 )
...
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add some APIs for SSDBImpl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* partially add GetVectorById
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* snapshot opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix typo
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update GetVectorByID framework
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename GetResFiles to GetResPath
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update GetVectorByIdSegmentHandler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add GetEntityByID
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update DataType
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update ParamField
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-07 02:23:04 +00:00
43496d5c0f
add background threads to SSDBImpl ( #2751 )
...
* add background threads to SSDBImpl
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* code format
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-07 01:45:58 +00:00
a49665fe85
(db/snapshot): add more APIs for Snapshot ( #2747 )
...
* (db/snapshot): Add more APIs for Snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Refactor field element related code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove some prints
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-06 11:08:07 +00:00
ef8171b90c
(db/snapshot): Bug fix in operations and unittest ( #2744 )
...
* (db/snapshot): fix bugs and update UT
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update Snapshot ToString
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add DropIndex API
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add DropIndex API UT
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-04 16:02:12 +00:00
36cb2479ba
Let codec singleton ( #2742 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-04 15:25:27 +00:00
62e437dc85
Fix compiling error
...
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-07-04 16:18:22 +08:00
aa549f0fba
* #2642 Create index failed caused by server crashed.
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* #2692 Milvus hangs during multi-thread concurrent search
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-04 15:02:05 +08:00
c594cf11c1
Increase GRPC concurrency ( #2671 )
...
* Increase GRPC concurrency
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix lint
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Add limitation
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix compilation error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix compilation error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix compilation error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-07-04 14:55:07 +08:00
3cbb5ceee9
typo
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-07-04 14:46:06 +08:00
5015557999
search combine configable
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-07-04 14:45:55 +08:00
3b898420d2
(db/snapshot): Add DropAllIndexOperation and related UT ( #2737 )
...
* (db/snapshot): add drop all index operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add ut for drop all index operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-04 04:09:21 +00:00
0672347775
Create index failed caused by server crashed ( #2736 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: groot <yhmo@zeronedata.com>
2020-07-04 01:25:48 +00:00
e8853b0681
(db/snapshot): Code Integration ( #2718 )
...
* (db/snapshot): integrate stage 1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 4
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 5
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 6
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 7
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add params for some resources
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add field type
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): implement iterate framework for Snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint errors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SnapshotHandlers
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SnapshotVistor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SizeField for all resource commit
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SegmentsToSearch collector
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Iterate handler update
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Update IterateHandler
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add some hooks for IterateHandler
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Small code refactor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add const for some snapshot APIs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add const for some snapshots APIs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Update create collection
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): merge to new ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint errors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add test_db in ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update test_ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add PartitionTest
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add DropIndexOperation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add IndexTest
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Fix some bugs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add RowCount for some resources
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): code refactor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix compile error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-03 20:35:58 +08:00
84c7701126
#2689 Construct Knowhere Index Without Data ( #2704 )
...
* Offset for IVFFlat
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* modify segment reader
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* add index_flat_nm
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix compilation issue
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix segment reader
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* Fix issue
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix hnsw_nm
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix nsg
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix bug
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* NSG
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix NSG issue
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* client test
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix index size
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* remove unnecessary things
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* add changelog
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix clang format
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* add changelog
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix compile error
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix compile error
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix compile error
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix compile error
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Fix issues
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Change data to shared_ptr
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix hnsw and nsg ut
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* clang-format
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* not to insert cache immediately
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* Fix macro issue
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-03 11:27:11 +00:00
f0abfa2c6c
Web fix 2690 ( #2691 )
...
* Remove body parser in show partitions endpoints (fix #2690 )
Signed-off-by: yhz <413554850@qq.com>
* Remove surplus output and comments
Signed-off-by: yhz <413554850@qq.com>
* Remove surplus output and comments
Signed-off-by: yhz <413554850@qq.com>
* Remove space line
Signed-off-by: yhz <413554850@qq.com>
* Remove surplus ut
Signed-off-by: yhz <413554850@qq.com>
* laten sleep time in case test_delete_vector_collection_count_no_flush
Signed-off-by: yhz <413554850@qq.com>
2020-07-02 15:34:37 +00:00
f7c1d0ad0d
snapshot integration ( #2716 )
...
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add some APIs for SSDBImpl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-02 13:09:49 +00:00
a5f6a7735c
fix a typo ( #2712 )
...
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-02 17:33:44 +08:00
a4b2c2b829
(db/snapshots): Integration and Enhancement ( #2673 )
...
* (db/snapshot): integrate stage 1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 4
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 5
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 6
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): integrate stage 7
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add params for some resources
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add field type
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): implement iterate framework for Snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint errors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SnapshotHandlers
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SnapshotVistor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SizeField for all resource commit
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add SegmentsToSearch collector
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Iterate handler update
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Update IterateHandler
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add some hooks for IterateHandler
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Small code refactor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add const for some snapshot APIs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add const for some snapshots APIs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Update create collection
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): merge to new ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint errors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add test_db in ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update test_ssdb
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-01 11:45:34 +00:00
cfa5b14e73
add SSDBImpl and ssdb for ut ( #2683 )
...
* rename StatusField to StateField
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix typo
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* enable GC in OnNoRefCallBack
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add SSDBimpl and unittest/ssdb
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-29 10:08:25 +08:00
d7fff5084a
merge #2659 from 0.10.1 ( #2666 )
...
* merge #2659 from 0.10.1
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* merge #2657 from 0.10.1
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* format
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-29 02:07:47 +00:00
18998a4395
print out system memory size when report invalid cpu cache size ( #2679 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-28 16:18:32 +08:00
ba26b677da
snapshot gc ( #2668 )
...
* rename StatusField to StateField
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix typo
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* enable GC in OnNoRefCallBack
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-24 12:39:57 +00:00
403753a15b
* remove annoy search parameter validation ( #2667 )
...
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix test failed
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error again
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: op-hunter <ophunter52@gmail.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-24 16:52:05 +08:00
99fe8d96d7
Fix naming inconsistency ( #2664 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: Xiaohai Xu <xiaohai.xu@zilliz.com>
2020-06-24 04:10:30 +00:00
170209453f
snapshot gc ( #2662 )
...
* add Event.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add ResourceHelper and update event handler
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update ResourceHelper
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add CollectionId for segment
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add CollectionId for SegmentFile
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update ResourceHelper
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* re-order Resources
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update ResourceHelper
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-24 01:46:43 +00:00
887b7f7f89
fix too many copies ( #2661 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-23 15:35:42 +00:00
d318b527de
Suit the range of HNSW parameters ( #2638 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-22 16:49:45 +08:00
39c7e8a4e8
(db/snapshot): Add SoftDeleteOperation and related test ( #2647 )
...
* (db/snapshot): add soft delete operation and related test
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): more readable error msg in snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix clang format
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix clang format
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-22 08:35:01 +00:00
5c833ca688
snapshot gc ( #2634 )
...
* opt test_snapshot
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename BaseHolders to ResourceHolder
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename DBBaseResource to BaseResource
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* remove ResourceHolder.inl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add GarbageCollector.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* combine ReferenceProxy.cpp into ReferenceProxy.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* reorg Resources.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename GarbageCollector to EventExecutor
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix test_snapshot
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* opt executor
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-21 02:40:08 +00:00
bcf7913414
Cherry-pick to master
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-20 20:35:54 +08:00
3cdff86852
Fix HNSW and RNSG index files size ( #2618 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-20 14:19:56 +08:00
f1a1ee8678
Snapshot improve ( #2608 )
...
* Format code style
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix bug && Format code style
* Format code style
* Format code style
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Update lint
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Update merge config
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix compiling error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-19 23:09:30 +08:00
b3e8856a34
move all APIs in utils into namespace milvus ( #2614 )
...
* move APIs in util into namespace milvus
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* combine BlockingQueue.h and src/utils/BlockingQueue.inl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 15:53:36 +08:00
2bc9c0cf25
snapshot code opt ( #2609 )
...
* opt test_snapshot
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename BaseHolders to ResourceHolder
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* rename DBBaseResource to BaseResource
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* remove ResourceHolder.inl
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 11:35:25 +08:00
ecc52d365c
(db/snapshot): fix ut errors ( #2605 )
...
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-19 10:37:41 +08:00
523ed18c6e
Update version of Clang tools to v7.0 in ubuntu18.04 build environment ( #2610 )
...
* Update clang tools version to v7.0 in ubuntu18.04 build environment
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update clang tools version to v7.0 in ubuntu18.04 build environment
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update clang tools version to v7.0 in ubuntu18.04 build environment
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-06-19 00:11:01 +08:00
eb5564af03
fix Milvus docker image report illegal instruction ( #2598 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-18 10:57:40 +00:00
b6b56a0852
Support IVF_PQ on GPU with using metric_type IP ( #2593 )
...
* Support IVF_PQ on GPU with using metric_type IP
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* remove useless code
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-18 08:56:34 +00:00
a12ce5bf6b
(db/snapshot): update ut ( #2588 )
...
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-18 01:55:46 +00:00
19c7d5f71b
fix #2578 ( #2591 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-18 01:36:04 +00:00
4ce1e6b2fc
optimize unittest build ( #2589 )
...
* optimize test_storage
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize test_metrics
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize test_server
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optmize test_server
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize test_schedule
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* move test_rpc and test_web to db
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-17 17:22:54 +08:00
e267bd2c4b
CreateHybridIndex.cpp compile error ( #2583 )
...
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-17 14:03:13 +08:00
c131fbe3ae
fix random crash of INSERT_DUPLICATE_ID case ( #2557 )
...
* fix random crash of INSERT_DUPLICATE_ID case
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* cahnge ver
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* sdk readme
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix ut failure
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-17 03:50:19 +00:00
df8e64ea42
Structured index support ( #2573 )
...
* Add ValidateBooleanQuery check
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add test_hybrid_collection cases
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* CreatedStructuredIndex implementation
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add attr_type in DefaultAttrIndexFormat
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add CreateHybridIndex sdk
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix LoadAttr
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Reconstruct CreateStructuredIndex code
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix CreateStructuredIndex bugs
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix merge problems
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* add changelog
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix gpu version compile bug
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix RpcHandlerTest.HYBRID_TEST bug
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix test_web bug
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Remove hybrid test cases
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-17 02:00:39 +00:00
a9df83766a
clean util dependency ( #2576 )
...
* move part of APIs from utils/ValidationUtil to server/ValidationUtils
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* move part of APIs from utils/ValidationUtil to config/Utils
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* move part of APIs from utils/CommonUtil to config/Utils
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* move EraseFromCache from utils/CommonUtil to db/Utils
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* remove SignalHandler dependency with server
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update fiu
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-16 22:13:25 +08:00
aa31b0c251
(db/snapshot): fix bugs and enhance unit test ( #2562 )
...
* (db/snapshot): add ToString for Snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bugs and enhance CompoundTest1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bug in operations
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): some code refactor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): small change
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-16 15:05:55 +08:00
75c7a1d0fa
fix cache.cache_size range check error ( #2565 )
...
* fix cache.cache_size range check error
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update ci
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-06-15 23:37:16 +08:00
b26ce06463
remove secondary path ( #2560 )
...
* config code clean
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* remove slave_paths_ from db options
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update version
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* enable all db unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update some comments
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update comments
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-15 23:23:10 +08:00
7351b97850
change ver ( #2559 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-15 17:59:40 +08:00
95d970cf62
count up query statistics for debug ease ( #2528 )
...
* clean duplicated code in SearchTask
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update Search interface and add SearchJob::time_stat()
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-15 07:13:25 +00:00
d1c4ab9136
Mishards 010 upgrade & Add config unit in demo config ( #2550 )
...
* Update mishards config
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Update mishards config
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Upgrade mishards for milvus v0.10.0 (fix #2548 )
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Launch server fail using demo config (#2549 )
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* [skip ci] Remove surplus config yaml
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-06-15 14:00:47 +08:00
18539c50a3
[skip ci]update comments in config file ( #2542 )
...
* update comments in config file
Signed-off-by: wxyu <xy.wang@zilliz.com>
* Update config
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-15 14:00:39 +08:00
9d9a453a5b
Upgrade config ( #2511 )
...
* enable cron test on 0.10.0
Signed-off-by: zw <zw@milvus.io>
* fix merge result
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* import test_during_creating_index_restart
Signed-off-by: zw <zw@milvus.io>
* config cluster
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config general
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config network
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config storage
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config wal
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config cache
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config gpu and metrics
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config logs
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update server_config.template
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update changelog
Signed-off-by: wxyu <xy.wang@zilliz.com>
* config with unit
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix clang-format
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update unittests
Signed-off-by: wxyu <xy.wang@zilliz.com>
* disable restart case
Signed-off-by: zw <zw@milvus.io>
* fix gpu compile failed
Signed-off-by: wxyu <xy.wang@zilliz.com>
* Fix lint and comments
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* fix wal unittest failed
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix some unittests
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update config test cases
Signed-off-by: zw <zw@milvus.io>
* Fix config error
Signed-off-by: yhz <413554850@qq.com>
* remove unused code
Signed-off-by: wxyu <xy.wang@zilliz.com>
* Update CHANGELOG
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* [skip-ci] fix test cases
Signed-off-by: zw <zw@milvus.io>
* fix config storage
Signed-off-by: wxyu <xy.wang@zilliz.com>
* make -j3 on ci
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update python test
Signed-off-by: wxyu <xy.wang@zilliz.com>
* remove unused code
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix ut
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix ut
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update status message
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix set config bug
Signed-off-by: wxyu <xy.wang@zilliz.com>
Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: yhz <413554850@qq.com>
2020-06-15 14:00:09 +08:00
2ee3b60e76
Fix test_hybrid_db and test_rpc error ( #2552 )
...
* Fix test_hybrid_db and test_rpc error
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* CI retry
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* CI retry
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-14 23:18:44 +08:00
5727cd139e
(db/snapshot): Fix bugs and more unit test ( #2538 )
...
* (db/snapshot): refactor CompoundBaseOperation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): some refactor for CompoundOperation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove one of Operations constructor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add DropPartition API and related test
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bugs for multi threads
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add some print
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): some bug fix
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): some bug fix 1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bugs
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add build worker for unittest
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
(db/snapshot): update ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
(db/snapshot): update ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
(db/snapshot): update ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix bug in ReferenceProxy
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-13 15:46:17 +08:00
c079804b81
Improve ut coverage ( #2516 ) ( #2522 )
...
* Improve ut coverage
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Delete unused code
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add fiu in HybridSearchRequest
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Update helm config
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Change BinaryQuery validation check
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* code format
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* code format
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* code format
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 15:40:45 +08:00
9fda27ff93
Improve ut coverage ( #2516 ) ( #2517 )
...
* Improve ut coverage (#2516 )
Signed-off-by: yhz <413554850@qq.com>
* Improve ut coverage
Signed-off-by: yhz <413554850@qq.com>
* CI retry
Signed-off-by: yhz <413554850@qq.com>
* Update helm config
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* CI retry
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* CI retry
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:20:11 +08:00
f6928f7282
Add exception throw on mysql meta error ( #2490 )
...
* Add exception throw on mysql meta error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix lint
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* [skip ci] update changelog
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Update
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Update
Signed-off-by: jinhai <hai.jin@zilliz.com>
* Fix Unit test
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:12:06 +08:00
e77a428885
return hnsw name ( #2501 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-13 14:11:50 +08:00
2f57d360eb
fix performance ( #2499 )
...
* optimize sq_get_distance_computer
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add sq_select_inverted_list_scanner_ref
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add sq_select_inverted_list_scanner_avx
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add sq_select_inverted_list_scanner_avx512
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize Codec
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize ScalarQuantizerCodec_avx.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* code format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize ScalarQuantizerCodec_avx512.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* clean code
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-13 14:11:37 +08:00
5953f05015
default nlist 2048 ( #2498 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-13 14:11:29 +08:00
63ae619951
Fix 2495: Add more reason of creating lock file failed ( #2496 )
...
* Fix 2495: Add more reason of creating lock file failed
Signed-off-by: jinhai <hai.jin@zilliz.com>
* Fix 2495: Update changelog
Signed-off-by: jinhai <hai.jin@zilliz.com>
* Fix lint
Signed-off-by: jinhai <hai.jin@zilliz.com>
* Fix lint
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
* Fix compile error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:11:18 +08:00
2decf951e2
read_only node detect delete vector operation using mishards #2368 ( #2473 )
...
* Add new request ReLoadSegments
Signed-off-by: yhz <413554850@qq.com>
* Finish load segments functionality
Signed-off-by: yhz <413554850@qq.com>
* Add api in grpc
Signed-off-by: yhz <413554850@qq.com>
* update Reloadsegments
Signed-off-by: yhz <413554850@qq.com>
* .
Signed-off-by: yhz <413554850@qq.com>
* create new blacklist if not exists
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* update api names
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Finish mishard for support api reloadsegments
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* update changlog
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Add more details when failed in search task
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Fix compile issue
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* update mishards requirements
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Code format
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* modify docker images in mysql all_in_one
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* update shards code
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Move updatedeldocs function to dbimpl
Signed-off-by: yhz <413554850@qq.com>
* Move reload segment del docs function to dbimpl
Signed-off-by: yhz <413554850@qq.com>
* [skip ci] correct shards requirements
Signed-off-by: yhz <413554850@qq.com>
2020-06-13 14:10:07 +08:00
4f8c4cacda
fix merge result ( #2463 )
...
* fix merge result
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix tests
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-12 11:18:18 +08:00
f64e4e77f0
#2394 Drop collection timeout if too many partitions created on colle… ( #2477 )
...
* #2349 Drop collection timeout if too many partitions created on collection
Signed-off-by: groot <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-12 11:18:09 +08:00
b2b787bdb0
optimize the implemention of k-selection on faiss gpu version ( #2467 )
...
* optimize the implemention of k-selection on faiss gpu version
Signed-off-by: cmli <chengming.li@zilliz.com>
* update change log, re-classified the pr from feature to improvement
Signed-off-by: cmli <chengming.li@zilliz.com>
* update change log
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-12 11:17:50 +08:00
08c919156c
merge before index
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-11 17:03:35 +08:00
30d6f986bf
fix bug of openblas library install fail ( #2457 )
...
* fix bug of openblas library install fail
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix openblas install cmakelist error
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-11 17:03:25 +08:00
6e8050996b
Fix inconsistent reading and writing when using mishards (# 2367) ( #2442 )
...
* Fix inconsistent reading and writing when using mishards (# 2367)
Signed-off-by: yhz <413554850@qq.com>
* CI retry
Signed-off-by: yhz <413554850@qq.com>
2020-06-11 11:54:26 +08:00
96c68edcb6
fix search by GPU ( #2453 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-11 11:54:11 +08:00
8b627cacfd
improve knowhere coverage ( #2444 )
...
* increase nb for NSG code coverage
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* test more APIs in test_annoy
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* disable get_vector_by_id and search_by_id
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* improve code coverage
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* improve code coverage
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* install test_instructionset
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-11 11:53:52 +08:00
e2b23e9cf2
performance issues ( #2438 )
...
* performance issues
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix bugs
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* preload collection issue
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* uncomment boring log
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* reduce unittest time
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* reduce metric test time cost
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-11 11:53:36 +08:00
0067883c30
add debug APIs and update GPU temp memory size ( #2420 )
...
* add debug APIs and update GPU temp memory size
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update thread name
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update DESIGN.md
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-10 18:34:54 +08:00
0ea259c340
optimize merge process ( #2419 )
...
* optimize merge process
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
* refine code
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* drop collecion issue
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-10 15:07:51 +08:00
5eb5f6b0a0
midify Binary IVF in faiss 1.6.3 ( #2416 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-10 15:07:43 +08:00
3b15160a92
#2395 Fix large nq cudaMalloc ( #2409 )
...
* fix large nq cuda malloc issue
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* fix clang format
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Change log
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
2020-06-10 15:07:36 +08:00
166b83a098
Wal lsn issue ( #2412 )
...
* set wal lsn after all mem files serialized
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
* return status
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-10 15:07:29 +08:00
459ab0502d
#2366 Reduce SQL execution times ( #2383 )
...
* #2366
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix ut
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix python test
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix ut
Signed-off-by: groot <yihua.mo@zilliz.com>
* changelog
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-10 15:07:20 +08:00
47f99f3fb5
mysql meta connection pool issue ( #2413 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-10 15:06:28 +08:00
c007e446b9
logging build index progress ( #2411 )
...
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-06-10 15:06:06 +08:00
d67157235f
* remove log_config from code and script
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update script
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-10 15:05:35 +08:00
078100926f
#2403 MySQL max_idle_time is 10 by default ( #2405 )
...
* #2403 MySQL max_idle_time is 10 by default
Signed-off-by: jinhai <hai.jin@zilliz.com>
* #2403 MySQL max_idle_time is 10 by default
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-06-10 15:04:43 +08:00
42806de462
fix MatchNlist ( #2401 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-10 15:04:17 +08:00
2be4005d83
upgrade faiss 1.6.3 ( #2400 )
...
* roll back to original faiss 1.6.0
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update to faiss_1.6.3
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* patch all change to faiss 1.6.3
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* faiss CPU version build pass
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* faiss GPU version build pass
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-10 15:03:12 +08:00
01cab6f365
#2378 ( #2388 )
...
* return partition lsn
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix wal lsn
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix wal issue
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* all collection include partition
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix build error
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix flush
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-10 15:01:01 +08:00
2f9873987a
clean compile warning ( #2380 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-10 14:55:45 +08:00
295105ff40
Snapshot code update ( #2497 )
...
* (db/snapshot): Add LSN in Resource
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add AddStepWithLsn
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add Lsn related logic
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add operator << for Operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint errors
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add more test
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): operation rollback api placeholder
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): remove hard code status code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): support more printable dump for some operations
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add ToString for Context
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): more friendly context print
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): rename some apis
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-06 05:03:02 +00:00
530ee76a76
(db/snapshot): Add new features for Snapshot Impl ( #2483 )
...
* (db/snapshot): Add new features for Snapshot Impl
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update changelog
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Small changes to remove some dummy code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Small changes to remove some dummy code
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-03 03:55:40 +00:00
9b2fd331fc
Add CreateCollection and DropCollection for Snapshot ( #2454 )
...
* (db/snapshot): add merge opration test
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add UpdatedOnField
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add Utils.h
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor Snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor snapshot 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor snapshot 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor snapshot 4
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor snapshot 5
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor snapshot 6
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add CreateCollectionOperation and related test
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): changes for code style and clang format
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): improve ut coverage 1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): improve ut coverage 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): improve ut coverage 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add soft delete collection operation
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update drop collection 1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update drop collection 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update drop collection 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update drop collection 4
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix lint error
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-01 06:36:56 +00:00
47303d2551
QPS decrease ( #2372 )
...
* optimize merge strategy
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* #2365
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix typo
Signed-off-by: groot <yihua.mo@zilliz.com>
* optimize search
Signed-off-by: groot <yihua.mo@zilliz.com>
* code format
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-05-29 20:25:42 +08:00
1b8a06daef
* #2370 clean compile warning
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-29 20:24:52 +08:00
2fe4cc10de
Snapshot code optimization 1 ( #2440 )
...
* (db/snapshot): add merge opration test
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add UpdatedOnField
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add Utils.h
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor Snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor snapshot 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor snapshot 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor snapshot 4
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor snapshot 5
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor snapshot 6
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-05-28 02:25:14 +00:00
2264aab084
Add new hybrid search api ( #2445 )
...
* Add json-string-dsl hybrid search api
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add C++ sdk for json-string-dsl hybrid search
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add C++ examples for new hybrid search api
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add unittest for new hybrid search api
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-05-28 02:01:28 +00:00
8908cba476
optimize snapshot ( #2435 )
...
* optimize ResourceHolder
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* optimize OperationExecutor
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* roll back ResourceHolders
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* roll back BaseHolders
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-27 05:37:21 +00:00
098f5a534d
Structured index ( #2418 )
...
* Support Structured Index Based on Sort
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint errors and make clang-format
Signed-off-by: cmli <chengming.li@zilliz.com>
* pick some advices from laojin
Signed-off-by: cmli <chengming.li@zilliz.com>
* make clang-format
Signed-off-by: cmli <chengming.li@zilliz.com>
* rename n_ to size_, replace self-implemented lower_bound and upper_bound by std implemention as laojin requires
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix lint error
Signed-off-by: cmli <chengming.li@zilliz.com>
* update implementation by shengjun's advice
Signed-off-by: cmli <chengming.li@zilliz.com>
* remove index member size_
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-05-26 14:41:21 +08:00
96166e1a5b
Metadata Snapshot Implementation Phase I ( #2414 )
...
* (db/snapshot): migrate code from milvus_metatdata_design repo
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add license info
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add namespace snapshot
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add test_snapshot unit test
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add ScopedResourceTest
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): refactor
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): add ResourceHodlersTest
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): update ResourceHodlersTest
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add Operation Test and fix some bug
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Update Operation Test
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Update Operation Test to test build
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): Add Reset API for static instance
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix snapshot bug
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* (db/snapshot): fix dead lock bug
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* fix bug in concurrent mode
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* fix lint error in ut
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* fix lint error in snapshot 1
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* fix lint error in snapshot 2
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* fix lint error in snapshot 3
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* fix lint error in snapshot 4
Signed-off-by: peng.xu <peng.xu@zilliz.com>
* fix lint error in snapshot 5
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-05-25 10:50:34 +08:00
110d4d066a
TargetEntry implementation ( #2391 )
...
* Add GetEntitiesByID in DBImpl
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add operator overload functions in ConcurrentBitset
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add GetEntityByID interface
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Change format of Attributes
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Change hybrid search for new rules
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix C++ sdk for new format of hybrid interfaces
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix compile bugs
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix GetEntityByID
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix unittest bugs
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix WebControllerTest:test_hybrid bugs
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix field names bug in HybridSearch
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix HYBRID_SEARCH_TEST caused by const auto&
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add ConvertRowToColumnJson in WebRequestHandler
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Return target entry in WebServer
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Change ValidateBinaryQuery
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add GetEntityByID in WebServer
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* [skip ci]Removed unused code in C++ sdk
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-05-23 14:25:24 +08:00
c902b0fdb5
Caiyd 090 rename config ( #2340 )
...
* rename config items
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update config unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update config unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update log rotate default value
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update parameter usage in LogUtil.cpp (#3 )
Signed-off-by: wxyu <xy.wang@zilliz.com>
* retry ci
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix ut
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: Wang XiangYu <scsven@qq.com>
2020-05-15 20:54:34 +08:00
f6c03f47dc
[skip ci] Update http document for v0.9.0 ( #2347 ) ( #2348 )
...
Signed-off-by: yhz <413554850@qq.com>
2020-05-15 20:52:49 +08:00
3561014f5b
Change url for behavior 'get_entities_by_id' ( #2330 ) ( #2336 )
...
Signed-off-by: yhz <413554850@qq.com>
2020-05-15 20:52:15 +08:00
d5f7526cd5
#2338 ( #2345 )
...
* fix #2264
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix ut
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* Remove unnecessary memcpy
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* remove SearchByID from C++SDK
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* #2338
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-15 20:51:49 +08:00
299cd9cd94
Remove unnecessary memcpy ( #2337 )
...
* fix #2264
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix ut
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* Remove unnecessary memcpy
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-15 20:51:24 +08:00
b2967574ef
enable config unittest ( #2329 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-05-14 10:12:19 +08:00
5e777450b3
fix #2264 ( #2323 )
...
* fix #2264
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix ut
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-13 21:01:50 +08:00
5293607359
Caiyd support disable sptag ( #2308 )
...
* remove KNOWHERE_GPU_VERSION
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add MILVUS_SUPPORT_SPTAG
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix SPTAG build issue
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-13 16:52:48 +08:00
1b7aeaffdf
Fix server start failed when mode is 'read_only' ( #2294 )
...
* fix deploy bug
Signed-off-by: yhz <413554850@qq.com>
* fix unittest fail
Signed-off-by: yhz <413554850@qq.com>
* remove unused yml
Signed-off-by: yhz <413554850@qq.com>
* change to readonly
Signed-off-by: yhz <413554850@qq.com>
2020-05-12 09:28:10 +08:00
fb49e580c5
rewrite HasPartition ( #2289 )
...
* rewrite HasPartition
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* miss file
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* use layered merge
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* sdk HasPartition
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* rename sdk api
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix ut
Signed-off-by: groot <yihua.mo@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-05-12 09:27:58 +08:00
bc74d16376
#2283 Suspend the building tasks when any query command arrives ( #2285 )
...
* add builder suspend
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* IndexIVF builder check wait
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* Build suspend for all IVF
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* HNSW suspend
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* HNSW suspend
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* HNSW suspend
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* HNSW suspend
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* HNSW suspend
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>
* cpubuilder
Signed-off-by: wxyu <xy.wang@zilliz.com>
* add suspend check during annoy build index
Signed-off-by: cmli <chengming.li@zilliz.com>
* fix hnsw
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix changelog
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix clang format
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* suspend nsg
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: sahuang <xiaohai.xu@zilliz.com>
Co-authored-by: wxyu <xy.wang@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-05-12 09:27:45 +08:00
b6e16af8f8
update unittest config version ( #2299 )
...
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-05-11 22:00:21 +08:00
f32cba6111
Merge strategy ( #2277 )
...
* merge manager
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* fix build error
Signed-off-by: groot <yihua.mo@zilliz.com>
* layered merge strategy
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix ut
Signed-off-by: groot <yihua.mo@zilliz.com>
* use simple
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix parallel multi collections query crashissue
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-05-11 01:38:50 +00:00
369743c10d
Config upgrade ( #2281 )
...
* upgrade config to 0.4
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update milvus version in cmakelists.txt
Signed-off-by: wxyu <xy.wang@zilliz.com>
* disable make VERBOSE
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-05-10 22:55:46 +08:00
8d732656a7
Fix show partitions failed in http module ( fix #2228 ) ( #2234 )
...
* Fix show partitions failed in http module (fix #2228 )
Signed-off-by: yhz <413554850@qq.com>
* catch json string convert error
Signed-off-by: yhz <413554850@qq.com>
2020-05-10 12:49:26 +00:00
e63904867b
update debug log ( #2275 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-10 00:19:33 +08:00
a37f9b0764
Openblas macro ( #2269 )
...
* save local work 4 add openblas macro to chose compile openblas lib or not
Signed-off-by: cmli <chengming.li@zilliz.com>
* compile openblas by auto detect
Signed-off-by: cmli <chengming.li@zilliz.com>
* Update build environment docker images
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* update changelog
Signed-off-by: cmli <chengming.li@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* Update CMakeLists.txt
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
* set OpenBLAS_SOURCE auto in build.sh
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-05-09 22:47:03 +08:00
aeb9edf5c6
#2261 ( #2270 )
...
* has collection bug
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* #2261
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix typo
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-05-09 18:32:20 +08:00
2c04cc0afe
upgrade config to 0.4 ( #2273 )
...
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-05-09 18:28:03 +08:00
cf6be092ab
code clean ( #2265 )
...
* use macro STATUS_CHECK
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* comment S3 related config APIs
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix clang-format
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-09 01:51:04 +00:00
4cea320943
k-means L2 ( #2258 )
...
* k-means L2
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
* fix change log
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-05-08 14:03:50 +00:00
cf68c9918e
#1925 ( #2257 )
...
* write error
Signed-off-by: groot <yihua.mo@zilliz.com>
* out of storage
Signed-off-by: groot <yihua.mo@zilliz.com>
* clang format
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix ut
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix #1955
Signed-off-by: groot <yihua.mo@zilliz.com>
* refine code
Signed-off-by: groot <yihua.mo@zilliz.com>
* compact threashold
Signed-off-by: groot <yihua.mo@zilliz.com>
* changelog
Signed-off-by: groot <yihua.mo@zilliz.com>
* search by id for hnsw/pq/annoy
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix python test
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* file cleanup timeout config
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* #1925\
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-08 11:40:41 +00:00
1811254de6
Obtain running rpc requests information ( #2240 ) ( #2241 )
...
* add cmd to get requests
Signed-off-by: yhz <413554850@qq.com>
* forward class declaration
Signed-off-by: yhz <413554850@qq.com>
* Obtain running rpc requests information (fix #2240 )
Signed-off-by: yhz <413554850@qq.com>
* Fix unittest compile failed issue
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Log error with msg
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-05-08 11:17:48 +00:00
7b0998885c
Fix #2248 : use hostname and port as instance label of metrics ( #2249 )
...
Signed-off-by: caosiyang <csy3228@gmail.com>
2020-05-08 19:08:04 +08:00
104cda1a76
print warning on stdout ( #2243 )
...
* print warning on stdout
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix compile failed
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-05-08 08:57:08 +00:00
da55622042
file cleanup timeout config ( #2250 )
...
* write error
Signed-off-by: groot <yihua.mo@zilliz.com>
* out of storage
Signed-off-by: groot <yihua.mo@zilliz.com>
* clang format
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix ut
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix #1955
Signed-off-by: groot <yihua.mo@zilliz.com>
* refine code
Signed-off-by: groot <yihua.mo@zilliz.com>
* compact threashold
Signed-off-by: groot <yihua.mo@zilliz.com>
* changelog
Signed-off-by: groot <yihua.mo@zilliz.com>
* search by id for hnsw/pq/annoy
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix python test
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* file cleanup timeout config
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* typo
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* changelog
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-08 06:24:29 +00:00
ac69f1ab8d
add_dependencies in cmake ( #2239 )
...
* add_dependencies in cmake
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-05-07 14:51:38 +00:00
7e3ba4ddd2
Catch exception and print in stderr ( #2238 )
...
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-05-07 10:04:35 +00:00
1516d71a55
Add attributes to mysql and wal ( #2229 )
...
* Add hybrid operation into mysql
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add Hybrid into wal
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add unittest for hybrid wal
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix wal unittest bug
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix hybrid_insert unittest bugs
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix codacy/PR quality
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Change test_rpc port to 19531
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-05-07 06:14:58 +00:00
6de141b2af
rm libopenblsa-dev ( #2235 )
...
Signed-off-by: zhoubo <zhoubo@hotmail.com>
Co-authored-by: zhoubo <zhoubo@hotmail.com>
2020-05-06 21:46:33 +08:00
882ed76cda
[skip ci] remove 'openblas-devel' ( #2208 )
...
Signed-of-by: yaohong<dyhyfu@126.com>
2020-05-06 21:39:14 +08:00
f218217064
optimize similarity template ( #2227 )
...
* optimize similarity template
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-06 08:53:16 +00:00
ac8e9ff020
write error ( #2184 )
...
* write error
Signed-off-by: groot <yihua.mo@zilliz.com>
* out of storage
Signed-off-by: groot <yihua.mo@zilliz.com>
* clang format
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix ut
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix #1955
Signed-off-by: groot <yihua.mo@zilliz.com>
* refine code
Signed-off-by: groot <yihua.mo@zilliz.com>
* compact threashold
Signed-off-by: groot <yihua.mo@zilliz.com>
* changelog
Signed-off-by: groot <yihua.mo@zilliz.com>
* search by id for hnsw/pq/annoy
Signed-off-by: groot <yihua.mo@zilliz.com>
* fix python test
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-05-06 16:52:37 +08:00
82ab21aec3
Http update endpoints ( #2187 )
...
* replace table with collection in http module
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* add search_by_ids in http module
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* add API
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* add new API and unittest
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Update endpoint in http module(#2186 ); change vector id to string format(2185)
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* allow filter when show partition
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* move constant variables defination to .cpp
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-05-06 14:07:35 +08:00
86510aa295
check the return value of fscanf ( #2218 )
...
Signed-off-by: e <ych05zh@163.com>
2020-05-03 01:49:57 +00:00
ed2ee8873b
fix SingleIndexTest.IVFSQHybrid unittest ( #2215 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-05-02 10:24:35 +00:00
ff48427911
Log file rotating ( #2207 )
...
* log file rotating
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update changelog
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix compile failed
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-30 23:31:14 +08:00
0032c13445
Error out when index SQ8H run in CPU mode ( #2204 )
...
* Error out when index SQ8H run in CPU mode
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix error
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update cmake
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* code opt
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-30 15:22:29 +00:00
b458034ad5
Fix Fail to access WAL storage path ( #2196 ) ( #2198 )
...
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-30 09:47:12 +00:00
14b3e95960
add head file to fix two build warnings ( #2195 )
...
* add head file to fix two build warnings
Signed-off-by: zxf <zhuxiaofeng07936@hellobike.com>
* fstream.h => fstream
Signed-off-by: zxf <zhuxiaofeng07936@hellobike.com>
Co-authored-by: zxf <zhuxiaofeng07936@hellobike.com>
2020-04-30 06:10:44 +00:00
be2feb1531
Log with request id ( #2192 )
...
* add function to get request id
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update log in GrpcRequestHandler
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update request id format
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-30 13:39:55 +08:00
e77fd12286
Feisiyicl 2190 memory usage ( #2191 )
...
* rename MILVUS_ENABLE_PROFILING to ENABLE_CPU_PROFILING, and add ENABLE_MEM_PROFILING
Signed-off-by: feisiyicl <7764126@qq.com>
* optimize ReadOnlyArrayInvertedLists
Signed-off-by: feisiyicl <7764126@qq.com>
* update changelog
Signed-off-by: feisiyicl <7764126@qq.com>
* retry ci
Signed-off-by: feisiyicl <7764126@qq.com>
* fix build issue
Signed-off-by: feisiyicl <7764126@qq.com>
2020-04-30 02:03:54 +00:00
2c889636ab
Check Storage permission ( #2174 )
...
* add class storage checker
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* code format
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* code format
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* add cpu instruction sets check (#2055 )
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* add more details for storage
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Check storage path permission (fix #2173 )
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* Fix logs path access fail issue
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-29 17:23:32 +00:00
9b2368016c
#1946 fix load index file CPU2GPU fail during searching ( #2182 )
...
Signed-off-by: feisiyicl <7764126@qq.com>
2020-04-29 20:01:52 +08:00
a76e705a92
Using elkan K-Means to improve IVF ( #2180 )
...
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-04-29 19:58:50 +08:00
b3aa06fce1
add config simd_type ( #2179 )
...
* add config simd_type
Signed-off-by: feisiyicl <7764126@qq.com>
* update changelog
Signed-off-by: feisiyicl <7764126@qq.com>
2020-04-29 17:46:36 +08:00
0a2b77c363
Upgrade http thirdparty oatpp to v1.0.0 ( fix #2153 ) ( #2157 )
...
* Upgrade thirdparty oatpp to v1.0.0 (fix #2153 )
Signed-off-by: yhz <413554850@qq.com>
* adjust header order
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-28 15:57:51 +00:00
ebb01aa9b2
Fix failed to open file ( #2138 )
...
* file reference
Signed-off-by: groot <yihua.mo@zilliz.com>
* print info
Signed-off-by: groot <yihua.mo@zilliz.com>
* avoid metric crash
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* refine code
Signed-off-by: yhmo <yihua.mo@zilliz.com>
* apply delete bug
Signed-off-by: groot <yihua.mo@zilliz.com>
* has partition check
Signed-off-by: groot <yihua.mo@zilliz.com>
* duplicate id search
Signed-off-by: groot <yihua.mo@zilliz.com>
* changelog
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-28 21:10:01 +08:00
3be8aad1af
Merge config file ( #2168 )
...
* Using el::Configurations Class init easylog
Signed-off-by: wxyu <xy.wang@zilliz.com>
* add logs config constant
Signed-off-by: wxyu <xy.wang@zilliz.com>
* add config check function
Signed-off-by: wxyu <xy.wang@zilliz.com>
* add config get function
Signed-off-by: wxyu <xy.wang@zilliz.com>
* logs config set function
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update InitLog function
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix clang-format
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update server_config.template
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update changelog
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-28 20:21:21 +08:00
4ab92f3032
Add config to switch off web server ( #2137 )
...
* Add a config parameter to switch off http server (fix #2057 )
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* modify web enable config name
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* fix cmake bug & output exanption msg
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* rename web enbale config as web_enable
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* optimize code
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* log out errno
Signed-off-by: yhz <413554850@qq.com>
2020-04-28 17:04:04 +08:00
1b25bacee2
build openblas by source ( #2154 )
...
* build openblas by source
Signed-off-by: cmli <chengming.li@zilliz.com>
* remove openblas md5
Signed-off-by: cmli <chengming.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-27 22:48:14 +08:00
b65f5072f3
fix server start failed if wal directory exist ( #2144 )
...
* fix server start failed if wal directory exist
Signed-off-by: wxyu <xy.wang@zilliz.com>
* add wal_enable check
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix compile error
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-27 20:05:04 +08:00
bf42dbd5b6
Gpu check ( #2129 )
...
* add GpuCheck class
Signed-off-by: yhz <413554850@qq.com>
* Check gpu environment
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
* add server checker unittest
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-27 16:56:12 +08:00
1f0466708f
merge server_cpu_config.template and server_gpu_config.template ( #2150 )
...
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-04-27 16:27:44 +08:00
b6d681f458
Merge pull request #2139 from longer-is-better/patch-1
...
delete unused file
2020-04-27 16:17:15 +08:00
2a7e116a67
remove uint64_t j from parameters ( #2147 )
...
Signed-off-by: talentAN <adam_an02@163.com>
2020-04-27 16:15:49 +08:00
b3028baed8
Add log for error status ( #2148 )
...
Signed-off-by: talentAN <adam_an02@163.com>
2020-04-27 16:13:11 +08:00
1384484b7d
Caiyd 2039 avx sse ( #2136 )
...
* #2039 clean redundant compile options
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* #2039 add distances_avx.h and distances_avx.cpp
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* #2039 add ScalarQuantizerDC_avx.cpp, ScalarQuantizerDC_avx.h and ScalarQuantizerCodec_avx.h
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* #2039 remove -mavx2 from default compile option
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update changelog
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* #2039 clean code
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* #2039 clean code
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-27 15:01:47 +08:00
509b87f103
replace int with size_t ( #2140 )
...
Signed-off-by: Gracieeea <sly_freesoul@icloud.com>
2020-04-27 04:52:11 +00:00
7b1b75a90d
Fix lint error
...
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-27 10:26:05 +08:00
adc6dc38bf
Fix compiling error
...
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-27 10:25:07 +08:00
2c6fcacdbd
delete unused file
...
Signed-off-by: dongwei <d279617552@gmail.com>
2020-04-26 23:21:06 +08:00
150cca6bbf
Caiyd 1946 cpu2gpu getres ( #2124 )
...
* #1946 use new log format in knowhere
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* #1946 add more debug info for GPU device initialize
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry CI
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* retry CI
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-04-26 18:29:21 +08:00
12b3bf0693
Add instance lock ( #2060 )
...
* Add instance lock
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update message
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update unittest CMakeLists.txt
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
* fix clang-format
Signed-off-by: wxyu <xy.wang@zilliz.com>
* update
Signed-off-by: wxyu <xy.wang@zilliz.com>
Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-04-26 08:31:18 +00:00
7ecafbd93a
rename the variable name collection_schema ( #2126 )
...
Signed-off-by: DanielHuang1983 <huanghe888@hotmail.com>
2020-04-26 08:03:52 +00:00
53133aafac
train func add catch excetpion ( #2096 )
...
* train func add catch excetpion
Signed-off-by: shana0325 <645842705@qq.com>
* Fix lint error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-26 11:29:20 +08:00
d22f3c2c5f
[skip ci] remove useless log ( #2119 )
...
Signed-off-by: yiuluchen 1093633856@qq.com
2020-04-26 10:19:31 +08:00
8edb7b6a09
add cpu instruction sets check ( #2055 )
...
Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-04-26 00:59:25 +08:00
777c36a0ac
#1997 index file missed after compact ( #2108 )
...
* #1997 index file missed after compact
Signed-off-by: groot <yihua.mo@zilliz.com>
* changelog
Signed-off-by: groot <yihua.mo@zilliz.com>
* typo
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-04-25 22:06:26 +08:00
9db19a9a20
Merge remote-tracking branch 'main/master'
2020-04-25 18:58:59 +08:00
fed865c7c9
Values of type 'const size_t' may not fit into the receiver type 'int' ( #2109 )
...
Signed-off-by: GuoRentong <rt.guo@outlook.com>
Co-authored-by: guo rentong <grt@guodeMacBook-Pro.local>
2020-04-25 18:35:04 +08:00
4c108c7578
Rename MemMenagerFactory.cpp/h to MemManagerFactory.cpp/h ( #2093 )
...
* Rename MemMenagerFactory.cpp to MemManagerFactory.cpp
Signed-off-by: yxm1536 <62009483+yxm1536@users.noreply.github.com>
* Rename MemMenagerFactory.h to MemManagerFactory.h
Signed-off-by: yxm1536 <62009483+yxm1536@users.noreply.github.com>
* Fix compilation error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-04-25 16:57:14 +08:00
b454f462c1
put error when call Flush ( #2097 )
...
Signed-off-by: beautifulpython <2426009680@qq.com>
2020-04-25 16:13:32 +08:00
6fe2cf1580
Update ConfAdapter.cpp ( #2106 )
...
Signed-off-by: grtoverflow <grtoverflow@gmail.com>
2020-04-25 16:12:58 +08:00
42b1983625
Delete useless boolean ( #2107 )
...
Signed-off-by: xge-zilliz <xi.ge@zilliz.com>
2020-04-25 15:49:43 +08:00
c33df2ee20
Use emplace_back instead of push_back ( #2103 )
...
Signed-off-by: dd-He <1843757857@qq.com>
2020-04-25 15:37:23 +08:00
6d2c50ac21
Use std::copy to replace forloop assignment ( #2087 )
...
Signed-off-by: chengpu <chengpu@live.com>
2020-04-25 15:28:31 +08:00
2247d58cc4
Unused variable: ids ( #2088 )
...
Signed-off-by: binbin12580 <1565196967@qq.com>
2020-04-25 15:26:16 +08:00
7deef8029f
Update GrpcRequestHandler.cpp ( #2085 )
...
Signed-off-by: Cross <xinyuanm9@gmail.com>
2020-04-25 14:53:49 +08:00
82f4c289eb
New api HasPartition, SearchByID, GetVectorsByID ( #2056 )
...
* modify proto
Signed-off-by: groot <yihua.mo@zilliz.com>
* modify c sdk
Signed-off-by: groot <yihua.mo@zilliz.com>
* rewrite search by id
Signed-off-by: groot <yihua.mo@zilliz.com>
* change cases
Signed-off-by: zw <zw@zilliz.com>
* fix cases
Signed-off-by: zw <zw@zilliz.com>
* fix wal case
Signed-off-by: zw <zw@zilliz.com>
* fix search by id
Signed-off-by: groot <yihua.mo@zilliz.com>
* modify sdk
Signed-off-by: groot <yihua.mo@zilliz.com>
* refine code
Signed-off-by: groot <yihua.mo@zilliz.com>
* refine code
Signed-off-by: groot <yihua.mo@zilliz.com>
* changelog
Signed-off-by: groot <yihua.mo@zilliz.com>
* add search_by_ids case
Signed-off-by: zw <zw@zilliz.com>
* disable test_search)by_ids
Signed-off-by: zw <zw@zilliz.com>
Co-authored-by: zw <zw@zilliz.com>
Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com>
2020-04-25 14:49:32 +08:00
7e52815fb6
DBImpl.cpp::ExecWalRecord need to process error status in case wal::MXLogType::Entity ( #2083 )
...
Signed-off-by: lwglgy <1486111482@qq.com>
2020-04-25 13:24:24 +08:00
67d8a9b54c
Add http interface for hybrid search ( #2079 )
...
* Add http interface for hybrid search
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add unittest for http hybrid interface
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* clang format
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Fix codacy quality
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-04-25 13:19:17 +08:00
6d87dc3c72
Merge pull request #2084 from lee-eve/master
...
Unused variable: offsets
2020-04-25 13:18:19 +08:00
cqy123456
XuPeng-SH
Cai Yudong
Xiaohai Xu
op-hunter
groot
shengjun.li
JinHai-CN
groot
BossZou
shengjun.li
quicksilver
yukun
Wang XiangYu
Wang XiangYu
caosiyang
zhoubo0317
dyhyfu
jackyu2020
zxf2017
feisiyicl
AzAz
Gracieeea
dongwei
DanielHuang1983
shana0325
chen lu
GuoRentong
yxm1536
czp
grtoverflow
xge-zilliz
dd-He
chengpu
binbin12580
Cross
lwglgy