Commit Graph

1145 Commits (3b898420d23fdd953b11d3d1e53621a377708a2d)

Author SHA1 Message Date
XuPeng-SH 3b898420d2
(db/snapshot): Add DropAllIndexOperation and related UT (#2737)
* (db/snapshot): add drop all index operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add ut for drop all index operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-04 04:09:21 +00:00
shengjun.li 0672347775
Create index failed caused by server crashed (#2736)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: groot <yhmo@zeronedata.com>
2020-07-04 01:25:48 +00:00
XuPeng-SH e8853b0681
(db/snapshot): Code Integration (#2718)
* (db/snapshot): integrate stage 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 4

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 5

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 6

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 7

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add params for some resources

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add field type

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): implement iterate framework for Snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor code

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint errors

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SnapshotHandlers

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SnapshotVistor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SizeField for all resource commit

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SegmentsToSearch collector

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Iterate handler update

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Update IterateHandler

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add some hooks for IterateHandler

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Small code refactor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add const for some snapshot APIs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add const for some snapshots APIs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Update create collection

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): merge to new ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint errors

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add test_db in ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update test_ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add PartitionTest

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add DropIndexOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add IndexTest

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Fix some bugs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add RowCount for some resources

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): code refactor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix compile error

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-03 20:35:58 +08:00
Xiaohai Xu 84c7701126
#2689 Construct Knowhere Index Without Data (#2704)
* Offset for IVFFlat

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* modify segment reader

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* add index_flat_nm

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix compilation issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix segment reader

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* Fix issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix hnsw_nm

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix nsg

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix bug

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* NSG

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix NSG issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* client test

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix index size

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* remove unnecessary things

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix clang format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix issues

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Change data to shared_ptr

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix hnsw and nsg ut

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* clang-format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* not to insert cache immediately

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* Fix macro issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-03 11:27:11 +00:00
BossZou f0abfa2c6c
Web fix 2690 (#2691)
* Remove body parser in show partitions endpoints (fix #2690)

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus output and comments

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus output and comments

Signed-off-by: yhz <413554850@qq.com>

* Remove space line

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus ut

Signed-off-by: yhz <413554850@qq.com>

* laten sleep time in case test_delete_vector_collection_count_no_flush

Signed-off-by: yhz <413554850@qq.com>
2020-07-02 15:34:37 +00:00
Cai Yudong f7c1d0ad0d
snapshot integration (#2716)
* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add some APIs for SSDBImpl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-02 13:09:49 +00:00
op-hunter a5f6a7735c
fix a typo (#2712)
Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-02 17:33:44 +08:00
XuPeng-SH a4b2c2b829
(db/snapshots): Integration and Enhancement (#2673)
* (db/snapshot): integrate stage 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 4

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 5

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 6

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 7

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add params for some resources

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add field type

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): implement iterate framework for Snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor code

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint errors

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SnapshotHandlers

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SnapshotVistor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SizeField for all resource commit

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SegmentsToSearch collector

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Iterate handler update

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Update IterateHandler

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add some hooks for IterateHandler

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Small code refactor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add const for some snapshot APIs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add const for some snapshots APIs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Update create collection

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): merge to new ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint errors

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add test_db in ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update test_ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-01 11:45:34 +00:00
Cai Yudong cfa5b14e73
add SSDBImpl and ssdb for ut (#2683)
* rename StatusField to StateField

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix typo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable GC in OnNoRefCallBack

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add SSDBimpl and unittest/ssdb

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-29 10:08:25 +08:00
groot d7fff5084a
merge #2659 from 0.10.1 (#2666)
* merge #2659 from 0.10.1

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* merge #2657 from 0.10.1

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* format

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-29 02:07:47 +00:00
Cai Yudong 18998a4395
print out system memory size when report invalid cpu cache size (#2679)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-28 16:18:32 +08:00
Cai Yudong ba26b677da
snapshot gc (#2668)
* rename StatusField to StateField

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix typo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable GC in OnNoRefCallBack

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-24 12:39:57 +00:00
shengjun.li 403753a15b
* remove annoy search parameter validation (#2667)
Signed-off-by: cmli <chengming.li@zilliz.com>

* fix test failed

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error again

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: op-hunter <ophunter52@gmail.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-24 16:52:05 +08:00
shengjun.li 99fe8d96d7
Fix naming inconsistency (#2664)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: Xiaohai Xu <xiaohai.xu@zilliz.com>
2020-06-24 04:10:30 +00:00
Cai Yudong 170209453f
snapshot gc (#2662)
* add Event.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add ResourceHelper and update event handler

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ResourceHelper

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add CollectionId for segment

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add CollectionId for SegmentFile

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ResourceHelper

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* re-order Resources

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ResourceHelper

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-24 01:46:43 +00:00
shengjun.li 887b7f7f89
fix too many copies (#2661)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-23 15:35:42 +00:00
shengjun.li d318b527de Suit the range of HNSW parameters (#2638)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-22 16:49:45 +08:00
XuPeng-SH 39c7e8a4e8
(db/snapshot): Add SoftDeleteOperation and related test (#2647)
* (db/snapshot): add soft delete operation and related test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): more readable error msg in snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix clang format

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix clang format

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-22 08:35:01 +00:00
Cai Yudong 5c833ca688
snapshot gc (#2634)
* opt test_snapshot

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename BaseHolders to ResourceHolder

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename DBBaseResource to BaseResource

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove ResourceHolder.inl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add GarbageCollector.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* combine ReferenceProxy.cpp into ReferenceProxy.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* reorg Resources.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename GarbageCollector to EventExecutor

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_snapshot

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* opt executor

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-21 02:40:08 +00:00
groot bcf7913414 Cherry-pick to master
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-20 20:35:54 +08:00
shengjun.li 3cdff86852 Fix HNSW and RNSG index files size (#2618)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-20 14:19:56 +08:00
Jin Hai f1a1ee8678
Snapshot improve (#2608)
* Format code style

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix bug && Format code style

* Format code style

* Format code style

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update merge config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compiling error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-19 23:09:30 +08:00
Cai Yudong b3e8856a34
move all APIs in utils into namespace milvus (#2614)
* move APIs in util into namespace milvus

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* combine BlockingQueue.h and src/utils/BlockingQueue.inl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 15:53:36 +08:00
Cai Yudong 2bc9c0cf25
snapshot code opt (#2609)
* opt test_snapshot

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename BaseHolders to ResourceHolder

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename DBBaseResource to BaseResource

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove ResourceHolder.inl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-19 11:35:25 +08:00
XuPeng-SH ecc52d365c
(db/snapshot): fix ut errors (#2605)
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-19 10:37:41 +08:00
quicksilver 523ed18c6e
Update version of Clang tools to v7.0 in ubuntu18.04 build environment (#2610)
* Update clang tools version to v7.0 in ubuntu18.04 build environment

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update clang tools version to v7.0 in ubuntu18.04 build environment

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Update clang tools version to v7.0 in ubuntu18.04 build environment

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-06-19 00:11:01 +08:00
Cai Yudong eb5564af03
fix Milvus docker image report illegal instruction (#2598)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-18 10:57:40 +00:00
shengjun.li b6b56a0852
Support IVF_PQ on GPU with using metric_type IP (#2593)
* Support IVF_PQ on GPU with using metric_type IP

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* remove useless code

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-18 08:56:34 +00:00
XuPeng-SH a12ce5bf6b
(db/snapshot): update ut (#2588)
Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-18 01:55:46 +00:00
groot 19c7d5f71b
fix #2578 (#2591)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-18 01:36:04 +00:00
Cai Yudong 4ce1e6b2fc
optimize unittest build (#2589)
* optimize test_storage

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_metrics

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optmize test_server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize test_schedule

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move test_rpc and test_web to db

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-17 17:22:54 +08:00
yukun e267bd2c4b
CreateHybridIndex.cpp compile error (#2583)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-17 14:03:13 +08:00
groot c131fbe3ae
fix random crash of INSERT_DUPLICATE_ID case (#2557)
* fix random crash of INSERT_DUPLICATE_ID case

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* cahnge ver

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* sdk readme

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut failure

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-17 03:50:19 +00:00
yukun df8e64ea42
Structured index support (#2573)
* Add ValidateBooleanQuery check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add test_hybrid_collection cases

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* CreatedStructuredIndex implementation

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add attr_type in DefaultAttrIndexFormat

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add CreateHybridIndex sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix LoadAttr

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Reconstruct CreateStructuredIndex code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix CreateStructuredIndex bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix merge problems

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* add changelog

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix gpu version compile bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix RpcHandlerTest.HYBRID_TEST bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix test_web bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove hybrid test cases

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-17 02:00:39 +00:00
Cai Yudong a9df83766a
clean util dependency (#2576)
* move part of APIs from utils/ValidationUtil to server/ValidationUtils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move part of APIs from utils/ValidationUtil to config/Utils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move part of APIs from utils/CommonUtil to config/Utils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move EraseFromCache from utils/CommonUtil to db/Utils

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove SignalHandler dependency with server

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update fiu

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-16 22:13:25 +08:00
XuPeng-SH aa31b0c251
(db/snapshot): fix bugs and enhance unit test (#2562)
* (db/snapshot): add ToString for Snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bugs and enhance CompoundTest1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bug in operations

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): some code refactor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): small change

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-16 15:05:55 +08:00
Wang XiangYu 75c7a1d0fa fix cache.cache_size range check error (#2565)
* fix cache.cache_size range check error

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update ci

Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-06-15 23:37:16 +08:00
Cai Yudong b26ce06463
remove secondary path (#2560)
* config code clean

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove slave_paths_ from db options

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update version

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable all db unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update some comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-15 23:23:10 +08:00
groot 7351b97850
change ver (#2559)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-15 17:59:40 +08:00
Cai Yudong 95d970cf62
count up query statistics for debug ease (#2528)
* clean duplicated code in SearchTask

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update Search interface and add SearchJob::time_stat()

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-15 07:13:25 +00:00
BossZou d1c4ab9136 Mishards 010 upgrade & Add config unit in demo config (#2550)
* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Update mishards config

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Upgrade mishards for milvus v0.10.0 (fix #2548)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* Launch server fail using demo config (#2549)

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* [skip ci] Remove surplus config yaml

Signed-off-by: Yhz <yinghao.zou@zilliz.com>
2020-06-15 14:00:47 +08:00
Wang XiangYu 18539c50a3 [skip ci]update comments in config file (#2542)
* update comments in config file

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-15 14:00:39 +08:00
Wang XiangYu 9d9a453a5b Upgrade config (#2511)
* enable cron test on 0.10.0

Signed-off-by: zw <zw@milvus.io>

* fix merge result

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* import test_during_creating_index_restart

Signed-off-by: zw <zw@milvus.io>

* config cluster

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config general

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config network

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config wal

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config cache

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config gpu and metrics

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config logs

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update server_config.template

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update changelog

Signed-off-by: wxyu <xy.wang@zilliz.com>

* config with unit

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix clang-format

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* disable restart case

Signed-off-by: zw <zw@milvus.io>

* fix gpu compile failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Fix lint and comments

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* fix wal unittest failed

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix some unittests

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update config test cases

Signed-off-by: zw <zw@milvus.io>

* Fix config error

Signed-off-by: yhz <413554850@qq.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* Update CHANGELOG

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip-ci] fix test cases

Signed-off-by: zw <zw@milvus.io>

* fix config storage

Signed-off-by: wxyu <xy.wang@zilliz.com>

* make -j3 on ci

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update python test

Signed-off-by: wxyu <xy.wang@zilliz.com>

* remove unused code

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix ut

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update status message

Signed-off-by: wxyu <xy.wang@zilliz.com>

* update

Signed-off-by: wxyu <xy.wang@zilliz.com>

* fix set config bug

Signed-off-by: wxyu <xy.wang@zilliz.com>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
Co-authored-by: yhz <413554850@qq.com>
2020-06-15 14:00:09 +08:00
yukun 2ee3b60e76
Fix test_hybrid_db and test_rpc error (#2552)
* Fix test_hybrid_db and test_rpc error

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* CI retry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* CI retry

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-06-14 23:18:44 +08:00
XuPeng-SH 5727cd139e
(db/snapshot): Fix bugs and more unit test (#2538)
* (db/snapshot): refactor CompoundBaseOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): some refactor for CompoundOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): remove one of Operations constructor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add DropPartition API and related test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bugs for multi threads

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add some print

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): some bug fix

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): some bug fix 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bugs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add build worker for unittest

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

(db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

(db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

(db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bug in ReferenceProxy

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-13 15:46:17 +08:00
yukun c079804b81 Improve ut coverage (#2516) (#2522)
* Improve ut coverage

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Delete unused code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add fiu in HybridSearchRequest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Change BinaryQuery validation check

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 15:40:45 +08:00
BossZou 9fda27ff93 Improve ut coverage (#2516) (#2517)
* Improve ut coverage (#2516)

Signed-off-by: yhz <413554850@qq.com>

* Improve ut coverage

Signed-off-by: yhz <413554850@qq.com>

* CI retry

Signed-off-by: yhz <413554850@qq.com>

* Update helm config

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

* CI retry

Signed-off-by: Yhz <yinghao.zou@zilliz.com>

Co-authored-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:20:11 +08:00
Jin Hai f6928f7282 Add exception throw on mysql meta error (#2490)
* Add exception throw on mysql meta error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* [skip ci] update changelog

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Update

Signed-off-by: jinhai <hai.jin@zilliz.com>

* Fix Unit test

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-13 14:12:06 +08:00
groot e77a428885 return hnsw name (#2501)
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-06-13 14:11:50 +08:00
Cai Yudong 2f57d360eb fix performance (#2499)
* optimize sq_get_distance_computer

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_ref

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add sq_select_inverted_list_scanner_avx512

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize Codec

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* optimize ScalarQuantizerCodec_avx512.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update changelog

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean code

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-13 14:11:37 +08:00