Commit Graph

5703 Commits (cbba262442a42cc22ae24e00842d901b58321c97)

Author SHA1 Message Date
groot 5857f6401e
rewrite merge manager 1 (#2797)
* rewrite merge manager 1

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* rewrite merge manager 2

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-10 06:07:09 +00:00
op-hunter 7bf3e6e1c3
Hnsw sq8 support (#2799)
* modify for SQ8

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* add hnsw-sq8-support

Signed-off-by: cmli <chengming.li@zilliz.com>

* make clang-format and update changelog

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* hide unittest error temporary

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix ut error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-10 13:42:24 +08:00
op-hunter b221cd6898 fix a bug of getting wrong nprobe limitation in knowhere on GPU version (#2788)
* fix a bug of getting wrong nprobe limitation in knowhere on GPU version

Signed-off-by: cmli <chengming.li@zilliz.com>

* update changelog and fix build error

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-10 09:33:53 +08:00
shengjun.li a8bd2cf4a6 fix too many data copies (#2784)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-09 11:18:37 +08:00
groot e587c28644
rewrite mem manager 1 (#2762)
* rewrite mem manager

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* rewrite mem manager 2

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* rewrite mem manager 3

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* merge manager 1

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-09 03:12:44 +00:00
Cai Yudong 523d97af6d
snapshot integrate (#2782)
* update HybridQuery

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add IterateHandler

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* IterateHandler opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-08 15:08:08 +00:00
jinhai 71d36534df Update template
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-07-08 21:39:22 +08:00
XuPeng-SH af573f93c9
(db/snapshot): Add some helper visitors (#2777)
* (db/snapshot): add Segment related visitors

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint error

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-08 08:56:56 +00:00
Cai Yudong b7d9c2a4db
(db/snapshot) snapshot integrate (#2770)
* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add some APIs for SSDBImpl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* partially add GetVectorById

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* snapshot opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix typo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update GetVectorByID framework

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename GetResFiles to GetResPath

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update GetVectorByIdSegmentHandler

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add GetEntityByID

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update DataType

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ParamField

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update GetEntityByIDSegmentHandler

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support INT8 and INT16 in GetEntityById

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add GetIDsInSegment

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update FieldType

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add InsertEntities and DeleteEntities

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add HybridQuery

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-07 15:36:38 +00:00
XuPeng-SH 5964adef49
(db/snapshot): Implement operation rollback (#2760)
* (db/snapshot): refactor Operations

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor Operations

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add rollback for operations

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-07 11:38:30 +00:00
Snyk bot 17a01fe5eb
fix: upgrade com.alibaba:fastjson from 1.2.70 to 1.2.71 (#2756)
Snyk has created this PR to upgrade com.alibaba:fastjson from 1.2.70 to 1.2.71.

See this package in NPM:
https://www.npmjs.com/package/com.alibaba:fastjson

See this project in Snyk:
https://app.snyk.io/org/cxie/project/2f701ce6-b421-4bbf-aaad-9d2bdb8569e4?utm_source=github&utm_medium=upgrade-pr
2020-07-07 14:45:06 +08:00
Cai Yudong 1b4e49a3f4
snapshot integrate GetEntityByID (#2753)
* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add some APIs for SSDBImpl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* partially add GetVectorById

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* snapshot opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix typo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update GetVectorByID framework

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename GetResFiles to GetResPath

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update GetVectorByIdSegmentHandler

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add GetEntityByID

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update DataType

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ParamField

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-07 02:23:04 +00:00
groot 43496d5c0f
add background threads to SSDBImpl (#2751)
* add background threads to SSDBImpl

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* code format

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-07 01:45:58 +00:00
XuPeng-SH a49665fe85
(db/snapshot): add more APIs for Snapshot (#2747)
* (db/snapshot): Add more APIs for Snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Refactor field element related code

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint error

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): remove some prints

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-06 11:08:07 +00:00
XuPeng-SH ef8171b90c
(db/snapshot): Bug fix in operations and unittest (#2744)
* (db/snapshot): fix bugs and update UT

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update Snapshot ToString

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add DropIndex API

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add DropIndex API UT

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-04 16:02:12 +00:00
shengjun.li 36cb2479ba
Let codec singleton (#2742)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-04 15:25:27 +00:00
JinHai-CN 004c20118f [skip ci] Update PR template
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-07-04 16:33:31 +08:00
JinHai-CN 62e437dc85 Fix compiling error
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-07-04 16:18:22 +08:00
groot aa549f0fba * #2642 Create index failed caused by server crashed.
Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* #2692 Milvus hangs during multi-thread concurrent search

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-04 15:02:05 +08:00
del-zhenwu 04eb5a89f6 enable ivf_pq <ip> search cases on GPU (#2708)
* update cron on 0.10.1

Signed-off-by: zw <zw@milvus.io>

* enable ivf_pq <ip> search cases on GPU

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
2020-07-04 15:01:33 +08:00
groot 045089e5f3 * #2642 Create index failed caused by server crashed.
Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* changelog

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-04 15:00:52 +08:00
quicksilver 51679fd436 Update nightly cron (#2681)
* Change Milvus Helm Version to v0.10.1

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* [skip ci] update nightly cron

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* [skip ci] update nightly cron

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-07-04 14:58:33 +08:00
quicksilver 32d77d3fea [skip ci] update nightly cron (#2680)
* Change Milvus Helm Version to v0.10.1

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* [skip ci] update nightly cron

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-07-04 14:56:25 +08:00
quicksilver bffa1d7e39 Change Milvus Helm Version to v0.10.1 (#2678)
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-07-04 14:55:44 +08:00
Jin Hai c594cf11c1 Increase GRPC concurrency (#2671)
* Increase GRPC concurrency

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix lint

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Add limitation

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>

* Fix compilation error

Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-07-04 14:55:07 +08:00
groot 3cbb5ceee9 typo
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-07-04 14:46:06 +08:00
groot 5015557999 search combine configable
Signed-off-by: groot <yihua.mo@zilliz.com>
2020-07-04 14:45:55 +08:00
XuPeng-SH 3b898420d2
(db/snapshot): Add DropAllIndexOperation and related UT (#2737)
* (db/snapshot): add drop all index operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add ut for drop all index operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-04 04:09:21 +00:00
shengjun.li 0672347775
Create index failed caused by server crashed (#2736)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: groot <yhmo@zeronedata.com>
2020-07-04 01:25:48 +00:00
quicksilver e892626a1c
Add pipeline step to print/get the container logs (#2730)
* Add pipeline step to print/get the container logs

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Add pipeline step to print/get the container logs

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2020-07-03 20:36:28 +08:00
XuPeng-SH e8853b0681
(db/snapshot): Code Integration (#2718)
* (db/snapshot): integrate stage 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 4

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 5

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 6

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 7

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add params for some resources

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add field type

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): implement iterate framework for Snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor code

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint errors

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SnapshotHandlers

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SnapshotVistor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SizeField for all resource commit

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SegmentsToSearch collector

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Iterate handler update

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Update IterateHandler

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add some hooks for IterateHandler

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Small code refactor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add const for some snapshot APIs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add const for some snapshots APIs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Update create collection

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): merge to new ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint errors

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add test_db in ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update test_ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add PartitionTest

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add DropIndexOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add IndexTest

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Fix some bugs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add RowCount for some resources

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): code refactor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix compile error

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-03 20:35:58 +08:00
Xiaohai Xu 84c7701126
#2689 Construct Knowhere Index Without Data (#2704)
* Offset for IVFFlat

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* modify segment reader

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* add index_flat_nm

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix compilation issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix segment reader

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* Fix issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix hnsw_nm

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix nsg

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* fix bug

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* NSG

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix NSG issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* client test

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix index size

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* remove unnecessary things

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix clang format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* add changelog

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix compile error

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Fix issues

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* Change data to shared_ptr

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* fix hnsw and nsg ut

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* clang-format

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

* not to insert cache immediately

Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

* Fix macro issue

Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-07-03 11:27:11 +00:00
BossZou f0abfa2c6c
Web fix 2690 (#2691)
* Remove body parser in show partitions endpoints (fix #2690)

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus output and comments

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus output and comments

Signed-off-by: yhz <413554850@qq.com>

* Remove space line

Signed-off-by: yhz <413554850@qq.com>

* Remove surplus ut

Signed-off-by: yhz <413554850@qq.com>

* laten sleep time in case test_delete_vector_collection_count_no_flush

Signed-off-by: yhz <413554850@qq.com>
2020-07-02 15:34:37 +00:00
Cai Yudong f7c1d0ad0d
snapshot integration (#2716)
* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add some APIs for SSDBImpl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-02 13:09:49 +00:00
op-hunter a5f6a7735c
fix a typo (#2712)
Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
2020-07-02 17:33:44 +08:00
XuPeng-SH a4b2c2b829
(db/snapshots): Integration and Enhancement (#2673)
* (db/snapshot): integrate stage 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 4

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 5

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 6

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): integrate stage 7

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add params for some resources

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add field type

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): implement iterate framework for Snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor code

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint errors

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SnapshotHandlers

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SnapshotVistor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SizeField for all resource commit

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add SegmentsToSearch collector

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Iterate handler update

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Update IterateHandler

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add some hooks for IterateHandler

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Small code refactor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add const for some snapshot APIs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add const for some snapshots APIs

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Update create collection

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): merge to new ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint errors

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add test_db in ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update test_ssdb

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-01 11:45:34 +00:00
JinHai-CN c454acdeec [skip ci] Update README remove CN pages
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-07-01 16:45:55 +08:00
JinHai-CN a8b764bae0 [skip ci] Update roadmap link
Signed-off-by: JinHai-CN <hai.jin@zilliz.com>
2020-06-29 16:04:30 +08:00
Cai Yudong cfa5b14e73
add SSDBImpl and ssdb for ut (#2683)
* rename StatusField to StateField

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix typo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable GC in OnNoRefCallBack

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add SSDBimpl and unittest/ssdb

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-29 10:08:25 +08:00
groot d7fff5084a
merge #2659 from 0.10.1 (#2666)
* merge #2659 from 0.10.1

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* merge #2657 from 0.10.1

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* format

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-06-29 02:07:47 +00:00
Cai Yudong 18998a4395
print out system memory size when report invalid cpu cache size (#2679)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-28 16:18:32 +08:00
Cai Yudong ba26b677da
snapshot gc (#2668)
* rename StatusField to StateField

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix typo

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* enable GC in OnNoRefCallBack

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-24 12:39:57 +00:00
shengjun.li 403753a15b
* remove annoy search parameter validation (#2667)
Signed-off-by: cmli <chengming.li@zilliz.com>

* fix test failed

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error

Signed-off-by: cmli <chengming.li@zilliz.com>

* fix lint error again

Signed-off-by: cmli <chengming.li@zilliz.com>

Co-authored-by: cmli <chengming.li@zilliz.com>
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: op-hunter <ophunter52@gmail.com>
Co-authored-by: cmli <chengming.li@zilliz.com>
2020-06-24 16:52:05 +08:00
shengjun.li 99fe8d96d7
Fix naming inconsistency (#2664)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>

Co-authored-by: Xiaohai Xu <xiaohai.xu@zilliz.com>
2020-06-24 04:10:30 +00:00
Cai Yudong 170209453f
snapshot gc (#2662)
* add Event.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add ResourceHelper and update event handler

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ResourceHelper

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add CollectionId for segment

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add CollectionId for SegmentFile

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ResourceHelper

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* re-order Resources

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update ResourceHelper

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* retry ci

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-24 01:46:43 +00:00
shengjun.li 887b7f7f89
fix too many copies (#2661)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-23 15:35:42 +00:00
Snyk bot ecc7d839d8
fix: upgrade com.alibaba:fastjson from 1.2.68 to 1.2.70 (#2656)
Snyk has created this PR to upgrade com.alibaba:fastjson from 1.2.68 to 1.2.70.

See this package in NPM:
https://www.npmjs.com/package/com.alibaba:fastjson

See this project in Snyk:
https://app.snyk.io/org/cxie/project/2f701ce6-b421-4bbf-aaad-9d2bdb8569e4?utm_source=github&utm_medium=upgrade-pr
2020-06-23 12:32:22 +08:00
shengjun.li d318b527de Suit the range of HNSW parameters (#2638)
Signed-off-by: shengjun.li <shengjun.li@zilliz.com>
2020-06-22 16:49:45 +08:00
XuPeng-SH 39c7e8a4e8
(db/snapshot): Add SoftDeleteOperation and related test (#2647)
* (db/snapshot): add soft delete operation and related test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): more readable error msg in snapshot

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix clang format

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix clang format

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-06-22 08:35:01 +00:00
Cai Yudong 5c833ca688
snapshot gc (#2634)
* opt test_snapshot

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename BaseHolders to ResourceHolder

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename DBBaseResource to BaseResource

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove ResourceHolder.inl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add GarbageCollector.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* combine ReferenceProxy.cpp into ReferenceProxy.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* reorg Resources.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename GarbageCollector to EventExecutor

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix test_snapshot

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* opt executor

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-06-21 02:40:08 +00:00