Commit Graph

5728 Commits (22ea1e34231a5ebb87b2903fcf0f9e73d6eed2d9)

Author SHA1 Message Date
del-zhenwu 840d9cfcf0
[skip ci] remove ip_collection (#3122)
* [skip ci] remove ip_collection

Signed-off-by: zw <zw@milvus.io>

* [skip ci] move index params

Signed-off-by: zw <zw@milvus.io>

* [skip ci] move metric_type outside

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-04 17:53:00 +08:00
BossZou 9aa364ce25
Optimize DeletedDocsFormat class file I/O (#3112)
* Optimize DeletedDocsFormat file I/O

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Update DeletedDocsFormat Read

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-04 15:45:44 +08:00
yukun 96d799655a
Add DBImpl::Query unittest (#3119)
* Fix Block Format Read bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix Search crash bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix CreateCollection bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add db->Query unittest

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-04 15:00:15 +08:00
groot 1f705a2096
define binary data type (#3115)
* define binary data type

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-04 14:58:18 +08:00
Xiaohai Xu 19c5dc079f
add logging (#3092)
Signed-off-by: sahuang <xiaohai.xu@zilliz.com>

Co-authored-by: shengjun.li <shengjun.li@zilliz.com>
2020-08-04 12:09:15 +08:00
groot 8f8fa0aaf2
use index type replace name (#3114)
* use index type replace name

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-08-04 10:45:41 +08:00
XuanYang-cn 6ecf7a276e
fix segment dependency sqlite (#3113)
* fix segment dependency

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add dependence mysqlpp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* quick fix

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-08-03 18:07:01 +08:00
del-zhenwu 769aa2279b
[skip ci] add some dsl cases (#3111)
* [skip ci] add some dsl cases

Signed-off-by: zw <zw@milvus.io>

* [skip ci] add some dsl cases

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-03 15:48:18 +08:00
chen qingxiang 62841c68d5
fix bug caused by wrong valid row initlization (#3108)
* fix bug caused by wrong valid row initlization

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* fix bug caused by wrong valid row initlization

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

Co-authored-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-03 15:47:48 +08:00
Wang Xiangyu 1232011de7
[skip ci]copy dablooms' license file from origin repository (#3110)
Signed-off-by: Wang Xiangyu <xy.wang@zilliz.com>
2020-08-03 15:44:40 +08:00
del-zhenwu 5d66f1e60a
[skip ci] add some dsl cases (#3109)
Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-03 14:30:13 +08:00
yukun 7e1fd3d1e8
Fix bug (#3106)
* Fix Block Format Read bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix Search crash bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix CreateCollection bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-03 11:59:02 +08:00
XuanYang-cn 28af7eb4a7
Rearrange db/CMKLists, seperate dir codecs, storage, cache and segment (#3105)
* change cmake minmum version requires to 3.14

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* apart server/CMKLists

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* apart cache

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* apart storage

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* apart codecs

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* apart segment

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* rm thirdparty dablooms from db/cmklists

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add comments about thirdparty lib

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add license

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-03 11:58:35 +08:00
BossZou f1d271c0e9
(db/snapshot) Optimize mysql (#3095)
* Move MysqlPool to backend

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Update header

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Optimize mysqlpool

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-03 11:58:08 +08:00
groot fd2e3de65d
fix build error (#3104)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-03 11:20:44 +08:00
yukun 1c54baeec3
Fix bug (#3099)
* Fix Block Format Read bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix Search crash bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-08-03 10:41:48 +08:00
XuPeng-SH dbff431d2a
(db/snapshot): Add new TypeNameField and update some ut (#3100)
* (db/snapshot): Add TypeNameField

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add test for DropAllIndexOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

Co-authored-by: Jin Hai <hai.jin@zilliz.com>
2020-08-03 09:54:55 +08:00
BossZou 69f4f4d54b
(db/snapshot) implement Delete (#3101)
* Update API DeleteEntity in MemMgr

Signed-off-by: yhz <413554850@qq.com>

* Stop flush automaticlly

Signed-off-by: yhz <413554850@qq.com>

* Doing ApplyDeletes

Signed-off-by: yhz <413554850@qq.com>

* Mysql Schema formt

Signed-off-by: yhz <413554850@qq.com>

* Delete code done

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Delete run pass

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>

* Allow auto flush

Signed-off-by: yinghao.zou <yinghao.zou@zilliz.com>
2020-08-03 09:51:37 +08:00
chen qingxiang 3f02104265
fix bugs when GetEntity with ID in mulity paritions (#3098)
* fix bugs when GetEntity with ID in mulity paritions

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* cancel annotation of index thread

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2020-08-01 18:32:55 +08:00
Cai Yudong 4024e5cc03
fix ci (#3097)
* rename CollectionMappings to FieldElementMappings

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code opt

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-08-01 17:03:05 +08:00
del-zhenwu c9d7d54df5
[skip ci] update default field name (#3096)
* [skip ci] update default field name

Signed-off-by: zw <zw@milvus.io>

* [skip ci] update field name

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
2020-08-01 15:29:00 +08:00
Wang XiangYu 9704db4c40
fix compile error (#3094)
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-08-01 12:02:38 +08:00
XuPeng-SH 4e143775de
(db/snapshot): Update DropAllIndexOperation and some small changes (#3087)
* (db/snapshot): update for row count

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bug in NewSegmentOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): remove dummy print

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add some test for row count

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update size logic

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update size logic related ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): rollback if operation is not done

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): clean store

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): remove some dependency

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update for store

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update Store.h

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update store related code

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add field element modification operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): change new operation name

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint error

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add Segment File Operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): crtp for BaseResource

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add InActiveResourcesGCEvent

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix ut error

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): small change

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update snapshot segmentcommit operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update drop all index operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint error

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix gc

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix gc segment files

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor Event related

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): change 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): change for GC event

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix build error for high version of boost filesystem

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): small change

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update compound operations

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add operation test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update for CompoundSegmentsOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add test for CompoundSegmentsOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add time stat in operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): timeout handling

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): timeout handling 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): timeout handling 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): small refactor

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update for DropAllIndexOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-01 11:06:57 +08:00
XuanYang-cn a85660bef8
Apart src/CMKLists to dir server/CMKLists (#3081)
* change cmake minmum version requires to 3.14

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* apart server/CMKLists

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-01 10:55:40 +08:00
yukun d0b3c74f9a
Fix Block Format Read bug (#3084)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-08-01 10:55:17 +08:00
del-zhenwu 64d748ff4d
[skip ci] remove index name && add metric type in index and search params (#3088)
Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-01 10:35:05 +08:00
groot 5c37a9faa3
refine insert code (#3090)
* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine insert code

Signed-off-by: groot <yihua.mo@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-01 10:17:32 +08:00
jinhai 6f441fee5a [skip ci] Update ISSUE template
Signed-off-by: jinhai <hai.jin@zilliz.com>
2020-08-01 10:15:11 +08:00
chen qingxiang 9d464bde1d
fix get entity by id bug (#3086)
* fix get entity by id bug

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* cancel annotation of index thread

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

* format code

Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-08-01 09:45:51 +08:00
groot f2a22a09e3
refine code (#3089)
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-08-01 09:10:46 +08:00
groot 36a1dea3b1
describe index request (#3085)
* describe index

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* build error

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-31 17:48:57 +08:00
Cai Yudong e426b18d0e
fix ci (#3080)
* rename partition name to tag

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move BaseReq/ReqType to Types.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* opt BaseReq

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-07-31 15:48:19 +08:00
Wang XiangYu 4e6cf5369b
[skip ci]change case level in delete_test (#3083)
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-07-31 15:43:28 +08:00
Wang XiangYu 81d09cd550
[skip ci]change case level in list_id_in_segment_test (#3082)
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-07-31 15:26:39 +08:00
groot d0543d21e3
fix bug (#3078)
* fix bug

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-31 14:21:06 +08:00
Wang XiangYu 717d0d7bae
[skip ci]change some case's level (#3079)
Signed-off-by: wxyu <xy.wang@zilliz.com>
2020-07-31 12:02:32 +08:00
yukun feb29256dc
Fix GetEntityByID bug (#3073)
* Add search implementation in ExecutionEngine

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove attr_type from query_ptr

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add CreateStructuredIndex in ExecutionEngine

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add index fields in QueryPtr

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix SearchReq bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add Search in scheduler

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix SearchJob and SearchTask

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add MergeTopkResultSet

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove nq in VectorQuery

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change segment_size to segment_row_count in C++ sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix row_count in Search

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix GetEntityByID bug

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2020-07-31 11:44:15 +08:00
BossZou 007356af41
(scalar) Update list-id-in-segment case (#3060)
* Update test list id

Signed-off-by: yhz <413554850@qq.com>

* Update list id in segment case

Signed-off-by: yhz <413554850@qq.com>
2020-07-31 10:33:31 +08:00
groot 2c3fc9684a
remove MetaTypes.h (#3072)
* remove MetaTypes.h

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* remove MetaConsts.h

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-31 09:42:20 +08:00
XuPeng-SH 1449cf0932
(db/snapshot): Handle timeout with external metastore (#3075)
* (db/snapshot): update for row count

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bug in NewSegmentOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): remove dummy print

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add some test for row count

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update size logic

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update size logic related ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): rollback if operation is not done

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): clean store

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): remove some dependency

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update for store

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update Store.h

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update store related code

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add field element modification operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): change new operation name

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint error

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add Segment File Operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): crtp for BaseResource

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add InActiveResourcesGCEvent

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix ut error

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): small change

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update snapshot segmentcommit operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update drop all index operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint error

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix gc

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix gc segment files

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor Event related

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): change 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): change for GC event

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix build error for high version of boost filesystem

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): small change

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update compound operations

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add operation test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update for CompoundSegmentsOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add test for CompoundSegmentsOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add time stat in operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): timeout handling

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): timeout handling 2

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): timeout handling 3

Signed-off-by: peng.xu <peng.xu@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-07-30 22:31:54 +08:00
Cai Yudong 68e1a8da2b
add field name and segment size validation check when create collection (#3065)
* add field name and segment size validation check when create collection

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add collection name check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* clean header files

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* move constants to Types.h

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-30 18:17:10 +08:00
yukun 5e59056acf
Fix DBImpl and scheduler for search (#3062)
* Add search implementation in ExecutionEngine

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove attr_type from query_ptr

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add CreateStructuredIndex in ExecutionEngine

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add index fields in QueryPtr

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix SearchReq bugs

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* code format

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add Search in scheduler

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix SearchJob and SearchTask

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add MergeTopkResultSet

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Remove nq in VectorQuery

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Change segment_size to segment_row_count in C++ sdk

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Fix row_count in Search

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-07-30 17:07:14 +08:00
XuanYang-cn 7cc4862d4f
[skip ci]quick fix (#3071)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2020-07-30 17:06:20 +08:00
groot 841a7586d9
get collection stats (#3069)
* build index

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix some problem

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* build index ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* avoid ut error

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* get collection stats

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* remove EngineType

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-30 16:55:51 +08:00
del-zhenwu 52a59396ce
[skip ci] update segment_size to segment_row_count (#3068)
* [skip ci] update segment_size to segment_row_count

Signed-off-by: zw <zw@milvus.io>

* [skip ci] remove metric type in collection params

Signed-off-by: zw <zw@milvus.io>

Co-authored-by: zw <zw@milvus.io>
Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-07-30 15:48:38 +08:00
XuanYang-cn 86652416c5
ThirdParty libs won't be re-downloaded after make clean (#3059)
* reformate all CMKLists to make it more readable

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Thirdparty libs won't be re-download after make clean

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: Wang XiangYu <xy.wang@zilliz.com>
2020-07-30 15:14:16 +08:00
groot 4872fff4e0
fix build index bug (#3066)
* build index

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix some problem

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* build index ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* avoid ut error

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-30 14:06:13 +08:00
groot 824ff3e1ff
build index (#3058)
* build index

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix some problem

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* build index ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* refine code

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* fix ut

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* typo

Signed-off-by: yhmo <yihua.mo@zilliz.com>

* avoid ut error

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2020-07-30 10:53:35 +08:00
Cai Yudong d91facc86d
fix collection ci (#3061)
* fix set config issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add auto_flush_interval observer in DBImpl

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix some issues in SegmentReader

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update CreateCollection interface

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix clang-format

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2020-07-30 09:37:27 +08:00
XuPeng-SH 412eac3e8f
(db/snapshot): add CompoundSegmentsOperation and related test (#3057)
* (db/snapshot): update for row count

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix bug in NewSegmentOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): remove dummy print

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add some test for row count

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update size logic

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update size logic related ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): rollback if operation is not done

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): clean store

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): remove some dependency

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update for store

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update Store.h

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update store related code

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add field element modification operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): change new operation name

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint error

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): Add Segment File Operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): crtp for BaseResource

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add InActiveResourcesGCEvent

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix ut error

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): small change

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update snapshot segmentcommit operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update drop all index operation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update ut

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix lint error

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix gc

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix gc segment files

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): refactor Event related

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): change 1

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): change for GC event

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): fix build error for high version of boost filesystem

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): small change

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update compound operations

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add operation test

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): update for CompoundSegmentsOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>

* (db/snapshot): add test for CompoundSegmentsOperation

Signed-off-by: peng.xu <peng.xu@zilliz.com>
2020-07-29 20:09:04 +08:00