8a9ab69369
fix: Skip statslog generation flushing empty L0 segment ( #28733 )
...
See also #27675
When L0 segment contains only delta data, merged statslog shall be
skiped when performing sync task
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-25 15:10:25 +08:00
ece592a42f
Deliver L0 segments delete records ( #27722 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-07 01:44:18 +08:00
7358c3527b
Add iterators ( #27643 )
...
See also: #27606
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-10-18 19:34:08 +08:00
2f16339aac
Enhance InsertData and FieldData ( #27436 )
...
1. Add NewInsertData
2. Add GetRowNum(), GetMemorySize(), and, Append() for InsertData
3. Add AppendRow() for FieldData for compaction
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-10-17 17:36:11 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
9166011c4a
Add float16 vector ( #25852 )
...
Signed-off-by: Writer-X <1256866856@qq.com>
2023-09-08 10:03:16 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
ebd0279d3f
Check error by Error() and NoError() for better report message ( #24736 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
c84bdcea49
merge stats log when segment flushing or compacting ( #23570 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-05-29 10:21:28 +08:00
967a97b9bd
Support json & array types ( #23408 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2023-04-20 11:32:31 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
081572d31c
Refactor QueryNode ( #21625 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
949d5d078f
Fix memory calculation in dataCodec ( #21800 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-28 11:09:52 +08:00
633a749880
Recude IndexCodec Load Memory ( #20621 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-18 10:47:08 +08:00
a55f739608
Separate public proto files ( #19782 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
d7f38a803d
Separate some proto files ( #19218 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
4de1bfe5bc
Add cpp data codec ( #18538 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: zhagnlu lu.zhang@zilliz.com
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-09 22:12:34 +08:00
99984b88e1
Support delete varChar value ( #16229 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-04-02 17:43:29 +08:00
bccf65ec67
[skip e2e]Update license for storage datacodec ( #14039 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-23 12:01:36 +08:00
7e56f08747
Add payload bytes interface. ( #13467 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-16 16:35:42 +08:00
48b45d82e5
Add ut for binlog_io to 100 coverage ( #12283 )
...
Make DN ut coverage upto 90%
Resolves : #8058
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-26 17:43:17 +08:00
9d5bcd3e3a
Close event and binlog reader ( #12173 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-22 17:27:14 +08:00
863f1bb34e
Fix multi delete data not effect ( #11422 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-09 15:01:17 +08:00
cd06f50645
Remove schema in delete codec ( #10517 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-24 09:59:10 +08:00
ffc0c07610
Change delete data primary key to int64 ( #10438 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 15:37:12 +08:00
2255fe0b45
Change deserialize deltelog from 1 blob to blobs ( #10085 )
...
See also: #9530
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-19 10:28:35 +08:00
59ab0e441c
Add bloom filter for stats ( #9630 )
...
* Add bloom filter for stats
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* trigger GitHub actions
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-13 10:22:33 +08:00
dedf745b76
Rename IndexParamsFile to IndexParamsKey ( #9563 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-10-09 19:27:02 +08:00
818cf3ffa0
Split blob into several string rows when index file is large ( #8919 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-30 17:57:01 +08:00
cf8600077f
Refactor the index file format ( #8514 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-09-29 09:52:12 +08:00
af173dd2a0
Add delete codec ( #8736 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-09-28 14:30:02 +08:00
db94d7771f
Read vector from disk ( #6707 )
...
* Read vector from disk
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* go fmt
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix git action error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix test error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix action error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix caculate error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change var name
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* remove unused method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* remove unused method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix len error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* remove unused code
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change bytes to float method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* change float to bytes method
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix action error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-07-24 09:25:22 +08:00
a992dcf6a8
Support query return vector output field ( #6570 )
...
* improve code readibility
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add offset in RetrieveResults
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add VectorFieldInfo into Segment struct
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add new interface for query vector
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update load vector field logic
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update load vector field logic
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fill in field name in query result
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add FieldId into FieldData
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add fillVectorOutputFieldsIfNeeded
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update data_codec_test.go
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add DeserializeFieldData
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* realize query return vector output field
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix static-check
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* disable query vector case
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-07-16 17:19:55 +08:00
1c6786f85c
Add blob info ( #5792 )
...
* Add blob info
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
* fix error
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-16 12:03:57 +08:00
23c4de0eb8
Flush statistics for all int64 fields ( #5318 )
...
Resolves : #5262
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-20 10:38:45 +00:00
82ccd4cec0
Rename module ( #4988 )
...
* Rename module
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
0dfcb90881
Add storage copyright
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-19 11:32:24 +08:00
a5ad70a5ab
Add unittest for storage
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-19 10:36:19 +08:00
f3649f0419
Refactor interface and proto
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-12 14:22:09 +08:00
01e9dc8e3f
Remove collection name
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 11:52:19 +08:00
5e781b9370
Remove field name in query node and segCore
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-03 10:10:07 +08:00
45b99c0cf3
Save indexName and indexID in IndexCodec
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-02-02 19:56:11 +08:00
73d2b6a101
Get index param from minio and filter by vector fields
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-29 15:22:24 +08:00
3a866dab3d
Update proto
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-28 17:25:43 +08:00
2ebeade25e
Fix get component states
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-28 17:13:00 +08:00
2be8cc1c4b
Add index params serde to IndexCodec
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-28 16:41:24 +08:00
7ce0f27ebc
Add buffer to minIO for binlogs
...
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-23 18:06:04 +08:00
e7ebfcb05a
Save index meta to meta table
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 15:13:45 +08:00
e52b130ba6
Refactor data codec code
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2020-12-23 15:46:16 +08:00
0b9cffaf87
Remove index_name in IndexRequest and remove indexType in FieldIndexMeta
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2020-12-23 11:34:35 +08:00
congqixia
yah01
XuanYang-cn
SimFG
Xu Tong
yah01
aoiasd
Enwei Jiao
jaime
Xiaofan
xige-16
godchen
dragondriver
Cai Yudong
Xiangyu Wang
bigsheeper
sunby
neza2017