milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
15,999 Commits
127 Branches
115 Tags
560 MiB
Commit Graph

312 Commits (2aba154ad3e59f9478103a37b1d779060908aa6c)

Author SHA1 Message Date
shaoyue 09ea38615e
Fix gcp oauth token not cached (#20380) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-11-08 12:43:02 +08:00
Xiaofan 766957e75a
Recuce memory consumption in MinioChunkManager (#20284) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-03 12:19:35 +08:00
shaoyue 9fac1476f3
Add support for GCS(GoogleCloudStorage) with IAM (#20164) 
go mod tidy

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-11-01 11:07:35 +08:00
Xiaofan 2bfecf5b4e
Refine bloomfilter and memory usage (#20168) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-31 17:41:34 +08:00
SimFG a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
yah01 89c9cb3680
Fix scan may break GC limitation (#19670) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-15 15:13:24 +08:00
jaime 1919353f02
Fix parse segment id error from delta log (#19566) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-09-30 14:18:55 +08:00
congqixia 838a633584
Add ctx parameter for ChunkManager methods (#19546) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-29 16:18:56 +08:00
Ten Thousand Leaves b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-09-26 18:06:54 +08:00
Xiaofan 928a213e31
Fix Chunk Manager Path mis use (#19141) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-09-23 14:40:51 +08:00
congqixia f24e16a66f
Handle multiple parquet RowGroup/Column (#19283) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-21 09:12:50 +08:00
SimFG d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
groot b161aec95e
Support input empty string (#19111) (#19144) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-09-13 13:36:29 +08:00
xige-16 4de1bfe5bc
Add cpp data codec (#18538) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: zhagnlu lu.zhang@zilliz.com

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-09-09 22:12:34 +08:00
congqixia ce9ba0dd02
Remove debug log in minio chunk manager (#19134) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-09 17:32:37 +08:00
congqixia db23a256c2
Make minioChunkManager ListObject level by level when recursive is true (#19096) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-08 14:58:34 +08:00
congqixia 68a6587374
Set insert&stats binlog timestamp range (#19005) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-04 09:05:09 +08:00
congqixia c91bb0b015
Unify rootPath in configs and ChunkManager (#18808) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-25 19:32:53 +08:00
codeman 70254c3ec5
Unified catalog interface for segment (#18289) (#18290) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-08-20 10:24:51 +08:00
xige-16 e40061b864
Update binlog event format (#18347) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-08-11 14:06:38 +08:00
jaime efa5dfaa7b
Fix Parse binlog path failure (#18583) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-08-10 17:04:37 +08:00
jaime 6e7831470c
Revert settings of Pulsar address and refine config code (#18494) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-08-02 21:26:33 +08:00
jaime c9174d55ba
Refine merge operation during compacting phase (#18399) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-07-28 14:52:31 +08:00
congqixia e9d777b336
Fix ParseSegmentIDBinlog panicks with bad input (#18413) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-26 19:32:30 +08:00
xing.zhao d90313a1a8
Use chunkManager rather than minio.Client in datanode garbage collection (#18092) 
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
 2022-07-22 22:10:28 +08:00
Xiaofan 6d82ef8c20
fix garbage collector err handling (#18277) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-15 13:54:26 +08:00
cai.zhang 0ad15a22c8
IndexCoord handle events correctly (#17878) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 14:44:21 +08:00
Letian Jiang 4ae1ca2cac
Fix arrow builder nullptr check in FinishPayloadWriter (#17873) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-06-28 20:04:17 +08:00
Enwei Jiao 16c3aedc15
refine complie configuration (#17502) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-06-24 21:12:15 +08:00
congqixia 37464a281d
Fix datasorter swap vector logic (#17661) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-21 11:38:12 +08:00
shaoyue 04f836f516
Fix s3 delete file (#17574) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-06-16 18:14:10 +08:00
yah01 70f8bea4b4
Avoid growing slice as deserializing binlogs (#17421) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-08 11:46:06 +08:00
shaoyue 76eaa3fc50
Support access s3 through AWS IAM role (#17292) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-06-02 19:42:03 +08:00
Enwei Jiao d28a2db46c
move arrow from storage to core (#17061) 
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
 2022-05-22 20:03:58 +08:00
Xiaofan 5355153805
Minio Error is not Handled gracefully (#17003) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-16 19:25:55 +08:00
Letian Jiang 72bbe40254
Make PayloadReader read column data in batch (#16826) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-10 11:37:52 +08:00
cai.zhang 2be46a01c0
Reduce the number of retries and add error log (#16754) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-05-05 09:31:51 +08:00
Letian Jiang 5b2b917987
Benchmark go/cgo parquet payload readers (#16736) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-29 15:59:47 +08:00
xige-16 515d0369de
Support string type in segcore (#16546) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 13:35:49 +08:00
jaime 68b1b82faf
Remove DataKV interface (#16692) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-28 21:03:47 +08:00
yah01 7af02fa531
Improve load performance, load binlogs concurrently per file, deserialize binlogs concurrently per field/segment (#16514) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-25 15:57:47 +08:00
Ten Thousand Leaves 4ef2df8cb9
Fix an issue where query coord calls MinioChunkManager during init (#16629) 
It causes Milvus not runnable with local storage.

issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-25 11:11:46 +08:00
congqixia 9769426e84
Use fmt.Errorf instead of string concat in local_chunk_manager.go (#16442) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-19 18:39:39 +08:00
Ten Thousand Leaves e76a8c5ec2
Update compile options and config for embedded Milvus (#16472) 
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-14 19:57:34 +08:00
godchen 4781db8a2a
Add datanode import (#16414) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-12 22:19:34 +08:00
godchen bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
xige-16 99984b88e1
Support delete varChar value (#16229) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-04-02 17:43:29 +08:00
Jiquan Long ba37531456
Add support for loading multiple indexes (#16138) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-30 21:11:28 +08:00
Xiaofan 801eeffbcc
Replace cgo parquet reader to go parquet reader (#16199) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-03-30 15:21:28 +08:00
Xiaofan b6b3c9863c
Fix incompatibility of stats log (#16277) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-03-30 10:15:28 +08:00