Commit Graph

341 Commits (ef82e0f1a4f5d0561aa1546a0555e9cccc2b454a)

Author SHA1 Message Date
yah01 546080dcdd
Support to retrieve json (#23563)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-04-21 11:46:32 +08:00
Enwei Jiao 967a97b9bd
Support json & array types (#23408)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2023-04-20 11:32:31 +08:00
cai.zhang 9288020da3
Fix bug for batch delete files on gcp of minio (#23052) (#23083) (#23090)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-09 16:22:30 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
yah01 081572d31c
Refactor QueryNode (#21625)
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
xige-16 9aa99aedbb
[Cherry-Pick] Remove arrow uasge in FieldData (#22726)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-03-20 10:41:56 +08:00
huanghaoyuanhhy 024beddfe6
Make GCS OAuth token thread-safe (#22714)
Signed-off-by: huanghaoyuan <haoyuan.huang@zilliz.com>
2023-03-14 18:09:54 +08:00
congqixia 732986aa04
Remove fmt.Print from internal package (#22722)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-14 17:36:05 +08:00
shaoyue 32581e6452
Support aliyun oss as object storage with ak or IAM (#22376)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-03-09 16:33:52 +08:00
yah01 90a5aa6265
Refine errors, re-define error codes (#22501)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-09 15:47:52 +08:00
jaime d126f06946
Decouple mq module from internal proto definition (#22536)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-04 23:21:50 +08:00
congqixia ea4ab9e4a4
Replace ErrorList with multiErrors (#22532)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-03 14:59:49 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
jaime 5936723904
Refine Read method of MinioChunkManager (#22235)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-02-21 16:22:26 +08:00
congqixia f2575e5fa8
Add unconvert & durationcheck linters and fix issues (#22161)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-02-15 17:22:34 +08:00
Xiaofan 6004867853
Add bucket name in error message (#21887)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-02-10 17:44:31 +08:00
Xiaofan 949d5d078f
Fix memory calculation in dataCodec (#21800)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-28 11:09:52 +08:00
congqixia 5986106037
Make paramtable init only once (#21782)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-01-19 14:53:44 +08:00
Enwei Jiao 264f08953d
Refactor Rocksmq's config (#21304)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-26 19:11:30 +08:00
Enwei Jiao 166e9f0da5
Refactor GrpcConfig (#21142)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-16 15:59:23 +08:00
Xiaofan 908023a06e
Add more test on file reader (#21082)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-12-09 11:55:19 +08:00
cai.zhang eb7ef01b9a
Fix bug for ListWithPrefix from minio with recursive is false (#21059)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-12-08 15:47:18 +08:00
Enwei Jiao 89b810a4db
Refactor all params into ParamItem (#20987)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
jaime 548e90ec68
Support an analogous Java Guava cache implementation (#20831)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-12-05 20:29:18 +08:00
congqixia f745d7f489
Fix compaction target segment rowNum is always 0 (#20937)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-12-01 20:33:17 +08:00
yah01 f76ea292d2
Change LRU to a generic type (#20744)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-11-22 12:11:11 +08:00
groot bf2107ecf5
Fix a regression of local storage (#20653)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-11-21 10:19:10 +08:00
Xiaofan 633a749880
Recude IndexCodec Load Memory (#20621)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-18 10:47:08 +08:00
Enwei Jiao c05b9ad539
Add event dispatcher for config (#20393)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-11-17 18:59:09 +08:00
shaoyue 09ea38615e
Fix gcp oauth token not cached (#20380)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-11-08 12:43:02 +08:00
Xiaofan 766957e75a
Recuce memory consumption in MinioChunkManager (#20284)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-11-03 12:19:35 +08:00
shaoyue 9fac1476f3
Add support for GCS(GoogleCloudStorage) with IAM (#20164)
go mod tidy

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-11-01 11:07:35 +08:00
Xiaofan 2bfecf5b4e
Refine bloomfilter and memory usage (#20168)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-10-31 17:41:34 +08:00
SimFG a55f739608
Separate public proto files (#19782)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-10-16 20:49:27 +08:00
yah01 89c9cb3680
Fix scan may break GC limitation (#19670)
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-10-15 15:13:24 +08:00
jaime 1919353f02
Fix parse segment id error from delta log (#19566)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-09-30 14:18:55 +08:00
congqixia 838a633584
Add ctx parameter for ChunkManager methods (#19546)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-29 16:18:56 +08:00
Ten Thousand Leaves b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982)
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
Xiaofan 928a213e31
Fix Chunk Manager Path mis use (#19141)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-09-23 14:40:51 +08:00
congqixia f24e16a66f
Handle multiple parquet RowGroup/Column (#19283)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-21 09:12:50 +08:00
SimFG d7f38a803d
Separate some proto files (#19218)
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
groot b161aec95e
Support input empty string (#19111) (#19144)
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-09-13 13:36:29 +08:00
xige-16 4de1bfe5bc
Add cpp data codec (#18538)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: zhagnlu lu.zhang@zilliz.com

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-09-09 22:12:34 +08:00
congqixia ce9ba0dd02
Remove debug log in minio chunk manager (#19134)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-09 17:32:37 +08:00
congqixia db23a256c2
Make minioChunkManager ListObject level by level when recursive is true (#19096)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-08 14:58:34 +08:00
congqixia 68a6587374
Set insert&stats binlog timestamp range (#19005)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-04 09:05:09 +08:00
congqixia c91bb0b015
Unify rootPath in configs and ChunkManager (#18808)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-08-25 19:32:53 +08:00
codeman 70254c3ec5
Unified catalog interface for segment (#18289) (#18290)
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-08-20 10:24:51 +08:00
xige-16 e40061b864
Update binlog event format (#18347)
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-08-11 14:06:38 +08:00
jaime efa5dfaa7b
Fix Parse binlog path failure (#18583)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-08-10 17:04:37 +08:00