milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
15,720 Commits
146 Branches
120 Tags
569 MiB
Commit Graph

1144 Commits (ae8ec28925348a5bdf7d97410ae19fae126af449)

Author SHA1 Message Date
bigsheeper 641fb96958
Return malloc_trim result and log if malloc_trim successfully (#17959) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-01 10:44:18 +08:00
yah01 ef6859a4d6
Refactor handoff handler (#17806) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-29 23:26:19 +08:00
codeman 95a3830dc6
fix password comparison too slow (#17874) (#17897) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-06-29 20:02:18 +08:00
jaime 762e3b78a6
support configurable sasl mechanisms for kafka (#17542) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-29 10:32:17 +08:00
Enwei Jiao 16c3aedc15
refine complie configuration (#17502) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-06-24 21:12:15 +08:00
shaoyue 96b97f7f58
RESTful api use same port as metrics & healthcheck (#17732) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-06-24 15:34:14 +08:00
SimFG 6189a117a4
Remove the `WithOptions` for avoiding the log object being cloned continuously (#17746) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-06-24 10:36:15 +08:00
SimFG 50d5d04552
Refine some codes about the datanode (#17720) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-06-23 09:58:14 +08:00
SimFG f4c6a6734e
Fix rpc retry policy not taking effect (#17673) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-06-22 22:22:13 +08:00
Xiaofan 7d51b65260
Estimate varchar length more reasonablly (#17688) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-22 13:02:13 +08:00
SimFG eb1f0bc805
Add the consistency check tool (#17136) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-06-21 10:28:12 +08:00
groot 371a44f018
Fix a bug of bulkload (#17620) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-17 20:52:11 +08:00
bigsheeper 92d06b2e30
Purge memory by the memory state and try to purge after each search (#17565) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-17 17:46:10 +08:00
SimFG 2008f60e50
Simplify the implementations of the ToPhysicalChannel and the ConvertChannelName func (#17605) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-06-17 10:22:11 +08:00
cai.zhang 53a49a9255
GetIndexState only get the state generated call CreateIndex (#17549) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-16 20:12:11 +08:00
Xiaofan 1f6fbf91b2
Fix pulsar unsubsribe issue (#17562) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-16 17:28:11 +08:00
Xiaofan 54aeb077e9
Refine compaction selection poliy (#17486) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-15 23:14:10 +08:00
Ten Thousand Leaves 65ab847815
Add segment reference lock for bulk load (#17457) 
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-15 12:20:10 +08:00
congqixia ed3a1164dd
Handle wrapped grpc error in funcutil.IsGrpcError (#17547) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-14 17:10:10 +08:00
groot ab9b186f15
Change bulkload bufsize for row-based parser (#17473) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-14 16:18:09 +08:00
congqixia a0d3f37c0d
Sync grpc DefaultKeepAliveTimeout to milvus.yaml (#17519) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-13 16:18:09 +08:00
congqixia d1de8cabdd
Fix %w verb for error wrapping in grpcclient (#17484) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-13 12:48:08 +08:00
yah01 a38790f586
Predict the loading memory usage more accurate (#17451) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-09 18:16:07 +08:00
groot 9210299706
Fix bulkload OOM issue (#17461) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-09 16:52:07 +08:00
xige-16 b5c11a216d
Alter varChar type params's name to max_length (#17409) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-07 15:58:06 +08:00
Xiaofan a9ec1f8864
Fix compatibility of upgrade (#17406) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-07 14:52:06 +08:00
congqixia de8f2f76c5
Fix shardLeader cache with ErrConnect check (#17402) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-07 12:20:06 +08:00
Xiaofan af994b5e1c
Fix etcd watch error (#17364) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-06 16:26:06 +08:00
XuanYang-cn c70af73005
Fix datacoord set wrong state for node registering (#17376) 
Fix datacoord datarace

See also: #17335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-06 15:10:05 +08:00
bigsheeper 48cf63a2d7
Wait for flow graph to exit (#17341) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-02 19:48:04 +08:00
shaoyue 76eaa3fc50
Support access s3 through AWS IAM role (#17292) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-06-02 19:42:03 +08:00
SimFG 0986c29d7f
proxy tls support one-way authentication (#17348) 
Signed-off-by: xiyichan <2863768433@qq.com>

Co-authored-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>

Co-authored-by: xiyichan <2863768433@qq.com>
 2022-06-02 18:56:04 +08:00
Ten Thousand Leaves f5e63177d2
Fix an issue with bulk load with multiple DataNodes (#17324) 
issue: #17300
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-02 18:54:04 +08:00
yah01 4e83f37899
Improve load performance (#17273) 
Load segment's fields concurrently

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-02 17:56:03 +08:00
zhenshan.cao 2763efc9b0
Add cache of grpc client of ShardLeader in proxy (#17301) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-06-02 12:16:03 +08:00
jaime 5698bf4236
Add sasl configurations for kafka (#17323) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-02 12:12:03 +08:00
cai.zhang bcf3b7426a
Add distributed lock for segment refgerence (#16782) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-31 16:36:03 +08:00
Jiquan Long 6bd786bf50
Reduce default seal propotion (#17287) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-31 12:02:04 +08:00
congqixia c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-30 19:50:04 +08:00
zhenshan.cao b0d54ed0af
Rename Client to GrpcClient in mock package (#17230) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-30 17:16:08 +08:00
SimFG 731870211a
Support role-based access control (only pb) (#16987) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-05-28 00:04:01 +08:00
Ten Thousand Leaves 5e1e7a6896
Fix an issue that causes flush hanging (#17169) 
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-25 14:34:00 +08:00
congqixia a0cef3a0a2
Fix sessionutil log always datanode (#17184) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-24 16:05:59 +08:00
zhenshan.cao 2512e668f7
Update timestmap according to gracefultime for Bounded Consistency (#17171) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-24 12:05:59 +08:00
zhenshan.cao cce533c246
Fix unitest fail, change default index slice to 16 (#17162) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-23 17:44:25 +08:00
zhenshan.cao 08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-23 16:41:58 +08:00
ownbylichaobao 2f8a7e7793
Support tls/ssl connection to etcd (#7703) (#17012) 
Signed-off-by: lichaobao <1527563274@qq.com>
 2022-05-20 12:29:19 +08:00
groot d525e95550
Bulkload not allow duplicate numpy file (#17105) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-05-20 10:27:56 +08:00
bigsheeper 9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-19 10:13:56 +08:00
Xiaofan 5355153805
Minio Error is not Handled gracefully (#17003) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-16 19:25:55 +08:00
groot 168935f4cf
Fix bulkload bug for string primary key (#16958) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-05-13 16:07:54 +08:00
exia 4ee78eb5ad
proxy support tls (#16706) 
Signed-off-by: xiyichan <2863768433@qq.com>
 2022-05-10 20:05:53 +08:00
Bingyi Sun 76bfd41952
Refine DataNode logs. (#16849) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-05-09 16:25:52 +08:00
Enwei Jiao 67ce206cf0
make SegmentMaxLifetime configurable (#16783) 
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
 2022-05-06 23:41:51 +08:00
Jiquan Long 98ceb162aa
Support antlr as plan parser (#16696) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: xaxys <tpnnghd@163.com>

Co-authored-by: xaxys <tpnnghd@163.com>
 2022-05-06 17:43:51 +08:00
groot 8736372fd2
Fix bulkload bugs (#16760) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-05-06 11:21:50 +08:00
Ten Thousand Leaves 1acd256481
Add DataQueryable and DataIndexed states for bulk load tasks (#16725) 
issue: #16607

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-05 21:17:50 +08:00
Xiaofan 333b439a6c
Support config grpc log level (#16618) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-05 10:35:50 +08:00
shaoyue fa97f86cda
Clean up not used id_cache related code (#16655) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-05-05 09:59:51 +08:00
Xiaofan 337ad53613
Support config small index nprobe and nlist (#16731) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-05 09:49:50 +08:00
cai.zhang 2be46a01c0
Reduce the number of retries and add error log (#16754) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-05-05 09:31:51 +08:00
Cai Yudong 6a62ff18bf
Support easylogging config for segcore and knowhere (#16751) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-05-03 08:39:49 +08:00
Jiquan Long 2fe8677cbf
Enable dimension check in Proxy when create index request received (#16718) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 18:01:49 +08:00
shaoyue 2f49db9d3c
Implements remaining rest APIs (#16387) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-04-29 15:15:47 +08:00
xige-16 515d0369de
Support string type in segcore (#16546) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 13:35:49 +08:00
groot 21b5973253
Refine list import task api (#16690) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-28 17:21:47 +08:00
yah01 206b554d49
Sync the distribution of segments to shard leaders (#16685) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-28 16:01:48 +08:00
Xiaofan ed51ffd50f
Fix Compact Entity ttl param (#16665) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-28 11:51:47 +08:00
congqixia 0bf176a7e1
Add SyncReplicaSegments interface for QueryNode (#16672) 
Add SyncReplicaSegments for LoadCollection/LoadPartition child task
This API notifies related ShardCluster to record segment Loaded
Preventing the Etcd events is delayed, which will cause some segment not search

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-27 10:41:46 +08:00
yah01 80ae6de323
Add comments for concurrency package (#16654) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-26 11:33:45 +08:00
groot bc5e9ec929
Add list import tasks api (#16605) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-25 17:37:46 +08:00
yah01 7af02fa531
Improve load performance, load binlogs concurrently per file, deserialize binlogs concurrently per field/segment (#16514) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-25 15:57:47 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-24 22:03:44 +08:00
groot a6a3b69d91
Report bulk load state (#16555) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-21 21:37:42 +08:00
codeman 91e84ffedf
fix: not create user root if exists, optimize log & imports (#16554) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-21 19:57:42 +08:00
Ten Thousand Leaves 6f75d02c65
Disable knowhere logging for embedded Milvus and some other tweaks (#16496) 
/kind enhancement

issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-20 17:23:46 +08:00
Ten Thousand Leaves 289e468a7a
Make bulk load fully work (#16512) 
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-20 14:03:40 +08:00
codeman 3a1b2cedd2
create default root user for authentication (#16545) (#16549) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-20 13:03:40 +08:00
Xiaofan f367249b9e
Fix storage type config key (#16542) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-19 17:47:39 +08:00
codeman f5e1ded854
fix #16504 (#16523) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-19 16:35:39 +08:00
Xiaofan fce8f6cfab
Remove Timetick channel (#16449) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-17 23:47:36 +08:00
godchen 4781db8a2a
Add datanode import (#16414) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-12 22:19:34 +08:00
jaime ed3c29eae7
Support Kafka (#16149) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-12 19:47:33 +08:00
codeman 4e12271d79
Support login with username and password (#15656) (#16341) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-11 19:49:34 +08:00
congqixia 994bb829d3
Use fmt.Errorf instead of string concat in schema.go (#16404) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-11 09:47:37 +08:00
Cai Yudong 71cd7ba67a
Add configuration common.indexSliceSize (#16438) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-08 20:29:33 +08:00
godchen bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
Ten Thousand Leaves 1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. (#16374) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-06 15:33:32 +08:00
XuanYang-cn 78200009a3
Impl GetReplicas in Proxy (#16390) 
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-06 14:57:31 +08:00
Ten Thousand Leaves 2bd0a9526a
Implement task expiring logic and make all deadlines configurable. (#16355) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-03 11:37:29 +08:00
groot be8d9a8b6b
Support import numpy file (#16348) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-03 11:27:29 +08:00
xige-16 99984b88e1
Support delete varChar value (#16229) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-04-02 17:43:29 +08:00
Letian Jiang 028c5cb882
Modify grpc interface for replica Search/Query in QueryNode (#16326) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-02 14:15:31 +08:00
XuanYang-cn 5494f3c318
Remove offLineNodes from watch states check (#16347) 
This PR also
- adds more information in log
- makes Session able to logged by zap.Any/zap.String

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-01 22:33:29 +08:00
yah01 c005f07ccc
Modify proto to add GetReplicas() interface (#16328) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-01 16:15:29 +08:00
Jiquan Long fd589baca7
Integrates marisa trie index (#16192) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-01 15:31:29 +08:00
Ten Thousand Leaves 45be3deb3f
Implement RC, DC, DN calling path for import. (#16321) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-01 11:33:28 +08:00
groot bd241cd28b
Add performance test case for import (#16296) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-01 10:07:28 +08:00
XuanYang-cn 58addbf9f4
Add GetShardLeaders rpc in querycoord proto (#16299) 
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-31 16:39:29 +08:00
congqixia f529f8c1fc
Use fmt.Errorf instead of str concat in func.go (#16315) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-31 15:03:28 +08:00
Jiquan Long ba37531456
Add support for loading multiple indexes (#16138) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-30 21:11:28 +08:00
groot ffa06c77b6
Import util functions (#16237) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-03-30 16:25:30 +08:00
congqixia 18a3e9f265
Add grpc interface for replica Search/Query in QueryNode (#16197) 
Resolves #16195
Add Search and Query grpc interface in query proto and types/types.go

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-30 12:03:27 +08:00
godchen 478f6ca11e
Add cache for vector chunk (#15912) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-26 22:05:26 +08:00
xige-16 205c92e54b
Support insert string data (#15993) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-25 14:27:25 +08:00
Cai Yudong f4ebd3a9ce
Upgrade to knowhere v1.1.0 (#16186) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-25 13:49:25 +08:00
Ten Thousand Leaves a23ca745bb
Add Etcd support for Import Tasks. (#16164) 
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-25 11:03:25 +08:00
bigsheeper 2fcf7b9760
Fix GetMemoryCount, get min as total memory (#15931) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-24 20:41:25 +08:00
xiyichan ca129d4308
Support configurable msgstream (#16131) 
Signed-off-by: xiyichan <2863768433@qq.com>
 2022-03-24 10:15:25 +08:00
groot 1c4b949a1d
modify import rpc interface (#16129) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-03-22 15:11:24 +08:00
Jiquan Long f8d9bc919d
Unify interface of vector index & scalar index. (#15959) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-21 14:23:24 +08:00
Xiaofan 787d4f8bfd
Refine prometheus metrics and etcd log (#16084) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-03-18 15:51:22 +08:00
Ji Bin 3cd28420f1
Support compile under windows (#15786) 
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-17 17:17:22 +08:00
edward.zeng d4ad785b76
Bump go version in documentation (#15979) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2022-03-16 18:51:21 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00
zhenshan.cao 0bd07daae3
Add grpc healthy check (#16050) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 15:05:21 +08:00
XuanYang-cn d4a0c5eba6
Alter comments in parallel.go for readability (#16032) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-15 10:26:01 +08:00
xige-16 09b2c46b61
Support create collection with VarChar field (#15904) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-14 23:20:02 +08:00
jaime 29975a7a26
Remove Consume() API from mqstream interface (#15886) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-11 20:09:59 +08:00
groot 89356ebcb9
Modify import rpc interfaces (#16006) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-03-11 17:13:59 +08:00
yah01 7d2934e4c2
Add methods to compress/decompress small blocks (#15980) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-11 14:55:59 +08:00
Ten Thousand Leaves d5bae6710f
Make all logs configurable for standalone/embedded Milvus. (#15926) 
Resolves: #15919, #15708
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-10 10:35:59 +08:00
groot 5fdef607d5
Add rpc interfaces for import (#15930) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-03-09 18:33:59 +08:00
yah01 565524ed3c
Add zstd compressor in util (#15779) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-08 18:17:58 +08:00
Jiquan Long f71651e294
Support column-based insert data in message stream (#15802) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-04 15:09:56 +08:00
Cai Yudong 3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-04 11:17:56 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-03 21:57:56 +08:00
Cai Yudong 92c8e32ebd
Let MemoryKV.Load return error when key not exist (#15814) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-02 18:51:55 +08:00
XuanYang-cn e394ba8c88
Refactor rocksmq comments and logs for readability (#15801) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-01 18:55:55 +08:00
Jiquan Long 001567640e
Fix shallow copy of typeutil.AppendFieldData (#15790) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-01 15:13:54 +08:00
Bingyi Sun d6857f4b24
Remove globalZapWrapper in logutil.go (#15726) 
issue: #15720

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-02-28 17:23:54 +08:00
yah01 867cf620ee
Add Extract() method to take out C pointer from BytesConverter without releasing it (#15614) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2022-02-27 09:47:53 +08:00
Nemo cdcb3627b4
Make Etcd log level configurable. (#15737) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-02-25 15:03:53 +08:00
godchen fcf0887d92
Add and implement chunk manager interface. (#15541) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-02-24 23:53:53 +08:00
shaoyue 83fdc1da7c
Add web server in proxy with sample handler (#15647) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-02-23 14:37:52 +08:00
congqixia c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-15 15:07:48 +08:00
yah01 684110bc9a
Avoid to copy during converting C bytes to Go bytes (#15551) 
* Avoid to copy during converting C bytes to Go bytes

Signed-off-by: yah01 <yah2er0ne@outlook.com>

* Add BytesConverter to achieve no-copy converting C bytes to Go bytes

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2022-02-15 14:17:48 +08:00
Xiaofan be42627ab2
Move Logutil to Util dir (#15447) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-02-09 18:55:46 +08:00
Cai Yudong 8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig (#15501) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-09 14:41:45 +08:00
jaime 307a8ce535
Support compile and run on Mac (#15491) 
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
 2022-02-09 14:27:46 +08:00
xige-16 55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-02-08 21:57:46 +08:00
Cai Yudong 1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-08 20:57:47 +08:00
JunSung Park f6873d3dc1
Implement automatic item expiration on compaction (#14418) 
* Implement automatic item expiration on compaction

Signed-off-by: JunSung Park <writtic@gmail.com>

* Change entityExpiration value comparison logic

Signed-off-by: JunSung Park <writtic@gmail.com>

* Add logging for the number of expired entity

Signed-off-by: JunSung Park <writtic@gmail.com>

* Set default value of CompactionEntityExpiration to off

Signed-off-by: JunSung Park <writtic@gmail.com>

* Fix errors for code checker

Signed-off-by: JunSung Park <writtic@gmail.com>

* Set default entityExpiration to max value

Signed-off-by: JunSung Park <writtic@gmail.com>
 2022-02-07 22:45:46 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-07 10:09:45 +08:00
Cai Yudong 4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-02 00:35:43 +08:00
Bingyi Sun 515fe962d5
Fix subscription path (#15373) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-01-25 13:27:44 +08:00
Bingyi Sun 3987574448
Unsubscribe topic in DataCoord (#15353) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-01-24 19:47:38 +08:00
xige-16 1cd2363bd8
Delete usless params SkipQueryChannelRecovery (#15289) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-21 21:43:40 +08:00
Bingyi Sun c877f33084
Downgrade pulsar client (#15325) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-01-20 22:25:38 +08:00
congqixia 1e61112b5a
Add retry logic in pulsar consumer unsubscribe (#15284) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-01-20 01:23:37 +08:00