milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
14,932 Commits
141 Branches
117 Tags
549 MiB
Commit Graph

401 Commits (e751663fdc17e6984ddecfa9391eaa560d25ab83)

Author SHA1 Message Date
yah01 60f7fef3b3
Remove offline nodes from replica info after LoadBalanceTask done (#16653) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-26 18:23:46 +08:00
Xiaofan c1ff0cec8f
Change QueryCoord Log level (#16590) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-26 11:29:54 +08:00
Bingyi Sun 1682dc49fc
Fix unit test timeout (#16537) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-04-25 14:49:46 +08:00
Ten Thousand Leaves 4ef2df8cb9
Fix an issue where query coord calls MinioChunkManager during init (#16629) 
It causes Milvus not runnable with local storage.

issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-25 11:11:46 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-24 22:03:44 +08:00
yah01 5c66300408
Always add the shard leader into the nodes list of shard (#16616) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-24 17:37:46 +08:00
yah01 b82f6a1a0a
Fix GetReplicas() returns all nodes in ShardReplicas (#16597) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-24 11:35:45 +08:00
congqixia 69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470) 
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-20 16:15:41 +08:00
jaime ed3c29eae7
Support Kafka (#16149) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-12 19:47:33 +08:00
godchen bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
XuanYang-cn 78200009a3
Impl GetReplicas in Proxy (#16390) 
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-06 14:57:31 +08:00
Letian Jiang 028c5cb882
Modify grpc interface for replica Search/Query in QueryNode (#16326) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-02 14:15:31 +08:00
yah01 c005f07ccc
Modify proto to add GetReplicas() interface (#16328) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-01 16:15:29 +08:00
XuanYang-cn 58addbf9f4
Add GetShardLeaders rpc in querycoord proto (#16299) 
See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-31 16:39:29 +08:00
Bingyi Sun baa745b83e
Improve log (#16307) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-03-31 11:33:28 +08:00
Bingyi Sun 40fab91448
improve log (#16236) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-03-30 23:05:28 +08:00
Jiquan Long ba37531456
Add support for loading multiple indexes (#16138) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-30 21:11:28 +08:00
congqixia 18a3e9f265
Add grpc interface for replica Search/Query in QueryNode (#16197) 
Resolves #16195
Add Search and Query grpc interface in query proto and types/types.go

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-30 12:03:27 +08:00
yah01 178eb8cce1
Modify proto to support replicas (#16218) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-28 16:43:27 +08:00
Bingyi Sun e0ffe54547
Add interface to get replications by collection ID (#16213) 
issue: #16158
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-03-28 14:35:27 +08:00
yah01 227889b0d0
Add replica interfaces for Meta and Cluster (#16190) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-25 14:03:25 +08:00
yah01 9deb245d77
Fix data race in unittest (#16165) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-24 20:19:25 +08:00
xiyichan ca129d4308
Support configurable msgstream (#16131) 
Signed-off-by: xiyichan <2863768433@qq.com>
 2022-03-24 10:15:25 +08:00
Jiquan Long f8d9bc919d
Unify interface of vector index & scalar index. (#15959) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-21 14:23:24 +08:00
XuanYang-cn ad41e2a082
Tidy getQueryChannel for readability (#16096) 
- Remove useless codes in query_coord.go
- Tidy slice initialization in LoadCollection task
- Add ut for getQueryChannel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-21 11:25:24 +08:00
XuanYang-cn 15a3fe41c5
[skip e2e]Fix comments of exposed functions in QC (#16094) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-21 11:17:24 +08:00
godchen 78557ca6d7
Replace minio kv with minio chunk manager (#15936) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-17 18:03:23 +08:00
Ji Bin 3cd28420f1
Support compile under windows (#15786) 
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-17 17:17:22 +08:00
Jiquan Long ffdad82b9c
Set load info in reschedule tasks (#16065) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-16 11:43:21 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00
Jiquan Long c984c4ea73
Set load meta info in load balance task (#16060) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-15 20:37:21 +08:00
congqixia 6a66faa2fb
Explictly specify Load operation meta (#16027) 
Fix #15996

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-14 09:50:01 +08:00
yah01 7124e91d53
Refactor assigning loading segments task in QueryCoord (#15895) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-07 19:23:58 +08:00
Cai Yudong 3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-04 11:17:56 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-03 21:57:56 +08:00
cai.zhang 88527fc0a9
Fix metrics label conflict (#15839) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-03 16:05:57 +08:00
bigsheeper e1bfd14db1
Add Prometheus metrics for QueryCoord (#15606) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: jingkl <34296482+jingkl@users.noreply.github.com>
 2022-02-28 16:51:55 +08:00
Bingyi Sun b41992eda1
Fix querycoord ut bug (#15752) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-02-28 11:27:54 +08:00
xige-16 1e8e3eb912
Fix wrong flushSegmentInfos in watchdmChannel request (#15740) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-02-24 19:23:52 +08:00
Bingyi Sun ce9662c140
Fix data race in test (#15690) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-02-22 17:43:52 +08:00
Bingyi Sun 2e676a4b78
Extract segments info into a new struct. (#15537) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-02-18 15:27:50 +08:00
congqixia c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-15 15:07:48 +08:00
Cai Yudong 8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig (#15501) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-09 14:41:45 +08:00
xige-16 55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-02-08 21:57:46 +08:00
Cai Yudong 1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-08 20:57:47 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-07 10:09:45 +08:00
Cai Yudong 4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-02 00:35:43 +08:00
xige-16 b9d1396258
Delete ut's useless param for genCollectionSchema (#15119) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-26 14:21:40 +08:00
xige-16 6336e2326c
Set triggerCondition when queryCoord reload loadbalanceTask meta (#15380) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-25 17:26:13 +08:00
xige-16 dfc6670f0f
Unsubscribe channel after query node down (#15230) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-17 17:37:37 +08:00
xige-16 cc5bee4684
Stop handoff if segment has been compacted (#15181) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-13 21:49:35 +08:00
xige-16 384d3169d3
Don't remove nodeInfo when querycoord panic at loadBalanceTask (#15189) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-13 17:59:34 +08:00
xige-16 4e956ee10c
Panic when recover querynode failed (#15161) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-12 17:43:37 +08:00
Cai Yudong b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 17:29:34 +08:00
Cai Yudong 7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-09 23:45:33 +08:00
Cai Yudong f8f61fb116
Use BaseParams etcd configuration for all components (#15012) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-07 16:15:26 +08:00
shaoyue 545ff88f9d
[skip e2e] Add comment for querycoord param (#14989) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-01-07 13:31:22 +08:00
guoxiangzhou 4403429705
[skip e2e] Modify log text according to the naming conventions (#14971) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2022-01-07 13:11:28 +08:00
Xieql 1e7c51859a
[skip e2e] Improve annotation (#14799) 
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
 2022-01-05 10:33:30 +08:00
xige-16 a304684b55
Delete useless var in watchDeletaChannelTask (#14641) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-04 19:21:31 +08:00
Xieql c7d47cefcd
[skip e2e] Improve annotation (#14717) 
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
 2022-01-04 19:15:42 +08:00
xige-16 d4ac010165
Fix not load delete log when handoff segment (#14758) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-04 14:27:18 +08:00
guoxiangzhou 644205ce96
[skip e2e] Modify log text according to the naming conventions (#14285) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-31 11:35:40 +08:00
xige-16 95f0e9a4a6
Fix proxy send query before queryNode watchQueryChannel and search timeout (#14640) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-30 22:51:21 +08:00
xige-16 b9730cb564
Fix not load delete log when handoff segment (#14613) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-30 19:45:20 +08:00
xige-16 b063bfda8f
Add index info in loadSegmentRequest (#14524) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-30 19:09:33 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 14:35:21 +08:00
xige-16 903e67b0ec
Fix recovery failed after query node down (#14449) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-29 12:15:21 +08:00
xige-16 1ee9d21fb0
[skip e2e]Improve log for QueryNode spell (#14469) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-28 21:53:07 +08:00
xige-16 5ad98c5738
[skip e2e]Improve log for QueryNode spell (#14468) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-28 21:49:07 +08:00
xige-16 eba0178a43
[skip e2e]Improve log for QueryNode spell (#14467) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-28 21:46:45 +08:00
guoxiangzhou 3bc73963a9
[skip e2e] Modify log text according to the naming conventions (#14284) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-27 15:12:59 +08:00
xige-16 8e2e9c3735
Fix wrong meta when balance's watchQueryChannel task failed (#14235) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-25 23:12:50 +08:00
Jiquan Long 73c5653c76
Fix close querycoord panic (#14239) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-25 17:52:17 +08:00
xige-16 8d7791513d
[skip e2e]Improve log for QueryNode spell (#14209) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-24 19:40:51 +08:00
xige-16 f55c12e9e7
[skip e2e]Improve log for QueryNode spell (#14208) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-24 19:39:03 +08:00
xige-16 ec9aefee55
[skip e2e]Improve log for QueryNode spell (#14206) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-24 19:37:12 +08:00
Jiquan Long 3a591068b0
Fix querycoord ut timeout (#14150) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-24 11:39:15 +08:00
xige-16 e3771bee3d
Fix data loss after balance task rescheduled (#14096) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-24 10:59:31 +08:00
xige-16 28bf1f60a4
[skip e2e]Improve log for QueryNode spell (#14105) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-23 22:02:07 +08:00
xige-16 e0dcba987a
[skip e2e]Improve log for QueryNode spell (#14104) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-23 21:58:14 +08:00
xige-16 1a02ae9186
[skip e2e]Improve log for QueryNode spell (#14103) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-23 21:56:22 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
xige-16 324f8b9b87
[skip e2e]Improve log for QueryNode spell (#14012) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-22 21:01:19 +08:00
congqixia ef3b71f22b
Use indexInfo SerializedSize when estimating segment size (#13969) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-22 20:27:34 +08:00
xige-16 d39a4a3f2d
Fix recover multi collection after query node down (#13952) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-22 16:29:06 +08:00
jaime 2524fbd1e8
[skip e2e] Fix grammar mistake in query_coord (#13918) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-21 21:55:12 +08:00
xige-16 db0b707dd2
[skip e2e]Improve log for QueryNode spell (#13913) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-21 21:27:09 +08:00
xige-16 fb0426afcc
[skip e2e]Improve log for QueryNode spell (#13912) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-21 21:25:15 +08:00
xige-16 76c4dccf71
[skip e2e]Improve log for QueryNode spell (#13911) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-21 21:23:19 +08:00
congqixia 55ab425010
Use binlog size from meta when estimate SegmentSize (#13891) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-21 20:01:03 +08:00
xige-16 4e4ff54d98
Watch query channel to near the latest position (#13682) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-21 13:50:54 +08:00
xige-16 11c8461c11
Refactor dmChannelWatchInfo to queryCoord meta (#13543) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-21 11:57:39 +08:00
XuanYang-cn 2e0f3695b2
Add more logs on failure path in load segments (#13785) 
This PR added more clarified logs
- in IO failures of estimateSegmentsSize
- in esitimate goroutines

See also: #13250

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-21 11:13:03 +08:00
congqixia 2e1e3d8304
[skip e2e] Fix typo in QueryCoord param table (#13687) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-20 12:28:43 +08:00
congqixia 1dcd06cc78
Add log file info for all kind of segment binlogs (#13606) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-19 20:00:42 +08:00
jaime 396685d4b5
[skip e2e] Fix grammar mistake in metrics_info (#13679) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-19 12:38:43 +08:00
jaime 417705dc11
[skip e2e] Fix grammar mistake in param_table.go (#13678) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-19 12:36:47 +08:00
xige-16 b7f987ef47
[skip e2e]Fix golint for loadSegments (#13674) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-17 22:32:42 +08:00
xige-16 7e0c772d64
[skip e2e]Fix golint for log QueryNode (#13672) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-17 22:30:46 +08:00