milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
14,709 Commits
142 Branches
117 Tags
566 MiB
Commit Graph

374 Commits (59bc0a70006816a053f8f873bcce61c924df5d12)

Author SHA1 Message Date
Ji Bin 3cd28420f1
Support compile under windows (#15786) 
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-17 17:17:22 +08:00
Jiquan Long ffdad82b9c
Set load info in reschedule tasks (#16065) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-16 11:43:21 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00
Jiquan Long c984c4ea73
Set load meta info in load balance task (#16060) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-15 20:37:21 +08:00
congqixia 6a66faa2fb
Explictly specify Load operation meta (#16027) 
Fix #15996

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-14 09:50:01 +08:00
yah01 7124e91d53
Refactor assigning loading segments task in QueryCoord (#15895) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-07 19:23:58 +08:00
Cai Yudong 3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-04 11:17:56 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-03 21:57:56 +08:00
cai.zhang 88527fc0a9
Fix metrics label conflict (#15839) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-03 16:05:57 +08:00
bigsheeper e1bfd14db1
Add Prometheus metrics for QueryCoord (#15606) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: jingkl <34296482+jingkl@users.noreply.github.com>
 2022-02-28 16:51:55 +08:00
Bingyi Sun b41992eda1
Fix querycoord ut bug (#15752) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-02-28 11:27:54 +08:00
xige-16 1e8e3eb912
Fix wrong flushSegmentInfos in watchdmChannel request (#15740) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-02-24 19:23:52 +08:00
Bingyi Sun ce9662c140
Fix data race in test (#15690) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-02-22 17:43:52 +08:00
Bingyi Sun 2e676a4b78
Extract segments info into a new struct. (#15537) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-02-18 15:27:50 +08:00
congqixia c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-15 15:07:48 +08:00
Cai Yudong 8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig (#15501) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-09 14:41:45 +08:00
xige-16 55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-02-08 21:57:46 +08:00
Cai Yudong 1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-08 20:57:47 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-07 10:09:45 +08:00
Cai Yudong 4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-02 00:35:43 +08:00
xige-16 b9d1396258
Delete ut's useless param for genCollectionSchema (#15119) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-26 14:21:40 +08:00
xige-16 6336e2326c
Set triggerCondition when queryCoord reload loadbalanceTask meta (#15380) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-25 17:26:13 +08:00
xige-16 dfc6670f0f
Unsubscribe channel after query node down (#15230) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-17 17:37:37 +08:00
xige-16 cc5bee4684
Stop handoff if segment has been compacted (#15181) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-13 21:49:35 +08:00
xige-16 384d3169d3
Don't remove nodeInfo when querycoord panic at loadBalanceTask (#15189) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-13 17:59:34 +08:00
xige-16 4e956ee10c
Panic when recover querynode failed (#15161) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-12 17:43:37 +08:00
Cai Yudong b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 17:29:34 +08:00
Cai Yudong 7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-09 23:45:33 +08:00
Cai Yudong f8f61fb116
Use BaseParams etcd configuration for all components (#15012) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-07 16:15:26 +08:00
shaoyue 545ff88f9d
[skip e2e] Add comment for querycoord param (#14989) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-01-07 13:31:22 +08:00
guoxiangzhou 4403429705
[skip e2e] Modify log text according to the naming conventions (#14971) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2022-01-07 13:11:28 +08:00
Xieql 1e7c51859a
[skip e2e] Improve annotation (#14799) 
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
 2022-01-05 10:33:30 +08:00
xige-16 a304684b55
Delete useless var in watchDeletaChannelTask (#14641) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-04 19:21:31 +08:00
Xieql c7d47cefcd
[skip e2e] Improve annotation (#14717) 
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
 2022-01-04 19:15:42 +08:00
xige-16 d4ac010165
Fix not load delete log when handoff segment (#14758) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-04 14:27:18 +08:00
guoxiangzhou 644205ce96
[skip e2e] Modify log text according to the naming conventions (#14285) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-31 11:35:40 +08:00
xige-16 95f0e9a4a6
Fix proxy send query before queryNode watchQueryChannel and search timeout (#14640) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-30 22:51:21 +08:00
xige-16 b9730cb564
Fix not load delete log when handoff segment (#14613) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-30 19:45:20 +08:00
xige-16 b063bfda8f
Add index info in loadSegmentRequest (#14524) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-30 19:09:33 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 14:35:21 +08:00
xige-16 903e67b0ec
Fix recovery failed after query node down (#14449) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-29 12:15:21 +08:00
xige-16 1ee9d21fb0
[skip e2e]Improve log for QueryNode spell (#14469) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-28 21:53:07 +08:00
xige-16 5ad98c5738
[skip e2e]Improve log for QueryNode spell (#14468) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-28 21:49:07 +08:00
xige-16 eba0178a43
[skip e2e]Improve log for QueryNode spell (#14467) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-28 21:46:45 +08:00
guoxiangzhou 3bc73963a9
[skip e2e] Modify log text according to the naming conventions (#14284) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-27 15:12:59 +08:00
xige-16 8e2e9c3735
Fix wrong meta when balance's watchQueryChannel task failed (#14235) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-25 23:12:50 +08:00
Jiquan Long 73c5653c76
Fix close querycoord panic (#14239) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-25 17:52:17 +08:00
xige-16 8d7791513d
[skip e2e]Improve log for QueryNode spell (#14209) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-24 19:40:51 +08:00
xige-16 f55c12e9e7
[skip e2e]Improve log for QueryNode spell (#14208) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-24 19:39:03 +08:00
xige-16 ec9aefee55
[skip e2e]Improve log for QueryNode spell (#14206) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-24 19:37:12 +08:00
Jiquan Long 3a591068b0
Fix querycoord ut timeout (#14150) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-24 11:39:15 +08:00
xige-16 e3771bee3d
Fix data loss after balance task rescheduled (#14096) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-24 10:59:31 +08:00
xige-16 28bf1f60a4
[skip e2e]Improve log for QueryNode spell (#14105) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-23 22:02:07 +08:00
xige-16 e0dcba987a
[skip e2e]Improve log for QueryNode spell (#14104) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-23 21:58:14 +08:00
xige-16 1a02ae9186
[skip e2e]Improve log for QueryNode spell (#14103) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-23 21:56:22 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
xige-16 324f8b9b87
[skip e2e]Improve log for QueryNode spell (#14012) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-22 21:01:19 +08:00
congqixia ef3b71f22b
Use indexInfo SerializedSize when estimating segment size (#13969) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-22 20:27:34 +08:00
xige-16 d39a4a3f2d
Fix recover multi collection after query node down (#13952) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-22 16:29:06 +08:00
jaime 2524fbd1e8
[skip e2e] Fix grammar mistake in query_coord (#13918) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-21 21:55:12 +08:00
xige-16 db0b707dd2
[skip e2e]Improve log for QueryNode spell (#13913) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-21 21:27:09 +08:00
xige-16 fb0426afcc
[skip e2e]Improve log for QueryNode spell (#13912) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-21 21:25:15 +08:00
xige-16 76c4dccf71
[skip e2e]Improve log for QueryNode spell (#13911) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-21 21:23:19 +08:00
congqixia 55ab425010
Use binlog size from meta when estimate SegmentSize (#13891) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-21 20:01:03 +08:00
xige-16 4e4ff54d98
Watch query channel to near the latest position (#13682) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-21 13:50:54 +08:00
xige-16 11c8461c11
Refactor dmChannelWatchInfo to queryCoord meta (#13543) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-21 11:57:39 +08:00
XuanYang-cn 2e0f3695b2
Add more logs on failure path in load segments (#13785) 
This PR added more clarified logs
- in IO failures of estimateSegmentsSize
- in esitimate goroutines

See also: #13250

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-21 11:13:03 +08:00
congqixia 2e1e3d8304
[skip e2e] Fix typo in QueryCoord param table (#13687) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-20 12:28:43 +08:00
congqixia 1dcd06cc78
Add log file info for all kind of segment binlogs (#13606) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-19 20:00:42 +08:00
jaime 396685d4b5
[skip e2e] Fix grammar mistake in metrics_info (#13679) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-19 12:38:43 +08:00
jaime 417705dc11
[skip e2e] Fix grammar mistake in param_table.go (#13678) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-19 12:36:47 +08:00
xige-16 b7f987ef47
[skip e2e]Fix golint for loadSegments (#13674) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-17 22:32:42 +08:00
xige-16 7e0c772d64
[skip e2e]Fix golint for log QueryNode (#13672) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-17 22:30:46 +08:00
xige-16 a965bdf325
[skip e2e]Fix golint for log QueryNode (#13671) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-17 22:26:50 +08:00
Xiaofan b8d3808052
Fix Query Coord UT and cleanup logic (#13618) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-17 21:30:42 +08:00
bigsheeper 1ca08d9138
Simplify flow graph in QueryNode (#13613) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-17 20:12:42 +08:00
shaoyue 7686280391
[skip e2e] Add comment for index checker (#13622) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-12-17 18:47:20 +08:00
xige-16 5864e5ea47
[skip ci] Fix golint for querycoord init (#13549) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-16 22:39:15 +08:00
xige-16 8259f81ae6
[skip ci] Fix golint for loadBalanceSegmentLoop (#13548) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-16 22:37:34 +08:00
xige-16 8f0b3ccd08
[skip ci] Fix golint for QueryNode (#13547) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-16 22:35:38 +08:00
xige-16 cf82a248d5
[skip ci]Fix golint for QueryNode (#13468) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-16 19:57:14 +08:00
xige-16 b404138ba7
[skip ci]Fix golint for chooseSegmentToBalance (#13303) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-15 22:19:10 +08:00
xige-16 42ac5f76ed
Add constraints to the combination of load and release (#13110) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-15 22:11:09 +08:00
xige-16 7c18be6e08
[skip ci]Fix golint for loadBalanceSegmentLoop (#13302) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-15 21:01:10 +08:00
xige-16 97d5231a0a
Reorganize querycoord proto (#13181) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-15 16:53:12 +08:00
cai.zhang 9f23fc7f2a
Register the service when the component state is healthy (#13248) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-15 11:47:10 +08:00
congqixia 322f0c5c5c
Remove unused variables (#13380) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-15 11:11:24 +08:00
xige-16 c68bae236e
[skip ci]Fix golint for watchHandoffSegmentLoop (#13301) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-15 07:01:09 +08:00
Xiangyu Wang 405b3cd932
Use the same lint rules with golint in revive (#13324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-14 15:31:07 +08:00
congqixia 835e0985fd
Reduce queryNodeCluster range prevent blocking online/offline events (#13249) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-13 21:51:10 +08:00
xige-16 6f8ea8cc7b
Fix querycoord panic at getRecoveryInfo (#12983) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-13 10:29:27 +08:00
Cai Yudong b7e97f67e5
Use typeutil.QueryCoordRole instead of Params.RoleName in querycoord (#13216) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-13 10:15:26 +08:00
dragondriver 255be8f4a8
Fix querycoord panic when close before init scheduler (#13051) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-11 10:15:13 +08:00
xige-16 9ccbffc4b1
[skip ci]Fix golint for watchNodeLoop (#13199) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-10 21:55:48 +08:00
xige-16 f85c1e8f1e
[skip ci]Fix golint for SetIndexCoord (#13198) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-10 21:54:01 +08:00
JackLCL 554b7918e9
[skip ci]Fix error log output format in task_scheduler.go (#13129) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-10 10:16:03 +08:00
jaime 874c24862b
[skip ci] Fix a grammar mistake in querycoord task (#13127) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-10 10:13:44 +08:00
jaime 4e538f9929
[skip ci] Fix a grammar mistake in querycoord task_scheduler (#13126) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
 2021-12-10 10:11:59 +08:00
xige-16 63ead50544
[skip ci]fix golint for SetDataCoord (#13121) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-09 22:25:08 +08:00
xige-16 60d70e82fd
[skip ci]fix golint for SetRootCoord (#13120) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-09 22:23:22 +08:00