milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
17,265 Commits
130 Branches
115 Tags
556 MiB
Commit Graph

212 Commits (f12574aaf34262c899fda8e4fbc86ab97ac44ee1)

Author SHA1 Message Date
wei liu 74da53c027
fix update load percentage (#23054) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-30 10:48:23 +08:00
wei liu e2096965c7
fix leader view (#23038) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-29 14:06:02 +08:00
yah01 dc6d4b913a
Fix partitions may be not recovered with double load partitions (#23061) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-28 21:38:02 +08:00
yah01 ae0f467c02
Fix segment/channel may be re-loaded/subscribed (#22969) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-27 18:28:00 +08:00
yah01 081572d31c
Refactor QueryNode (#21625) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-27 00:42:00 +08:00
XuanYang-cn 93bc805933
Enhance ID allocator in DataNode (#22905) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-23 19:43:57 +08:00
yah01 2b81933d13
Refine logs of DistHandler (#22879) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-21 18:11:57 +08:00
yah01 68b9cabb87
Fix GetShardLeader returns old leader (#22887) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-21 16:57:57 +08:00
yihao.dai 1f718118e9
Dynamic load/release partitions (#22655) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-20 14:55:57 +08:00
yah01 3d8f0156c7
Refine scheduler & executor of QueryCoord (#22761) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-16 17:43:55 +08:00
wei liu bb5088e605
fix unassign from rg (#22747) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-16 14:27:55 +08:00
SimFG b57e476089
Fix the nil point about the session (#22748) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-03-14 20:07:54 +08:00
yah01 21ba8182ee
Refine task create errors (#22745) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-14 18:51:53 +08:00
yah01 1a4732bb19
Use new errors to handle load failures cache (#22672) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-10 17:15:54 +08:00
yah01 90a5aa6265
Refine errors, re-define error codes (#22501) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-09 15:47:52 +08:00
wei liu 11f1f4226a
support replica observer assign node (#22604) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-08 18:57:51 +08:00
congqixia f3e2d4a39a
Fix querynode stop meet unexpected GetComponentStates (#22590) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-07 09:51:50 +08:00
yah01 d4fccdd135
Add reason for spawning task (#22549) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-06 10:13:50 +08:00
jaime d126f06946
Decouple mq module from internal proto definition (#22536) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-03-04 23:21:50 +08:00
wei liu c162c6ecc0
fix assign node err (#22479) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-01 11:11:47 +08:00
wei liu d433e95ce0
fix transfer node meta err (#22420) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-27 19:29:47 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
zhenshan.cao e768437681
Correct usage of Timer and Ticker (#22228) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-02-23 18:59:45 +08:00
wei liu a9a263d5a8
fix assign node to replica in nodeUp (#22323) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-23 14:15:45 +08:00
wei liu c3e8ad3629
fix balance generate reduce task (#22236) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-21 19:06:27 +08:00
wei liu 87a4ddc7e2
fix rg e2e (#22187) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-16 10:48:34 +08:00
wei liu 7b4511b8f4
fix transfer node (#22120) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-14 16:16:34 +08:00
cai.zhang 66b3566ac1
Update component state to healthy after start (#22118) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-10 17:40:32 +08:00
wei liu e9684ddb5a
refine rg capacity behavior (#22089) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-09 18:46:31 +08:00
wei liu d085abbd56
fix load collection with rg (#22083) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-09 16:24:31 +08:00
yah01 b1f31da77a
Fix activate standby server ignores all errors (#22073) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-09 15:24:31 +08:00
Bingyi Sun 4005508880
Fix load collection blocks when memory is limited. (#21990) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-02-06 16:07:53 +08:00
wei liu 5da6aade02
fix default capacity of default rg (#21965) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-06 15:55:56 +08:00
Ten Thousand Leaves 74e3cc64fb
Add refresh path to proxy (#21933) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-02-02 19:29:51 +08:00
jaime e14f96a8e4
[skip e2e]Fix missed logger with information (#21859) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-30 11:51:49 +08:00
cai.zhang bcae97b125
Register after start to prevent there are tow coordinator at the same time (#21641) (#21707) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-30 11:11:50 +08:00
wei liu 73c44d4b29
resource group impl (#21609) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-30 10:19:48 +08:00
Bingyi Sun b0b0929c6c
Change CreateAt to UpdatedAt when checking timeout (#21769) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-01-18 17:29:43 +08:00
Ten Thousand Leaves defb7660c8
Add refresh option for LoadCollection/LoadPartitions interfaces (#21776) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-18 16:41:44 +08:00
Bingyi Sun b91bb5a729
Fix load timeout after next target updates (#21759) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-01-17 15:15:46 +08:00
yah01 c8f89907b6
Fix current target may be updated to an invalid target (#21742) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-17 11:41:51 +08:00
congqixia 5f3d3dc4fc
generate-mockery for querycoordv2, querynode and rootcoort (#21714) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-16 13:59:42 +08:00
yah01 837e3162d7
Fix ready notifiers leak when collection released (#21712) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-14 21:55:41 +08:00
yah01 32fb409e57
Fix may update the current target to an unavailable target when node down (#21698) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-13 17:11:41 +08:00
yah01 c66ce4aeba
Add warning comments (#21697) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-13 16:43:41 +08:00
Enwei Jiao 90d9e165d4
Fix some configs not shown (#21653) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-13 15:31:41 +08:00
congqixia 782f942b6f
Add MLogger.WithRateGroup for logger (#21703) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-13 14:11:40 +08:00
wayblink 6a722396bd
Integration test framework (#21283) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-12 19:49:40 +08:00
Enwei Jiao fb42466c65
Use opentelemetry (#21509) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 16:09:39 +08:00
Enwei Jiao 9fccbbb92b
Remove factory in querycoord (#21659) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 15:59:39 +08:00
cai.zhang e127cf7b99
Reset indexpb for upgrade (#21620) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-11 14:35:40 +08:00
bigsheeper 2146af1fb2
Return insufficient memory error when load failed (#21574) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-01-10 20:35:39 +08:00
yah01 3b41a6931d
Add QueryCoord OWNERS file (#21552) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-10 16:15:37 +08:00
MrPresent-Han 6fb3542f2a
enable auto balance paramter(#21504) (#21507) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-01-06 14:45:35 +08:00
cai.zhang aa203acfb3
Low IndexCoord weight (#21548) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-06 14:21:37 +08:00
cai.zhang e5f408dceb
Merge IndexCoord and DataCoord (#21267) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-04 19:37:36 +08:00
yah01 7b39873ae0
limit the frequency of GetMetrics() log (#21514) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-04 17:39:35 +08:00
Jiquan Long ff2a68e65a
Fix collection not exist when tried to do recovery (#21471) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-04 16:37:35 +08:00
SimFG df6ccfa104
Fix wrong bool param in the `getDistribution` method (#21463) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-30 14:15:31 +08:00
SimFG e6d2849c9a
Check whether the node is stopping when using `load balance` api (#21438) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-29 15:47:31 +08:00
wayblink 0bcedbd2bf
quick fix queryCoordV2 active-standby (#21395) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-12-28 10:49:29 +08:00
SimFG 6a29a964df
Fix queryCoord panic during query node down (#21400) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-28 10:17:30 +08:00
Xiaofan d16b7c3c2d
Add more logs for debugging load balance (#21389) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-27 15:57:30 +08:00
yah01 396a85c926
Fix the number of executing tasks may break the limit (#21318) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-20 20:33:27 +08:00
bigsheeper fc10c74005
Use channel-cp as seekPosition when FromDmlCPLoadDelete (#21110) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-12-13 16:17:22 +08:00
smellthemoon b04eb17088
Fix load timeout (#21156) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-12 18:43:23 +08:00
yah01 5ba1a94858
Fix observers may update current target to a unfinished next target (#21107) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-12 15:15:22 +08:00
aoiasd de0ab9e2cf
Refactor showConfigurations to allow return global config rather than only return config of this component (#21063) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-12-09 14:31:21 +08:00
yah01 9ebaa10dec
Add more logs for GetShardLeaders (#21046) 
Also increase the heartbeatAvailableInterval from 2.5s to 10s

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-08 19:09:18 +08:00
Bingyi Sun 27219ce9c0
Fix load progress bug (#21069) 
issue: #21023

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-12-08 18:41:20 +08:00
Enwei Jiao 89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
SimFG f8cff79804
Support the graceful stop for the query node (#20851) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-06 22:59:19 +08:00
yah01 0297ab1a46
Make progress if any channel/segment was loaded on node (#20775) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-06 18:29:19 +08:00
yah01 11e4445ef7
Check whether segments are fully loaded while fetching shard leaders (#20991) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-06 18:05:18 +08:00
smellthemoon c49f20ea94
Load should base on the accurate loaded number (#20999) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-06 17:13:18 +08:00
yah01 aec347e591
Check last heartbeat response time while fetching shard leaders (#20968) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-05 15:09:20 +08:00
yah01 ddd29ea6ab
Optimize scheduler, increase merge tasks probability (#20922) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-01 16:25:16 +08:00
Enwei Jiao 2ecdb4ba4a
Etcd config source support TLS (#20874) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-30 18:23:15 +08:00
Jiquan Long 5a6a92d603
Refine log level (#20896) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-30 14:11:15 +08:00
yah01 060649b8aa
Refactor task scheduler and executor (#20828) 
Make the performance able to scale out

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-30 13:57:15 +08:00
MrPresent-Han b021d7c59c
[skip e2e] add log for querynode load and remove segment(#20510) (#20810) 
issue: #20510

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2022-11-25 17:33:13 +08:00
wei liu 67403fcb3b
fix mannual balance with empty segment list (#20738) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-21 19:29:12 +08:00
smellthemoon 8283d32ac4
Only balance segement in targets (#20709) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-21 16:21:11 +08:00
MrPresent-Han d44d50e735
fix getting query segment info error during the period of loading and unloading segments (#20549) 
issue: #20281

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2022-11-21 10:41:10 +08:00
Enwei Jiao c05b9ad539
Add event dispatcher for config (#20393) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-17 18:59:09 +08:00
yah01 cc371d6801
Task/Action won't finish util the RPC returned (#20669) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-17 17:55:09 +08:00
Enwei Jiao 19524a5344
Fix nodeID mismatch at standalone mode (#20648) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-17 17:15:08 +08:00
wei liu 1177ed1a33
fix duplicate watch channel task (#20667) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-17 15:05:08 +08:00
yah01 31872f436c
Only balance segments in targets (#20635) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-16 14:33:11 +08:00
yah01 2d806788fb
Add version for subscribed channel (#20585) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-15 13:21:07 +08:00
yah01 d37ebe538b
Record load latency for loaded collection (#20538) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-15 10:11:07 +08:00
smellthemoon 7325b3e1c3
Substitute traceid for msgid in rpc (#20450) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-14 15:29:06 +08:00
wei liu 6a2e458f90
fix collection not exist cause query coord panic (#20553) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-14 14:51:06 +08:00
yah01 a82c235599
Scheduler should returns err if the queue is full (#20465) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-11 17:55:05 +08:00
wei liu eaa5cfdcb5
fix leader observer sync logic (#20478) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-11 11:43:06 +08:00
wei liu 7537dbfa37
skip balance on loading collection (#20483) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-10 17:53:04 +08:00
yah01 c71c6378ff
Clear stale replicas (#20456) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-10 17:01:03 +08:00
yah01 a68cec85af
Refine QueryCoordV2 metrics (#20461) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-10 15:01:04 +08:00
Jiquan Long 854fb6aad2
Fix querycoord metrics (#20429) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-09 17:45:04 +08:00
SimFG 9c6436d72d
Add time log for methods of starting the node (#20313) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-08 20:13:03 +08:00