milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
20,631 Commits
133 Branches
115 Tags
559 MiB
Commit Graph

40 Commits (9d801376982b33dbb79b4c9f77bce11f503fce08)

Author SHA1 Message Date
wei liu c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 (#34394) 
issue: #34252

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-29 11:31:51 +08:00
XuanYang-cn 314f4d995b
enhance: Tidy dc channel manager (#34515) 
See also: #34518

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-09 18:26:12 +08:00
jaime 21fc5f5d46
enhance: Remove datanode reporting TT based on MQ implementation (#34421) 
issue: #34420

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-05 15:48:09 +08:00
jaime d1f57aa4ba
enhance: remove deprecated code within channel manager (#34340) 
issue: https://github.com/milvus-io/milvus/issues/33994

only remove deprecated code, no additional changes.

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-03 19:46:09 +08:00
jaime d6afb31b94
enhance: make subfunctions of datanode component modular (#33992) 
issue: #33994

also remove deprecated channel manager based on the etcd implementation

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-07-01 14:46:07 +08:00
jaime 9630974fbb
enhance: move rocksmq from internal to pkg module (#33881) 
issue: #33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-25 21:18:15 +08:00
XuanYang-cn 5e39aa9272
enhance: Make channel meta able to writer 200k plus segments (#33279) 
See also: #33125

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-28 12:33:42 +08:00
yiwangdr e895cfed84
fix: reduce redundant map operations in datacoord (#33343) 
More refactories will be added.
issue: #33342

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-05-24 12:47:40 +08:00
yiwangdr b1eacb2ae8
feat: datacoord/node watch based on rpc (#32036) 
issue: https://github.com/milvus-io/milvus/issues/25309

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-05-07 15:49:30 +08:00
yiwangdr 037de8e4d3
enhance: speed up minor functions calls in datacoord (#32389) 
Related to https://github.com/milvus-io/milvus/issues/32165

1. nodeid based channel store access should use map access instead of
iteration.

2. The join-ish functions calls are slow when # collections/segments
increases (e.g. 10k).
e.g.
getNumRowsOfCollectionUnsafe is O(num_segments); GetAllCollectionNumRows
is of O(num_collections*num_segments).

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-04-20 07:55:21 +08:00
congqixia 83da08c388
enhance: Use map instead of slice to maintain channel info (#32273) 
See also #32165

`ChannelManager.Match` is a frequent operation for datacoord. When the
collection number is large, iteration over all channels will cost lots
of CPU time and time consuming.

This PR change the data structure storing datanode-channel info to map
avoiding this iteration when checking channel existence.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-16 15:57:19 +08:00
XuanYang-cn a153950b10
Change channel to Interface (#27839) 
This PR changes `*channel` into RWChannel interface

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-13 11:16:18 +08:00
xige-16 33c2012675
Add more metrics (#25081) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-26 17:52:44 +08:00
congqixia ba84f52119
Fix watcher loop quit and channel shouldDrop logic (#23402) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-14 09:54:28 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
zhenshan.cao 1287ca699a
Refine usage of TimeRecorder.Record (#23142) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-30 18:56:22 +08:00
MrPresent-Han 77c9e33e70
support dml channel balancer on datacoord (#22324) (#22377) (#22692) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-03-20 10:01:56 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
congqixia 53782ab83e
Fill collection schema after DataCoord restarts (#21161) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-12 19:05:21 +08:00
Enwei Jiao 89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
SimFG 9c6436d72d
Add time log for methods of starting the node (#20313) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-08 20:13:03 +08:00
wayblink 43ae6791e6
Make VChannelInfo read compatible with 2.0.2 (#17995) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-07-07 21:58:22 +08:00
bigsheeper 230404499d
Check duplication before adding channel info in DataCoord (#17742) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-24 16:30:15 +08:00
Xiaofan 6426e3d037
Split Etcd save operation into small txns (#17386) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-08 15:08:07 +08:00
Nemo 3eb074a9b0
Update some comments for data coord with some minor code refactoring. (#15686) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-02-22 13:15:51 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
xiangzhou.guo a87e690d53
Correct spelling error in code comment (#13413) 
Signed-off-by: xiangzhou.guo <xiangzhou.guo@zilliz.com>
 2021-12-15 15:59:17 +08:00
xiangzhou.guo 78d583a3bb
Correct spelling error (#13412) 
Signed-off-by: xiangzhou.guo <xiangzhou.guo@zilliz.com>
 2021-12-15 15:57:09 +08:00
Xiangyu Wang 405b3cd932
Use the same lint rules with golint in revive (#13324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-14 15:31:07 +08:00
edward.zeng 260d3e124d
[skip ci] Refine datacoord channel store comment (#12991) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-08 17:57:05 +08:00
congqixia 06ced4d1d1
Add DropVirtualChannel for DataCoord (#12361) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-29 22:35:41 +08:00
congqixia 54738d2ac5
[skip ci] Fix lint for datacoord/channel_store.og (#10964) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-30 21:08:00 +08:00
congqixia 5c0b5c42ca
Fix merge caused compile error (#10693) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-26 21:46:21 +08:00
sunby 7fbb469081
Change channel's field to upper case to log details (#10567) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-26 20:15:41 +08:00
sunby 9b8e1c657e
Split watch operations to many transactions (#10655) 
issue: #10633
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-26 19:38:20 +08:00
XuanYang-cn acb4dd2069
[skip ci]Update license for dc channel manager (#10591) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-25 20:19:25 +08:00
congqixia b028dc3918
[skip ci] Fix golint issue for datacoord/channel_store.go (#9995) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-15 20:44:38 +08:00
sunby 003a7ed7cc
[skip ci]Add comment for MarshalLogObject (#9920) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 21:26:33 +08:00
sunby 3a6f6d399f
[skip ci]Add comment for ChannelOpTypeNames (#9919) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 21:24:41 +08:00
sunby 044b06421a
Use etcd to watch channel on datanode (#7115) 
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 15:44:34 +08:00