milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
21,796 Commits
153 Branches
122 Tags
592 MiB
Commit Graph

142 Commits (93212a9a6307c56cb51cf4c757228be42f79547e)

Author SHA1 Message Date
aoiasd 24d2bbc441
enhance: unmashall ts msg in dispatcher instead in msgstream (#38656) 
relate: https://github.com/milvus-io/milvus/issues/38655

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2025-02-14 12:04:13 +08:00
tinswzy 27229f7907
enhance: refine exists log print with ctx (#38080) 
issue: #35917 
Refines exists log print with ctx

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
 2024-12-14 22:36:44 +08:00
tinswzy 5768dbbb5d
enhance: refine pular related mq interfaces (#38007) 
issue: #35917 
Refines the pulsar-related mq APIs to allow the ctx to be passed down

Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
 2024-12-04 20:50:39 +08:00
jaime f348bd9441
feat: add segment,pipeline, replica and resourcegroup api for WebUI (#37344) 
issue: #36621

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-11-07 11:52:25 +08:00
SimFG c94b69c2f6
enhance: update the expr version and format the expr http response (#36406) 
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-09-26 14:27:20 +08:00
congqixia de8a266d8a
enhance: Enable linux code checker (#35084) 
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-07-30 15:53:51 +08:00
chyezh 39c7e06bc5
enhance: add message and msgstream msgpack adaptor (#34874) 
issue: #33285

- make message builder and message conversion type safe
- add adaptor type and function to adapt old msgstream msgpack and
message interface

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-07-22 20:59:42 +08:00
yihao.dai 0b404bff22
fix: Correct the update logic of timerecorder (#34339) 
Correct the update logic of timerecorder in the flowgraph to avoid false
failure: "some node(s) haven't received input".

issue: https://github.com/milvus-io/milvus/issues/34337

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-07-04 16:34:17 +08:00
jaime 9630974fbb
enhance: move rocksmq from internal to pkg module (#33881) 
issue: #33956

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-06-25 21:18:15 +08:00
Xiaofan 36cbce4def
enhance: optimize datanode cpu usage under large collection number (#33267) 
fix #33266 
try to improve cpu usage by refactoring the ttchecker logic and caching
string

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-05-25 04:43:41 +08:00
chyezh 5ee9f734c1
fix: Use determined order to lock in BlockAll to avoid deadlock (#29246) 
issue: #29104

Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2024-01-22 14:50:56 +08:00
smellthemoon 1c1f2a1371
enhance:change some logs (#29579) 
related #29588

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-01-05 16:12:48 +08:00
XuanYang-cn 7a6aa8552a
fix: add back existing datanode metrics (#29360) 
See also: #29204

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-22 14:20:43 +08:00
wayblink 6736f65345
feat: skip some empty ttMsg in Datanode flowgraph (#28756) 
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-07 01:00:37 +08:00
wayblink da339535d5
enhance: Merge flowgraph goroutines into 1 (#28654) 
/kind enhancement
#24826

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-11-23 19:52:25 +08:00
smellthemoon 0aa90de141
Reduce the goroutine in flowgraph to 2 (#28233) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-11-13 10:50:17 +08:00
XuanYang-cn 5c5f9aa05e
Enhance newDataSyncService (#27277) 
- Add flowgraph.Assemble assembles nodes in flowgraph.go
- remove fgCtx in newDataSyncService
- Add newServiceWithEtcdTickler func, reduce param numbers to 3
- Remove unnecessary params
  - config.maxQueueLength, config.maxParallelish

See also: #27207

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-27 11:07:25 +08:00
SimFG 26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
yihao.dai 4b2802033d
Fix datanode panic due to concurrent compaction and delete processing (#27167) 
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-09-18 19:35:29 +08:00
Jiquan Long 802a6a86c0
Fix printing too many warnings (#27015) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-12 13:39:17 +08:00
XuanYang-cn 7f1ae35e72
Add timeout in dispatcher, AsConsumer and Seek (#26686) 
See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-08 09:51:17 +08:00
chyezh 85886f039e
[Fixup] msg context access data race when msgdispatcher (#26463) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-08-21 19:16:22 +08:00
XuanYang-cn 08fd28b30b
Only do gracefully stop when DN Stop (#26399) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-08-21 11:16:20 +08:00
XuanYang-cn 84253f255e
Fix datanode graceful stop panic (#25932) 
See also: #25925

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-07-28 10:11:08 +08:00
xige-16 33c2012675
Add more metrics (#25081) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-26 17:52:44 +08:00
congqixia 41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
xige-16 732fe54775
Support partition Key (#24047) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-06 10:24:34 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
jaime d126f06946
Decouple mq module from internal proto definition (#22536) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-03-04 23:21:50 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
bigsheeper f4e7b246f2
Fix msgdispatcher concurrency issue and add prometheus metrics (#22174) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-02-16 14:58:34 +08:00
bigsheeper d2667064bb
Add msgDispatcher to support sharing msgs for different vChannel (#21917) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-02-13 16:38:33 +08:00
Enwei Jiao fb42466c65
Use opentelemetry (#21509) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 16:09:39 +08:00
Xiaofan 4b4944ecee
Support sync all segments while close (#21421) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-06 14:49:36 +08:00
Enwei Jiao 264f08953d
Refactor Rocksmq's config (#21304) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-26 19:11:30 +08:00
jaime 039e9ce4bb
Refine msgstream interface (#20832) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-05 20:55:17 +08:00
jaime 709a0a94e9
Add some msgstream metrics (#20296) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-07 10:15:02 +08:00
Xiaofan 1f170ce1ba
Refine kafka consumer (#19846) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-25 13:23:30 +08:00
Xiaofan 1b5e765307
Block Flowgraph before handle compacted segments (#19988) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-24 15:09:29 +08:00
SimFG a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
Eng Zer Jun 438790744d
test: use `T.Setenv` to set env vars in tests (#19649) 
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2022-10-11 18:43:23 +08:00
Xiaofan db3f4be49d
Simplify flow graph node to pipeline mode (#19667) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-10 22:15:22 +08:00
SimFG d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
SimFG 65fffa6d7d
Make sure that closing the graph is more secure (#19178) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-14 10:32:33 +08:00
SimFG ceea04c274
Improve the `close` method in the graph (#19100) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-09 10:00:36 +08:00
SimFG 50d5d04552
Refine some codes about the datanode (#17720) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-06-23 09:58:14 +08:00
bigsheeper 48cf63a2d7
Wait for flow graph to exit (#17341) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-02 19:48:04 +08:00
godchen bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
xiyichan ca129d4308
Support configurable msgstream (#16131) 
Signed-off-by: xiyichan <2863768433@qq.com>
 2022-03-24 10:15:25 +08:00
jaime 29975a7a26
Remove Consume() API from mqstream interface (#15886) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-11 20:09:59 +08:00