milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
21,884 Commits
170 Branches
145 Tags
578 MiB
Commit Graph

34 Commits (eb046863485fdf3e130fc60484485c901b81276b)

Author SHA1 Message Date
congqixia cb7f2fa6fd
enhance: Use v2 package name for pkg module (#39990) 
Related to #39095

https://go.dev/doc/modules/version-numbers

Update pkg version according to golang dep version convention

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2025-02-22 23:15:58 +08:00
yihao.dai 657550cf06
fix: Fix slow dist handle and slow observe (#38566) 
1. Provide partition&channel level indexing in the collection target.
2. Make `SegmentAction` not wait for distribution.
3. Remove scheduler and target manager mutex.
4. Optimize logging to reduce CPU overhead.

issue: https://github.com/milvus-io/milvus/issues/37630

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2025-01-15 20:17:00 +08:00
Zhen Ye bb8d1ab3bf
enhance: make new go package to manage proto (#39114) 
issue: #39095

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2025-01-10 10:49:01 +08:00
congqixia b0bd290a6e
enhance: Use internal json(sonic) to replace std json lib (#37708) 
Related to #35020

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-18 10:46:31 +08:00
jaime 1d06d4324b
fix: Int64 overflow in JSON encoding (#37657) 
issue: ##36621

- For simple types in a struct, add "string" to the JSON tag for
automatic string conversion during JSON encoding.
- For complex types in a struct, replace "int64" with "string."

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-11-14 22:52:30 +08:00
jaime 9d16b972ea
feat: add tasks page into management WebUI (#37002) 
issue: #36621

1. Add API to access task runtime metrics, including:
  - build index task
  - compaction task
  - import task
- balance (including load/release of segments/channels and some leader
tasks on querycoord)
  - sync task
2. Add a debug model to the webpage by using debug=true or debug=false
in the URL query parameters to enable or disable debug mode.

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-10-28 10:13:29 +08:00
Chun Han ca7ef26e4b
fix: sync part stats task cannot be finished(#30376) (#34027) 
related: #30376
also: refine log output for query_coord task by rephrasing action string

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-06-24 10:16:02 +08:00
wayblink a1232fafda
feat: Major compaction (#33620) 
#30633

Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
 2024-06-10 21:34:08 +08:00
chyezh 9f9ef8ac32
enhance: transfer resource group and dbname to querynode when load (#30936) 
issue: #30931

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-03-21 11:59:12 +08:00
wei liu 7c7375031d
enhance: Add metrics for task latency in querycoord scheduler (#31405) 
This PR add metrics for task latency in querycoord scheduler, so if any
kind of task stuck, it's easy to figure out by metrics

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-03-20 19:29:06 +08:00
congqixia 7b91fa3db8
fix: Make leader checker generate leader task instead of segment task (#30258) 
See also #30150

For leader view distribution with offline nodes, a release task can
never be sent to querynode due to targetNode online check logic. Even
the request is dispatched, normal release task does not have "force"
flag when calling `delegator.ReleaseSegment`.

This PR adds a new type of querycoord task: LeaderTask, the
responsibility of which is to rectify leader view distribtion.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-02-21 11:08:51 +08:00
congqixia c4ddfff2a7
enhance: make Load process traceable in querycoord (#29806) 
See also #29803

This PR:
- Add trace span for collection/partition load
- Use TraceSpan to generate Segment/ChannelTasks when loading
- Refine BaseTask trace tag usage

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-01-10 09:58:49 +08:00
yah01 70995383bf
enhance: modify log to avoid ambiguity and improve readability (#28331) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-10 14:32:20 +08:00
congqixia 852be152de
Change task sourceID to stringer interface (#27965) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-27 01:08:12 +08:00
smellthemoon a0ca982a52
Fix typo in priority name (#27558) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-10-11 14:19:35 +08:00
SimFG 26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
congqixia edde3cf1c7
Add tracer for querycoord tasks (#27058) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-14 09:59:19 +08:00
congqixia 065d1a962e
Add sourceID output for task.String and fill reduce channel reason (#26447) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-18 13:50:19 +08:00
yah01 2180ef180c
Record only failed task error (#26033) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-08-01 10:11:05 +08:00
congqixia 2b9ec565bb
Fix task executor can not dedup same task (#25901) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-26 08:51:02 +08:00
congqixia efcaa07fe9
Implement Stringer for task related enums (#25860) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-24 13:57:00 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
yah01 3d8f0156c7
Refine scheduler & executor of QueryCoord (#22761) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-16 17:43:55 +08:00
yah01 21ba8182ee
Refine task create errors (#22745) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-14 18:51:53 +08:00
yah01 d4fccdd135
Add reason for spawning task (#22549) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-06 10:13:50 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
yah01 060649b8aa
Refactor task scheduler and executor (#20828) 
Make the performance able to scale out

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-30 13:57:15 +08:00
wei liu eaa5cfdcb5
fix leader observer sync logic (#20478) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-11 11:43:06 +08:00
wei liu 4412cfcaaf
reduce querycoord unnecessary panic (#19925) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-10-28 17:15:32 +08:00
yah01 1c71844b8d
Add license header (#19678) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-11 11:39:22 +08:00
yah01 e6c7286bee
Remove task timeout (#19583) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-30 19:48:55 +08:00
yah01 377f856833
Fix balance may confuse leader observer (#19435) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-28 12:10:54 +08:00
yah01 6ba52366c5
Fix load segments can't be retried (#19414) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-09-26 10:54:52 +08:00
Bingyi Sun 626854cf0c
Refactor QueryCoord (#18836) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-15 18:48:32 +08:00