Commit Graph

24 Commits (lite)

Author SHA1 Message Date
jaime 1639779b5d
fix: [skip e2e]metrics josn tag unmarshall failed ()
issue: 

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-18 18:02:46 +08:00
jaime 4d5572d7ed
enhance: [skip e2e] refine display format ()
issue: 

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-17 22:30:44 +08:00
jaime 8ed019735c
enhance: add disk stats within system metrics ()
issue: ##36621

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-06 16:32:41 +08:00
jaime 60dd55f292
enhance: add end time stats for compaction and index build tasks ()
issue: 

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-06 16:28:40 +08:00
jaime 7bbfe86bcd
enhance: add list index and segment index retrieval API for WebUI ()
issue: 

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-22 16:58:34 +08:00
jaime 12ed40e125
fix: some missing metrics for WebUI restful API ()
issue: 

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-18 14:00:32 +08:00
jaime 1d06d4324b
fix: Int64 overflow in JSON encoding ()
issue: ##36621

- For simple types in a struct, add "string" to the JSON tag for
automatic string conversion during JSON encoding.
- For complex types in a struct, replace "int64" with "string."

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-14 22:52:30 +08:00
jaime 1e8ea4a7e7
feat: add segment/channel/task/slow query render ()
issue: 

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-12 17:44:29 +08:00
jaime f348bd9441
feat: add segment,pipeline, replica and resourcegroup api for WebUI ()
issue: 

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-07 11:52:25 +08:00
jaime 9d16b972ea
feat: add tasks page into management WebUI ()
issue: 

1. Add API to access task runtime metrics, including:
  - build index task
  - compaction task
  - import task
- balance (including load/release of segments/channels and some leader
tasks on querycoord)
  - sync task
2. Add a debug model to the webpage by using debug=true or debug=false
in the URL query parameters to enable or disable debug mode.

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-28 10:13:29 +08:00
jaime 4746f47282
feat: management WebUI homepage ()
issue: 
1. Implement an embedded web server for WebUI access.  
2. Complete the homepage development.

Home page demo:
<img width="2177" alt="iShot_2024-10-10_17 57 34"
src="https://github.com/user-attachments/assets/38539917-ce09-4e54-a5b5-7f4f7eaac353">

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-23 11:29:28 +08:00
congqixia 8593c4580a
enhance: Add delete buffer related quota logic ()
See also 

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-05 11:39:03 +08:00
jaime 24fb10114b
enhance: remove cooling off in rate limiter for read requests ()
issue: 

Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-04 14:39:10 +08:00
congqixia 67324eb809
enhance: Add l0 segment entry num quota ()
See also 

This PR add quota configuration for l0 segment entry number per
collection. If l0 compaction cannot keep up the insertion/upsertion
rate, this feature could back press the related rate.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-17 17:35:41 +08:00
SimFG c012e6786f
feat: support rate limiter based on db and partition levels ()
issue: https://github.com/milvus-io/milvus/issues/30577
co-author: @jaime0815

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-04-12 16:01:19 +08:00
SimFG b1a1cca10b
feat: add more operation detail info for better allocation ()
issue: 

---------

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-03-28 06:33:11 +08:00
Jiquan Long dc2cdbe387
enhance: add more metrics ()
/kind improvement
fix:  

This pr add more metrics, which are:
- Slow query count, which the duration considered as slow can be
configurable;
- Number of deleted entities;
- Number of entities imported;
- Number of entities per collection;
- Number of loaded entities per collection;
- Number of indexed entities;
- Number of indexed entities, per collection, per index and whether it's
a vetor index;
- Quota states (LongTimeTickDelay, MemoryExhuasted, DiskQuotaExhuasted)
per database;

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-03-19 15:23:06 +08:00
SimFG 26f06dd732
Format the code ()
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
congqixia 41af0a98fa
Use go-api/v2 for milvus-proto ()
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01 ebd0279d3f
Check error by Error() and NoError() for better report message ()
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
yihao.dai 7384d83d2c
Support rate limit based on growing segments size ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-17 09:57:22 +08:00
yihao.dai ed8836cd15
Add disk quota at the collection level ()
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-26 21:52:36 +08:00
wei liu 4fb8919a97
support collection level rate limit ()
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-25 15:54:35 +08:00
jaime c9d0c157ec
Move some modules from internal to public package ()
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00