milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
20,062 Commits
133 Branches
115 Tags
559 MiB
Commit Graph

29 Commits (32f685ff1283b37bb7f65ad4c357ccbd2b58e1f4)

Author SHA1 Message Date
congqixia c689ef4822
fix: Remove eviction operations out of lock (#33834) 
See also #33823

`EvictCacheValueByFormat` may be block by on going `CASCacheValue` and
cause possible deadlock

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-13 21:15:55 +08:00
congqixia b90999b741
fix: Compare config value then swap when caching param value (#33785) 
See also #33784

This PR change the behavior of `SetCacheValue` of config manager:

- Use mutex and map instead of concurrent map for `configCache`
- Compare config raw value before set cache value

With this implementation, concurrent caching & eviction shall always
have current output:

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |get     |        |old value|null     |
|t1  |CAS OK  |        |old value|old value|
|t2  |        |update  |new value|old value|
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |get     |        |old value|null     |
|t1  |        |update  |new value|null     |
|t2  |CAS fail|        |old value|null     |
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |get     |        |new value|null     |
|t2  |CAS OK  |        |new value|new value|
|t3  |        |eviction|new value|null     |

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |get     |        |new value|null     |
|t2  |        |eviction|new value|null     |
|t3  |CAS OK  |        |new value|new value|

|time|caching |eviction|config   |cached   |
|----|--------|------- |---------|---------|
|t0  |        |update  |new value|null     |
|t1  |        |eviction|new value|null     |
|t2  |get     |        |new value|null     |
|t3  |CAS OK  |        |new value|new value|

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-06-12 18:51:56 +08:00
aoiasd 2422084a29
fix: paramtable cache cause dynamic config non-dynamic (#33473) 
relate: https://github.com/milvus-io/milvus/issues/33461

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-06-04 11:39:46 +08:00
Xiaofan b6fefee0cf
fix: etcd not connectable when auth enabled (#31633) 
Fix etcd config source didn't respect auth enabled
Also removed pulsar recoverable error when pulsar return ConsumerBusy.
It could happen that pulsar didn't find the original consumer is dead
and recover takes some time.
fix #31631

Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
 2024-04-01 15:23:19 +08:00
congqixia 74b7de3814
enhance: Cache formatted key for param item (#31388) 
See also #30806

`formatKey` may cost lots of CPU on string processing under high QPS
scenario, this PR adds a formattedKeys cache preventing string operation
in each param get value.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-03-19 14:05:05 +08:00
aoiasd b724753137
enhance: Add runtime config to paramtable (#31006) 
relate: https://github.com/milvus-io/milvus/issues/30806
Avoid use string convert or format function when get some runtime
parameter

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-03-15 11:07:06 +08:00
Chun Han 3298e64bd3
enhance: cache config values for saving cpu cycles to parse config item (#30947) 
related: #30958

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2024-03-12 11:09:04 +08:00
aoiasd bbff9193d9
enhance: support clean paramtable config event in test (#30534) 
relate: https://github.com/milvus-io/milvus/issues/30441

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-02-20 14:16:51 +08:00
aoiasd 8385157717
enhance: adjust config source for support config event use paramtable (#29995) 
Adjust config source for support config event which for dynamic config
could use paramtable and not deadlock.
relate: https://github.com/milvus-io/milvus/issues/29807

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-01-26 19:07:00 +08:00
congqixia d73b534f1e
fix: use atomic.Pointer to store EventHandler in case of data race (#30205) 
Resolves #30204

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-01-23 19:09:04 +08:00
wei liu 797847904c
enhance: Change some frequency log to rated level (#29720) 
This PR change some frequency log to rated level

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-01-11 16:30:50 +08:00
wei liu 1f759837c4
fix: remove Unnecessary lock in config manager (#29836) 
issue: #29709 #291712
to avoid concurrent recursive RLock and Lock cause deadlock, This PR
remove the unnecessary lock in config manager

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-01-11 13:48:49 +08:00
MrPresent-Han ed644983e2
enhance: add param for bloomfilter(#29388) (#29490) 
related: #29388

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-12-28 18:10:46 +08:00
aoiasd 89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. (#28319) 
relate: https://github.com/milvus-io/milvus/issues/28086

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-28 15:32:31 +08:00
congqixia f1fc19e8a9
enhance: Add unittest for `config.EventDispatcher` (#28552) 
These is no enough unittest case for EventDispatcher, see also #28540. 
This PR add unit test case for all methods for EventDispatcher
Related to #28538

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-18 19:12:20 +08:00
congqixia c948a437a9
fix: protect `EventDispatcher` map with mutex (#28540) 
Add mutex protection for `EventDispatcher.registry` map 
Fix #28538

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-17 20:32:20 +08:00
SimFG 7dda2e8814
Change some log level in the pkg package (#28181) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-08 23:34:22 +08:00
SimFG 26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
Enwei Jiao fb0705df1b
Decouple basetable and componentparam (#26725) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 10:31:48 +08:00
yah01 c3f5856fbc
Fix data race for config with FileSource (#26518) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-24 09:18:24 +08:00
congqixia 0bc03ede0d
Add eventlog pkg and support grpc streaming event observation (#25812) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-25 17:23:01 +08:00
Gao de6e4817a2
Support dynamic tuning config (#25152) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-07-03 15:18:24 +08:00
yah01 ebd0279d3f
Check error by Error() and NoError() for better report message (#24736) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-08 15:36:36 +08:00
wei liu 8e3ba74648
fix qc service unstable ut (#24340) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-24 18:49:25 +08:00
congqixia 7e9ef36de4
Fix TestConfigFromRemote/close_manager ut is not stable (#24279) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-22 09:59:25 +08:00
congqixia 3a66e1de65
Use suite for integration test (#24253) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-19 15:57:24 +08:00
congqixia de4b4dafef
Refresh etcd source config with Searializable option (#23954) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-09 11:52:40 +08:00
Enwei Jiao 086f3bd748
Add it for refresh config (#23773) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-05-06 17:34:39 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00