d16320705e
enhance: [2.4] Add Segment Level in milvus segment info APIs ( #34763 ) ( #35023 )
...
Cherry-pick from master
pr: #34763
See also #34746
This PR add segment level field in response of
`GetPersistentSegmentInfo` and `GetQuerySegmentInfo`
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-29 10:11:52 +08:00
48063252b1
fix: [Cheryy-Pick] evict paramtable cache failed ( #34772 )
...
relate: https://github.com/milvus-io/milvus/issues/33461
pr: https://github.com/milvus-io/milvus/pull/34771
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-23 12:25:44 +08:00
15adb2feac
enhance: [2.4] add the seal segment when dispatch delete msgs ( #34566 )
...
/kind improvement
- pr: #34565
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-10 19:18:51 +08:00
ddb1015563
fix: [2.4] Check err is ErrKeyNotFound when CASCachedValue ( #34489 )
...
Cherry-pick from master
pr: #34488
See also #33785
When config item is not present in paramtable, CAS fails due to
GetConfig returns error.
This PR make this returned err instance of ErrKeyNotFound and check
error type in \`CASCachedValue\` methods.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-09 18:12:13 +08:00
62bd51ec6d
fix: [2.4] Compare config value then swap when caching param value ( #33785 ) ( #33797 )
...
Cherry-pick from master
pr: #33785
See also #33784
This PR change the behavior of `SetCacheValue` of config manager:
- Use mutex and map instead of concurrent map for `configCache`
- Compare config raw value before set cache value
With this implementation, concurrent caching & eviction shall always
have current output:
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 |get | |old value|null |
|t1 |CAS OK | |old value|old value|
|t2 | |update |new value|old value|
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 |get | |old value|null |
|t1 | |update |new value|null |
|t2 |CAS fail| |old value|null |
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 |get | |new value|null |
|t2 |CAS OK | |new value|new value|
|t3 | |eviction|new value|null |
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 |get | |new value|null |
|t2 | |eviction|new value|null |
|t3 |CAS OK | |new value|new value|
|time|caching |eviction|config |cached |
|----|--------|------- |---------|---------|
|t0 | |update |new value|null |
|t1 | |eviction|new value|null |
|t2 |get | |new value|null |
|t3 |CAS OK | |new value|new value|
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-13 17:51:57 +08:00
aaf6c85095
fix: paramtable cache cause dynamic config non-dynamic ( #33473 ) ( #33590 )
...
relate: https://github.com/milvus-io/milvus/issues/33461
pr: https://github.com/milvus-io/milvus/pull/33473
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-06-05 10:15:51 +08:00
b6fefee0cf
fix: etcd not connectable when auth enabled ( #31633 )
...
Fix etcd config source didn't respect auth enabled
Also removed pulsar recoverable error when pulsar return ConsumerBusy.
It could happen that pulsar didn't find the original consumer is dead
and recover takes some time.
fix #31631
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-04-01 15:23:19 +08:00
74b7de3814
enhance: Cache formatted key for param item ( #31388 )
...
See also #30806
`formatKey` may cost lots of CPU on string processing under high QPS
scenario, this PR adds a formattedKeys cache preventing string operation
in each param get value.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-19 14:05:05 +08:00
b724753137
enhance: Add runtime config to paramtable ( #31006 )
...
relate: https://github.com/milvus-io/milvus/issues/30806
Avoid use string convert or format function when get some runtime
parameter
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-03-15 11:07:06 +08:00
3298e64bd3
enhance: cache config values for saving cpu cycles to parse config item ( #30947 )
...
related: #30958
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-03-12 11:09:04 +08:00
bbff9193d9
enhance: support clean paramtable config event in test ( #30534 )
...
relate: https://github.com/milvus-io/milvus/issues/30441
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-02-20 14:16:51 +08:00
8385157717
enhance: adjust config source for support config event use paramtable ( #29995 )
...
Adjust config source for support config event which for dynamic config
could use paramtable and not deadlock.
relate: https://github.com/milvus-io/milvus/issues/29807
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-01-26 19:07:00 +08:00
d73b534f1e
fix: use atomic.Pointer to store EventHandler in case of data race ( #30205 )
...
Resolves #30204
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-23 19:09:04 +08:00
797847904c
enhance: Change some frequency log to rated level ( #29720 )
...
This PR change some frequency log to rated level
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-11 16:30:50 +08:00
1f759837c4
fix: remove Unnecessary lock in config manager ( #29836 )
...
issue: #29709 #291712
to avoid concurrent recursive RLock and Lock cause deadlock, This PR
remove the unnecessary lock in config manager
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-01-11 13:48:49 +08:00
ed644983e2
enhance: add param for bloomfilter( #29388 ) ( #29490 )
...
related: #29388
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-28 18:10:46 +08:00
89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. ( #28319 )
...
relate: https://github.com/milvus-io/milvus/issues/28086
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-28 15:32:31 +08:00
f1fc19e8a9
enhance: Add unittest for `config.EventDispatcher` ( #28552 )
...
These is no enough unittest case for EventDispatcher, see also #28540 .
This PR add unit test case for all methods for EventDispatcher
Related to #28538
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-18 19:12:20 +08:00
c948a437a9
fix: protect `EventDispatcher` map with mutex ( #28540 )
...
Add mutex protection for `EventDispatcher.registry` map
Fix #28538
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-17 20:32:20 +08:00
7dda2e8814
Change some log level in the pkg package ( #28181 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-08 23:34:22 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
fb0705df1b
Decouple basetable and componentparam ( #26725 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
c3f5856fbc
Fix data race for config with FileSource ( #26518 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-24 09:18:24 +08:00
0bc03ede0d
Add eventlog pkg and support grpc streaming event observation ( #25812 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-25 17:23:01 +08:00
de6e4817a2
Support dynamic tuning config ( #25152 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-07-03 15:18:24 +08:00
ebd0279d3f
Check error by Error() and NoError() for better report message ( #24736 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
8e3ba74648
fix qc service unstable ut ( #24340 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-05-24 18:49:25 +08:00
7e9ef36de4
Fix TestConfigFromRemote/close_manager ut is not stable ( #24279 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-22 09:59:25 +08:00
3a66e1de65
Use suite for integration test ( #24253 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-19 15:57:24 +08:00
de4b4dafef
Refresh etcd source config with Searializable option ( #23954 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-09 11:52:40 +08:00
086f3bd748
Add it for refresh config ( #23773 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-06 17:34:39 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
congqixia
aoiasd
SimFG
Xiaofan
Chun Han
wei liu
Enwei Jiao
yah01
Gao
yah01
jaime