875ad88d84
enhance: Support access log use stdout with write cache and auto flush as time ( #32213 )
...
relate: https://github.com/milvus-io/milvus/issues/28948
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-16 10:05:34 +08:00
9a56cba7af
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-05-15 12:00:50 +00:00
dd9919a7dc
fix: two-phase retrieval on lru-segment ( #32945 )
...
issue: #31822
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-15 17:53:34 +08:00
8cf2cf5c94
enhance: Add `go-deadlock` as unittest only dependency ( #33063 )
...
See also #33062
This PR:
- Add `lock.RWMutex` & `lock.Mutex` alias to switch implementation based
on build flags
- When build flags has `test` in it, use `go-deadlock` to detect
possible deadlocks
- Replace all `sync.RWMutex` & `sync.Mutex` in datacoord pkg
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-15 16:33:34 +08:00
7c60d725cc
fix: validate sparse vector in search request ( #32856 )
...
issue: #32368
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-15 15:39:33 +08:00
4382cf5283
enhance: Reduce the frequency of index ralted logs ( #33050 )
...
issue: #33001
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-05-15 14:37:34 +08:00
b2d83d3354
enhance: Bump milvus version to v2.4.2 ( #33048 )
...
Bumping version to v2.4.2. Also bump milvus-proto version to v2.4.3.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-15 13:57:35 +08:00
259bc97a2b
fix: Fix segments lost in flush response ( #33061 )
...
#33055
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-05-15 13:49:34 +08:00
6ea7633bd5
enhance: Add memory size for binlog ( #33025 )
...
issue: #33005
1. add `MemorySize` field for insert binlog.
2. `LogSize` means the file size in the storage object.
3. `MemorySize` means the size of the data in the memory.
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2024-05-15 12:59:34 +08:00
d4837307b3
fix: Make submit idempotent ( #33053 )
...
issue: #33054
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-14 22:09:34 +08:00
6fc00e42d7
fix: Fix binlog import ( #33027 )
...
binlog import should only import into one partition.
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-14 21:45:32 +08:00
3777eb2f68
enhance: update knowhere new version ( #33043 )
...
for milvus new release, knowhere update to latest release version
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-05-14 20:45:32 +08:00
f48a7ff8ff
enhance: use Delete instead of DeletePartialMatch to remove metrics ( #33029 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-05-14 18:49:33 +08:00
89a7c34c7a
fix: exceed etcd limit ( #33041 )
...
#32974
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-05-14 18:45:32 +08:00
dea7faf394
fix: Remove `RLock` re-entry in `GetNumRowsOfPartition` ( #33045 )
...
Resolves #33044
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-14 17:51:32 +08:00
3d85e6e028
fix: etcd txn exceeds limit due to too many fields ( #33040 )
...
fix : #33038
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-14 17:05:33 +08:00
ba3b2a91a0
test: Remove useless common types and refine error assert in negative cases ( #33023 )
...
Related issue: #32653
1. Remove some meaningless common types
2. Refine error assertion in negative cases
3. Remove some dup tests
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-05-14 15:03:33 +08:00
1d48d0aeb2
enhance: use different value to get related data size according to segment type ( #33017 )
...
issue: #30436
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-14 14:59:33 +08:00
4fc7915c70
enhance: unify data generation test APIs ( #32955 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-14 14:33:33 +08:00
96489b814d
fix: remove busy log ( #33042 )
...
issue: #32963
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-14 14:20:32 +08:00
861977ab60
fix: Start `LeaderCacheObserver` before `SyncAll` ( #33035 )
...
Related to #33033
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-14 13:25:32 +08:00
40e5c0be70
fix: Enable to balance released standby channel ( #32986 )
...
See also: #32879
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-14 11:51:32 +08:00
dc058eaf61
fix: should init access logger before init server ( #32976 )
...
relate: https://github.com/milvus-io/milvus/issues/32968
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-14 10:47:32 +08:00
3871009cf5
test: add timeout for GetMetrics grpc interface ( #33024 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-14 10:29:32 +08:00
4ae7cabb04
fix: Remove channel when create flowgraph timeout ( #33014 )
...
See also #33013
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-14 10:07:33 +08:00
cba2c7a3be
enhance: clean channel node info in meta store ( #32988 )
...
issue: #32910
see also: #32911
when channel exclusive mode is enabled, replica will record channel node
info in meta store, and if the balance policy changes, which means
channel exclusive mode is disabled, we should clean up the channel node
info in meta store, and stop to balance node between channels.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-14 10:05:40 +08:00
a984e46a29
enhance: Remove rootcoord from datanode broker ( #32818 )
...
issue: https://github.com/milvus-io/milvus/issues/32827
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-14 10:03:32 +08:00
09cca1fb51
[automated] Update Knowhere Commit ( #33030 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-14 01:47:32 +08:00
9e3f3d99f7
test: add timeout for import testcases ( #33006 )
...
add timeout for import testcases
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-05-13 19:35:32 +08:00
efdbd8e7c1
enhance: Enable to upload by batch ( #32788 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-13 19:07:32 +08:00
29b621f759
fix: Make compactor able to clear empty segments ( #32821 )
...
See also: #32553
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-05-13 18:21:32 +08:00
ba625835bc
enhance: Add metrics for segment index files size ( #32979 )
...
issue:#32980
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-05-13 17:59:32 +08:00
293f14a8b9
fix: remove redundant replica recover ( #32985 )
...
issue: #22288
- replica recover should be only triggered by replica recover
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-13 15:25:32 +08:00
f6777267e3
enhance: add score compute consistency config for knowhere ( #32997 )
...
issue: https://github.com/milvus-io/milvus/issues/32583
related: #32584
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-05-13 14:21:31 +08:00
4031abd2fa
enhance: change default partition num to 16 when using partition key ( #32950 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-13 14:19:31 +08:00
12ec3d61d9
fix: Fill deltalog entry num & time range in L0 compactions ( #33004 )
...
Resolves #33003
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-13 14:13:31 +08:00
4724779b3b
enhance: remove fallback keys for config generator ( #32946 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-13 13:33:31 +08:00
230c88b1de
enhance: improve reduce performance on very large topk ( #32871 )
...
fix #32870
reduce cpu usage for reduce on large k
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-13 11:55:31 +08:00
ef41f80bd0
enhance: Sync go.mod version & dep to main pkg ( #32973 )
...
Related to #31293
See also #32951 #32952 #32953 #32954
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-13 10:25:32 +08:00
0e5765b116
enhance: Utilize `TestLocations` ability to accelerate write & compaction ( #32948 )
...
See also #32642
This PR reuses hash locations for bloom filter prediction utilizing
`storage.Location`, like enhancement #32642 .
Also adds a utility struct in storage: `LocationCache` to storage
locations for variable K (numbers of hash functions)
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-13 10:15:32 +08:00
8a9a42198d
fix: Proxy crash due to shard leader cache data race ( #32971 )
...
issue: #32970
cause InvalidateShardLeaderCache use wrong lock, which may cause data
race in meta cache, then proxy may crash
This PR fixed that use leaderMut when try to access shard leader cache.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-11 14:32:12 +08:00
36f1ea93a5
enhance: optimize plan parser pool to avoid unnessary recycle ( #32869 )
...
fix #32868
plan parser takes too much cpu on high qps,this pr try to avoid create
lexer and parser too freequent
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-11 10:51:31 +08:00
b044e5503e
enhance:Improve load speed ( #32898 )
...
fix #32897
add memory check when load collection
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-05-11 10:29:31 +08:00
c0e62e6629
enhance: Use collection default consistency level for restv2 ( #32956 )
...
Set `UseDefaultConsistency` to true so that restv2 read API shall use
collection consistency level setting correctly.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-11 00:47:31 +08:00
4577394f4f
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-05-10 12:01:00 +00:00
1c84a1c9b6
fix: lru related issue fixup patch ( #32916 )
...
issue: #32206 , #32801
- search failure with some assertion, segment not loaded and resource
insufficient.
- segment leak when query segments
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-05-10 19:17:30 +08:00
25689859a1
fix: Load index metric use wrong time unit ( #32935 )
...
issue:#32899
This PR fix the wrong metric value of load index, which introduced by
pr#32567, use wrong time unit for load index metrics
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-10 18:07:30 +08:00
244d2c04f6
feat: Add `milvusclient` package and migrate GoSDK ( #32907 )
...
Related to #31293
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-10 18:01:30 +08:00
855192eb3d
fix: sync milvus.yaml ( #32920 )
...
issue: https://github.com/milvus-io/milvus/issues/25309
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-05-10 17:29:31 +08:00
e2332bdc17
enhance: Enable channel exclusive balance policy ( #32911 )
...
issue: #32910
* split replica's node list to channels when create replicas
* balance nodes among channels when node change happens
* implement channel level balance, let balance happens in channel level
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-05-10 17:27:31 +08:00
aoiasd
sre-ci-robot
Jiquan Long
congqixia
Buqian Zheng
cai.zhang
wayblink
XuanYang-cn
yihao.dai
Gao
jaime
smellthemoon
yanliang567
SimFG
Cai Yudong
chyezh
zhuwenxing
wei liu
sre-ci-robot
foxspy
Bingyi Sun
Xiaofan
yiwangdr