ed644983e2
enhance: add param for bloomfilter( #29388 ) ( #29490 )
...
related: #29388
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-28 18:10:46 +08:00
0a70e8b601
enhance: Remove multiple vector field limit ( #27827 )
...
issue: https://github.com/milvus-io/milvus/issues/25639
/kind improvement
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-12-28 16:40:46 +08:00
4b406e5973
enhance: Add CompactionTaskNum metrics ( #29518 )
...
See also: #27606
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-28 15:46:55 +08:00
8a4c0d4a3f
enhance: compile go assembly with build constraints for different arch ( #29515 )
...
issue #28657
follow https://pkg.go.dev/cmd/go#hdr-Build_constraints to compile go
assembly with different cpu arch
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-12-28 15:40:47 +08:00
8a630f733a
enhance: add new optimize param for queryhook ( #29495 )
...
add a flag to indicate if we use search param optimizations, default is
on
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-12-28 10:04:46 +08:00
fe04598900
enhance: Add compaction type label to metrics ( #29485 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-27 15:56:48 +08:00
839a72129e
fix: Auto balance param can't be updated by dynamic ( #29501 )
...
This PR fixed that auto balance param can't be updated by dynamic
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-27 14:30:53 +08:00
a76e3b2813
Refine delete by expression for forbid proxy dml task scheduler hang ( #29340 )
...
relate: https://github.com/milvus-io/milvus/issues/29146
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 19:52:48 +08:00
7c7003bff6
enhance:refine the range of chunk size config value( #29388 ) ( #29389 )
...
related: #29388
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-26 17:36:46 +08:00
9e6da45497
fix: Use uber atomic instead sync/atomic which only supported after go v1.20 ( #29377 )
...
relate: https://github.com/milvus-io/milvus/issues/29376
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 16:06:49 +08:00
4c979538a4
enhance: update cagra index params in config and add params check ( #29045 )
...
issue:https://github.com/milvus-io/milvus/issues/29230
this pr do two things about cagra index:
a.milvus yaml config support gpu memory settings
b.add cagra-params check
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
Co-authored-by: yusheng.ma <yusheng.ma@zilliz.com>
2023-12-26 11:04:47 +08:00
ac23e67f07
enhance: add default file path for access log ( #29460 )
...
relate:https://github.com/milvus-io/milvus/issues/29459
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 10:38:45 +08:00
bd3bde82f0
fix iterator lose data for duplicted result( #29406 ) ( #29451 )
...
related: #29406
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-25 23:28:47 +08:00
13988cbde7
fix: Partition related privileges should belong to Collection ( #29428 )
...
issue: #29427
it will try to find privilege_name from the object_type, while grant a
privilege to a role
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-24 20:32:45 +08:00
dd9c61831d
enhance: Support to get the param value in the runtime ( #29297 )
...
/kind improvement
issue: #29299
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-22 18:36:44 +08:00
7a6aa8552a
fix: add back existing datanode metrics ( #29360 )
...
See also: #29204
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-22 14:20:43 +08:00
a0e1a1eb31
feat: support enable/disable mmap for index ( #29005 )
...
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 18:07:24 +08:00
48f506b077
fix: Seperate msgstream initctx with lifecycle ctx ( #29308 )
...
The close of msgstream should be controlled by Close() not ctx from
outside when init clients
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-21 11:56:45 +08:00
fcba1c0d9e
fix: Rename invalid parameterutil package path ( #29334 )
...
This PR renames the invalid parameterutil package path
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-21 11:40:42 +08:00
cb0676153f
enhance: add metrics for stopping querynode balance progress ( #29201 )
...
This PR add three metrics to track the stopping balance progress.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-21 10:14:52 +08:00
f457b9f7c9
fix: Return time tick delay error and refine quota error messages ( #29289 )
...
This pr:
1. Handles the time tick delay error when converting old error codes to
milvus errors.
2. Enhances quota error messages by eliminating "force deny" and
substituting it with "quota exceeded."
issue: https://github.com/milvus-io/milvus/issues/29288
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-20 22:52:44 +08:00
46a937ac43
fix: msgstream memory leak caused by config event don't deregister ( #29266 )
...
relate: https://github.com/milvus-io/milvus/issues/28620
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-20 20:32:44 +08:00
7d32861d53
enhance: Add channel check interval params ( #29311 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-20 15:06:42 +08:00
010d8362ad
fix: datanode L0 segment num wrong ( #29050 )
...
relate: https://github.com/milvus-io/milvus/issues/27675
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-20 11:20:44 +08:00
cbf0f9c527
enhance: change cp metric to absolute unix ts ( #29328 )
...
See also #29327
Change channel checkpoint metrics to unix seconds instead of checkpoint
timestamp lag value
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-20 11:04:45 +08:00
bcd6865b29
enhance: add 3 builtin roles ( #28961 )
...
issue: #28960 [milvus-proto
#212 ](https://github.com/milvus-io/milvus-proto/issues/212 )
add new configuration: builtinRoles
user can define roles in config file: `milvus.yaml`
there is an example:
1. db_ro, only have read privileges, include load
2. db_rw, read and write privileges, include create/drop/rename
collection
3. db_admin, not only read and write privileges, but also user
administration
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-18 14:28:41 +08:00
a602171d06
enhance: Refactor runtime and expr framework ( #28166 )
...
#28165
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-12-18 12:04:42 +08:00
820ee692fc
enhance: Add config for querycoord auto balance channel ( #29231 )
...
issue: #23726
This PR add control config to querycoord's background auto balance
channel operation
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-18 10:00:40 +08:00
88b4b8b77c
enhance: make segments SQPool & LoadPool resizable ( #29239 )
...
See also #29223
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-15 20:22:39 +08:00
4731c1b0d5
enhance: make SyncManager pool size refreshable ( #29224 )
...
See also #29223
This PR make `conc.Pool` resizable by adding `Resize` method for it.
Also make newly added datanode `MaxParallelSyncMgrTasks` config
refreshable
---------
Signed-off-by: Congqi.Xia <congqi.xia@zilliz.com>
2023-12-15 09:58:43 +08:00
25a4525297
enhance: Change sync manager parallel config item ( #29216 )
...
Since the sync manager is global in datanode now, the old
`maxParallelSyncTaskNum` does not fit into current implementation
anymore.
This PR add a new param item for sync mgr parallel control and enlarge
default value
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-14 20:46:41 +08:00
9407be6eb2
enhance: Add a config item for partition name as regexp feature and disable it by default ( #29154 )
...
See also #29177
Add a config item for partition name as regexp feature and disable it by
default
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-14 10:04:38 +08:00
b5ee563914
fix: accesslog can not print search expression ( #28899 )
...
relate: https://github.com/milvus-io/milvus/issues/28893
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-13 18:50:42 +08:00
520c07ec9d
fix: autoindex panic with flat index type ( #29071 )
...
issue: https://github.com/milvus-io/milvus/issues/29048
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-12 22:08:46 +08:00
e5f155612a
fix: txn control not applied to snapshot.Remove ( #29124 )
...
issue: #29123
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-12 14:48:38 +08:00
51f870da7e
feat: Introduce channelCheckpointUpdater to reduce goroutine use in ttNode ( #28570 )
...
/kind enhancement
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-12-12 13:48:42 +08:00
69a71e11d1
enhance: Add config for auto upgrade segment index ( #29112 )
...
This PR add config for auto upgrade segment index to index engine's
version.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-12 11:56:37 +08:00
0e65e90338
enhance: Support otlp with insecure ( #29115 )
...
issue: https://github.com/milvus-io/milvus/issues/28914
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-12-12 11:14:37 +08:00
0a87724f18
enhance: remove merger for load segments ( #29062 )
...
remove merger as now QueryNode could load segments concurrently
fix #29063
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-12 10:48:37 +08:00
2b05460ef9
enhance: Make import-related error message clearer ( #28978 )
...
issue: #28976
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-12-08 10:12:38 +08:00
43abe9ce31
enhance: Delete datanode metric after flowgraph released ( #29003 )
...
`DataNodeFlowGraphBufferDataSize` monitoring metric is leaked. This PR
resolve it.
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-07 17:00:35 +08:00
9b3e06ae86
enhance: add more metrics for level zero segments ( #29029 )
...
- Add SegmentNum metric for level zero segments
- Add level zero segments size metirc
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-07 14:34:35 +08:00
6736f65345
feat: skip some empty ttMsg in Datanode flowgraph ( #28756 )
...
/kind feature
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-12-07 01:00:37 +08:00
4a067a4c8c
enhance: Add proxy.ginLogSkipPaths ( #28945 )
...
Fix #28944
/cc @xiaofan-luan
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-12-06 10:26:35 +08:00
ad1daebc8e
fix: Change ranges of HNSW parameters according to Knowhere ( #28875 )
...
fix: https://github.com/milvus-io/milvus/issues/28860
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-12-05 16:20:42 +08:00
bf633bb5d7
enhance: refine the retry error ( #28573 )
...
return the last error but not combining all errors, to improve
readability and erorr handling
resolve : #28572
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-30 18:34:32 +08:00
8b2b0d412c
enhance: storeageType default value change to remote ( #28792 )
...
/kind enhancement
/cc @PowderLi
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-11-30 11:34:27 +08:00
f9bb8e9648
enhance: Change const magic number in querycoord to param ( #28819 )
...
See also #28817
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-30 09:06:28 +08:00
3b1b14dd78
fix: update binlog index memory uasge before loading segments ( #28528 )
...
issue: #27678
when interimIndex = true, memory predict should be update with the
memory usage of binlog index build process.
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2023-11-29 16:42:27 +08:00
5bd73d5503
enhance: add efficient distance computations in Go ( #28657 )
...
Related #28656
Add more efficient calc_distance at Go side.
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-11-28 18:20:26 +08:00
MrPresent-Han
xige-16
XuanYang-cn
Gao
wei liu
aoiasd
cqy123456
PowderLi
SimFG
yah01
yihao.dai
congqixia
zhagnlu
Jiquan Long
wayblink
Enwei Jiao
yah01
cai.zhang
shaoyue
groot