f0b3942a08
enhance: Limit import job number ( #36891 )
...
issue: https://github.com/milvus-io/milvus/issues/36890
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-23 16:01:28 +08:00
f3d9d05a28
fix: use binlog counter to trigger flush but not stats log ( #37037 )
...
issue: #36804
Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-23 15:07:29 +08:00
4746f47282
feat: management WebUI homepage ( #36822 )
...
issue: #36784
1. Implement an embedded web server for WebUI access.
2. Complete the homepage development.
Home page demo:
<img width="2177" alt="iShot_2024-10-10_17 57 34"
src="https://github.com/user-attachments/assets/38539917-ce09-4e54-a5b5-7f4f7eaac353 ">
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-10-23 11:29:28 +08:00
30121a5a0d
fix: Rectify delete buffer row count quota value ( #37060 )
...
Related to #37057
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-23 10:15:29 +08:00
90b3907a92
fix: fix missing return value in chunked column ( #37064 )
...
issue: https://github.com/milvus-io/milvus/issues/36834
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-10-22 10:29:19 -07:00
e2f2fd55a5
enhance: avoid limiting ddl operations repeatedly( #37006 ) ( #37010 )
...
related: #37006
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-10-22 20:11:27 +08:00
5dd3f44cc1
enhance: Preallocate delete data slice to avoid growslice ( #37043 )
...
Related to #36887
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-22 19:07:29 +08:00
0d8f20f7ce
fix: Pass full field list when partial load enabled ( #37053 )
...
Related to #37038
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-22 18:43:27 +08:00
5a1f752272
enhance: [bitset] multiple 'and' and 'or' in a single op ( #33345 )
...
issue #34117
* Refactoring
* Added a capability to perform multiple bitwise `and` and `or`
operations in a single op
* AVX2, AVX512, ARM NEON, ARM SVE backed bitwise `and`, `op`, `xor` and
`sub` ops
* more unit tests for bitset
* fixed a bug in `or_with_count` for certain bitset sizes
* fixed a bug for certain offset values for inplace operations that take
two bitsets
Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
2024-10-22 16:25:33 +08:00
6bedc7e8c8
fix: not set valid_data in bitmap index when mmap ( #37023 )
...
#37013
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-22 12:03:26 +08:00
50607a5ca7
enhance: set the rpc error code to avoid the invalid retry ( #37024 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-22 10:33:26 +08:00
346510ed23
enhance: Update Knowhere version ( #37000 )
...
Signed-off-by: foxspy <xian_hust@foxmail.com>
2024-10-21 11:39:26 +08:00
ac178eeea5
enhance: make delegator lock critical smaller ( #36997 )
...
issue: #36804
Signed-off-by: chyezh <chyezh@outlook.com>
2024-10-21 11:33:25 +08:00
70339820fa
fix: Check whether new collection name is alias ( #36981 )
...
Related to #36963
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-21 10:25:25 +08:00
2e7670908f
enhance: Track complex delete rates ( #36957 )
...
See also: #36953
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-18 17:47:30 +08:00
0516624ab0
enhance: Log when limit writing by deletion ( #36954 )
...
See also: #36953
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-18 17:25:25 +08:00
8669153ff2
fix: not add isClusteringKey when describe collection ( #36930 )
...
#36928
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-18 12:57:26 +08:00
2441c92ae0
fix: Query node panic during sending rpc to worker ( #36975 )
...
issue: #36976
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-10-18 12:33:25 +08:00
304098cd40
fix:Chunk Id out of range in vector BF search, after the growing index removes the vec chunks. ( #36939 )
...
issue: https://github.com/milvus-io/milvus/issues/36871
related pr: https://github.com/milvus-io/milvus/pull/36938
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-10-18 12:21:25 +08:00
fbe177d6e7
fix: avoid panic when load segment with pkoracle and idforacle already exist ( #36959 )
...
relate: https://github.com/milvus-io/milvus/issues/36949
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-18 11:57:24 +08:00
50da48a30d
enhance: adding mix compaction first prioritizer ( #36956 )
...
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-10-18 11:37:24 +08:00
903c18ba26
enhance: consider the mmap chunck cache config when resource usage estimate ( #36814 )
...
- issue: #36530
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-18 10:17:23 +08:00
3de57ec4fa
enhance: add vector index mgr to remove vector index type dependency ( #36843 )
...
issue: #34298
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-17 22:15:25 +08:00
eb3e4583ec
enhance: all op(Null) is false in expr ( #35527 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-17 21:14:30 +08:00
04c306e63f
fix: Fix clustering compaction task leak ( #36800 )
...
issue: #36686
bug reason:
- The clustering compaction tasks on the datanode were never cleaned up.
- The clustering compaction task contains a mapping from clustering key
to buffer, this caused a large memory leak.
fix:
- clean the tasks on datanode by datacoord when clustering compaction
finished.
- reset the mapping that from clustering key to buffer on datanode when
clustering finished.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-17 20:43:30 +08:00
b474374ea5
enhance: use growingMmapEnabled to control the behavior of interim index, not vectorField ( #36500 )
...
issue:https://github.com/milvus-io/milvus/issues/36392
related pr: https://github.com/milvus-io/milvus/pull/36391
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-10-17 20:25:24 +08:00
277c50fd31
fix:add log in mixCompactionTask and set fail/timeout task to clean ( #35970 )
...
issue: #35966
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-10-17 18:05:23 +08:00
eccc326e8b
enhance: report err when group_size is wrong( #36146 ) ( #36908 )
...
related: #36146
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-10-17 16:05:29 +08:00
20750c061b
fix: loss small bm25 binlogs ( #36942 )
...
Sync task don't flush small bm25 logs, cause growing segment bm25 stats
loss.
relate: https://github.com/milvus-io/milvus/issues/36805
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-17 14:49:23 +08:00
605e0890fb
enhance: enable parallel execution of L0 compactions ( #36816 )
...
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-10-17 14:47:31 +08:00
60e51f1076
fix: unicode replacement character (0xFFFD) are not supported as csv delimiter ( #36310 )
...
https://github.com/milvus-io/milvus/issues/36309
Signed-off-by: OxalisCu <2127298698@qq.com>
2024-10-17 14:45:40 +08:00
1184319644
fix: Load original key if ts is MaxTimestamp ( #36934 )
...
Related to #36933
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-17 14:11:29 +08:00
d51a808851
fix: Rootcoord stuck at graceful stop progress ( #36880 )
...
issue: #34553
when rootcoord trigger graceful stop progress, it will block until all
rpc finished. for create collection request, rootcoord need to block
until datacoord finish to watch all channels, but datacoord need to call
`rootcoord.Alloc` during watch channel, and rootcoord doesn't respond to
new request anymore. which cause create collection stucks, and graceful
stop progress stucks.
This PR remove the func call `rootcoord.Alloc` to solve the logic dead
lock during graceful stop progress.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-10-17 12:15:25 +08:00
f8ac91f1db
fix: datacoord stuck at stopping progress ( #36852 )
...
issue: #36868
if datacoord is syncing segments to datanode, and stop datacoord
happens, datacoord's stop progress will stuck until syncing segment
finished.
This PR add ctx to syncing segment, which will failed if stopping
datacoord happens.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-10-17 12:13:37 +08:00
6851738fd1
fix: fix `make generate-mockery` panic with go1.22 ( #36830 )
...
https://github.com/milvus-io/milvus/issues/36831
Fix `make generate-mockery` panic.
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-10-17 12:11:31 +08:00
463c47ced1
enhance: support default value in import ( #36700 )
...
https://github.com/milvus-io/milvus/issues/31728
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-17 12:05:24 +08:00
1375d690dd
enhance: adjust some err msg ( #36840 )
...
related with #36495
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-17 12:01:25 +08:00
b172ea1093
fix: Remove enableLevelZeroSegment config ( #36535 )
...
See also: #36504
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-17 11:59:24 +08:00
06b5e186a7
fix: return error if searching against BM25 output field with incorrect metric type ( #36910 )
...
issue: https://github.com/milvus-io/milvus/issues/36835
currently searching BM25 output field using IP will end up in an error
in segcore which is hard to understand. now returning error in query
node delegator and provide more useful error message
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-16 19:45:23 +08:00
903450f5c6
enhance: add ts support for iterator( #22718 ) ( #36572 )
...
related: #22718
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-10-16 18:51:23 +08:00
447ff342fb
fix: Direct forward delta exclude l0 segments ( #36899 )
...
Related to #36887
Forward delete to L0 segment will return error and mark l0 segment
offline causing delegator unserviceable
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-16 14:05:23 +08:00
caeab0cc1f
enhance: Fill start pos & level for growing segment ( #36888 )
...
Start position & level info is missing for growing segment loaded in
watch dml channel operation.
Level is important for metrics and start position is crucial for growing
exclude logic.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-16 14:03:31 +08:00
44d80c1355
fix: not return err if consistencyLevel is not set to a valid value ( #36714 )
...
https://github.com/milvus-io/milvus/issues/36444
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-16 13:23:22 +08:00
72dc07ba48
fix: bm25 search failed when nq > 1 and remove idf oracle when no bm25 field exist. ( #36886 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-16 12:51:23 +08:00
b2037c95a8
fix: use chunk_row_nums to iterate ( #36882 )
...
Fix segmentation fault error and remove useless codes.
https://github.com/milvus-io/milvus/issues/36834
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-10-16 11:15:25 +08:00
9997c5de34
fix: remove excessive logging ( #36859 )
...
issue: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-16 10:47:22 +08:00
f3b6792a25
enhance: Enhance segment log ( #36848 )
...
/kind improvement
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-15 20:43:30 +08:00
ba25320aea
fix: Unify loaded partition check to delegator ( #36879 )
...
Related to #36370
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-15 19:15:23 +08:00
1bd3228635
enhance: Ignore db not found error in quota center ( #36821 )
...
In quota center, ignore the "DB not found error" to prevent it from
affecting the rate limiting of other databases.
/kind improvement
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-15 15:55:22 +08:00
aa904be6ec
enhance: support sparse vector mmap in growing segment type ( #36566 )
...
issue: https://github.com/milvus-io/milvus/issues/32984
related pr: https://github.com/milvus-io/milvus/pull/36565
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-10-15 10:59:23 +08:00
yihao.dai
Zhen Ye
jaime
congqixia
Bingyi Sun
Chun Han
Alexander Guzhva
smellthemoon
SimFG
foxspy
XuanYang-cn
wei liu
cqy123456
aoiasd
Ted Xu
foxspy
cai.zhang
wayblink
OxalisCu
Buqian Zheng