304098cd40
fix:Chunk Id out of range in vector BF search, after the growing index removes the vec chunks. ( #36939 )
...
issue: https://github.com/milvus-io/milvus/issues/36871
related pr: https://github.com/milvus-io/milvus/pull/36938
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-10-18 12:21:25 +08:00
fbe177d6e7
fix: avoid panic when load segment with pkoracle and idforacle already exist ( #36959 )
...
relate: https://github.com/milvus-io/milvus/issues/36949
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-18 11:57:24 +08:00
50da48a30d
enhance: adding mix compaction first prioritizer ( #36956 )
...
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-10-18 11:37:24 +08:00
b7ffa8383c
enhance: [GoSDK] write back auto id value to row based input ( #36964 )
...
Related to #33460
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-18 11:19:24 +08:00
903c18ba26
enhance: consider the mmap chunck cache config when resource usage estimate ( #36814 )
...
- issue: #36530
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-18 10:17:23 +08:00
3de57ec4fa
enhance: add vector index mgr to remove vector index type dependency ( #36843 )
...
issue: #34298
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-17 22:15:25 +08:00
eb3e4583ec
enhance: all op(Null) is false in expr ( #35527 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-17 21:14:30 +08:00
04c306e63f
fix: Fix clustering compaction task leak ( #36800 )
...
issue: #36686
bug reason:
- The clustering compaction tasks on the datanode were never cleaned up.
- The clustering compaction task contains a mapping from clustering key
to buffer, this caused a large memory leak.
fix:
- clean the tasks on datanode by datacoord when clustering compaction
finished.
- reset the mapping that from clustering key to buffer on datanode when
clustering finished.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-10-17 20:43:30 +08:00
b474374ea5
enhance: use growingMmapEnabled to control the behavior of interim index, not vectorField ( #36500 )
...
issue:https://github.com/milvus-io/milvus/issues/36392
related pr: https://github.com/milvus-io/milvus/pull/36391
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-10-17 20:25:24 +08:00
97ff012c67
test: check query response data ( #36952 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-10-17 19:13:25 +08:00
277c50fd31
fix:add log in mixCompactionTask and set fail/timeout task to clean ( #35970 )
...
issue: #35966
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-10-17 18:05:23 +08:00
eccc326e8b
enhance: report err when group_size is wrong( #36146 ) ( #36908 )
...
related: #36146
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-10-17 16:05:29 +08:00
20750c061b
fix: loss small bm25 binlogs ( #36942 )
...
Sync task don't flush small bm25 logs, cause growing segment bm25 stats
loss.
relate: https://github.com/milvus-io/milvus/issues/36805
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-17 14:49:23 +08:00
605e0890fb
enhance: enable parallel execution of L0 compactions ( #36816 )
...
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-10-17 14:47:31 +08:00
60e51f1076
fix: unicode replacement character (0xFFFD) are not supported as csv delimiter ( #36310 )
...
https://github.com/milvus-io/milvus/issues/36309
Signed-off-by: OxalisCu <2127298698@qq.com>
2024-10-17 14:45:40 +08:00
eb62433973
enhance: [GoSDK] Set default max send recv size ( #36945 )
...
Related to milvus-io/milvus-sdk-go#831
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-17 14:33:23 +08:00
aee61973fb
[automated] Bump milvus version to v2.4.13-hotfix ( #36947 )
...
Bump milvus version to v2.4.13-hotfix
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-17 14:17:28 +08:00
1184319644
fix: Load original key if ts is MaxTimestamp ( #36934 )
...
Related to #36933
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-17 14:11:29 +08:00
811f6f2402
enhance: mmap for faiss hnsw ( #36943 )
...
Unblock the mmap check for Faiss-based HNSW
Signed-off-by: Li Liu <li.liu@zilliz.com>
2024-10-17 13:05:23 +08:00
821750d902
enhance: force to stop buffer message when receiving the drop collection message ( #36916 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-17 12:53:29 +08:00
d51a808851
fix: Rootcoord stuck at graceful stop progress ( #36880 )
...
issue: #34553
when rootcoord trigger graceful stop progress, it will block until all
rpc finished. for create collection request, rootcoord need to block
until datacoord finish to watch all channels, but datacoord need to call
`rootcoord.Alloc` during watch channel, and rootcoord doesn't respond to
new request anymore. which cause create collection stucks, and graceful
stop progress stucks.
This PR remove the func call `rootcoord.Alloc` to solve the logic dead
lock during graceful stop progress.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-10-17 12:15:25 +08:00
f8ac91f1db
fix: datacoord stuck at stopping progress ( #36852 )
...
issue: #36868
if datacoord is syncing segments to datanode, and stop datacoord
happens, datacoord's stop progress will stuck until syncing segment
finished.
This PR add ctx to syncing segment, which will failed if stopping
datacoord happens.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-10-17 12:13:37 +08:00
6851738fd1
fix: fix `make generate-mockery` panic with go1.22 ( #36830 )
...
https://github.com/milvus-io/milvus/issues/36831
Fix `make generate-mockery` panic.
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-10-17 12:11:31 +08:00
463c47ced1
enhance: support default value in import ( #36700 )
...
https://github.com/milvus-io/milvus/issues/31728
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-17 12:05:24 +08:00
1375d690dd
enhance: adjust some err msg ( #36840 )
...
related with #36495
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-17 12:01:25 +08:00
b172ea1093
fix: Remove enableLevelZeroSegment config ( #36535 )
...
See also: #36504
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-10-17 11:59:24 +08:00
4d08eec1af
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-10-16 12:01:07 +00:00
06b5e186a7
fix: return error if searching against BM25 output field with incorrect metric type ( #36910 )
...
issue: https://github.com/milvus-io/milvus/issues/36835
currently searching BM25 output field using IP will end up in an error
in segcore which is hard to understand. now returning error in query
node delegator and provide more useful error message
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-16 19:45:23 +08:00
51f13ba7cf
fix: add checks for bm25 k1 and b in sparse index checker ( #36907 )
...
issue: https://github.com/milvus-io/milvus/issues/36883 ,
https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-16 19:43:24 +08:00
e5948bd039
enhance: [GoSDK] Use variadic params for options ( #36912 )
...
Use variadic parameter function for options make client options easier
to use.
Related to #31293
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-16 19:33:23 +08:00
903450f5c6
enhance: add ts support for iterator( #22718 ) ( #36572 )
...
related: #22718
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-10-16 18:51:23 +08:00
c271c21f17
enhance: improve scripts standalone_embed,add upgrade ( #36924 )
...
[[skip ci] improve scripts standalone_embed,add upgrade](enhance:
improvement scripts standalone_embed,add upgrade #36921 )
Signed-off-by: zilliz <zilliz@zillizdeMacBook-Pro.local>
Co-authored-by: zilliz <zilliz@zillizdeMacBook-Pro.local>
2024-10-16 17:51:23 +08:00
447ff342fb
fix: Direct forward delta exclude l0 segments ( #36899 )
...
Related to #36887
Forward delete to L0 segment will return error and mark l0 segment
offline causing delegator unserviceable
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-16 14:05:23 +08:00
caeab0cc1f
enhance: Fill start pos & level for growing segment ( #36888 )
...
Start position & level info is missing for growing segment loaded in
watch dml channel operation.
Level is important for metrics and start position is crucial for growing
exclude logic.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-16 14:03:31 +08:00
44d80c1355
fix: not return err if consistencyLevel is not set to a valid value ( #36714 )
...
https://github.com/milvus-io/milvus/issues/36444
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-16 13:23:22 +08:00
72dc07ba48
fix: bm25 search failed when nq > 1 and remove idf oracle when no bm25 field exist. ( #36886 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-10-16 12:51:23 +08:00
b2037c95a8
fix: use chunk_row_nums to iterate ( #36882 )
...
Fix segmentation fault error and remove useless codes.
https://github.com/milvus-io/milvus/issues/36834
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-10-16 11:15:25 +08:00
9997c5de34
fix: remove excessive logging ( #36859 )
...
issue: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-10-16 10:47:22 +08:00
f3b6792a25
enhance: Enhance segment log ( #36848 )
...
/kind improvement
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-15 20:43:30 +08:00
ba25320aea
fix: Unify loaded partition check to delegator ( #36879 )
...
Related to #36370
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-15 19:15:23 +08:00
2291d0c9f5
test: update test cases ( #36878 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-10-15 19:13:24 +08:00
e43af16e93
enhance: [ci] separate ccache by branch for e2e pipeline ( #36870 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-10-15 17:21:22 +08:00
1bd3228635
enhance: Ignore db not found error in quota center ( #36821 )
...
In quota center, ignore the "DB not found error" to prevent it from
affecting the rate limiting of other databases.
/kind improvement
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-10-15 15:55:22 +08:00
b0d5866507
enhance: add checkers for Faiss-based HNSW ( #36857 )
...
Knowhere introduced FAISS-based HNSQ Flat/SQ/PQ/PRQ. This PR adds param
check in Milvus side.
This PR actually do nothing but let the check pass, since:
1. All indexes param check will be moved to Knowhere recently. @foxspy
2. The index name will be changed from `FAISS_HNSW_xx` to `HNSW_xx`
after QA test. @alexanderguzhva
Signed-off-by: Li Liu <li.liu@zilliz.com>
2024-10-15 12:29:22 +08:00
aa904be6ec
enhance: support sparse vector mmap in growing segment type ( #36566 )
...
issue: https://github.com/milvus-io/milvus/issues/32984
related pr: https://github.com/milvus-io/milvus/pull/36565
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-10-15 10:59:23 +08:00
bb3ef5349f
enhance: update the expr version to support automatic conversion of variable types ( #36832 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-10-15 10:53:22 +08:00
65608ca8d9
enhance: able to upload pytest logs when test is done ( #36842 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-10-15 10:39:22 +08:00
744a36c287
enhance: [GoSDK] support unmarshal result set into orm receiver ( #36789 )
...
Related to milvus-io/milvus-sdk-go#800
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-10-15 10:37:23 +08:00
0ec92513a5
test: Update Diskann supporting group by ( #36853 )
...
related issue: #36813
milvus diskann does not support group_search before, it supports now.
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-15 10:35:23 +08:00
c9752bd2e6
enhance: refactor createCollection in RESTful API ( #36790 )
...
1. support isClusteringKey in restful api;
2. throw err if passed invalid 'enableDynamicField' params
3. parameters in indexparams are not processed properly, related with
#36365
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-10-15 10:29:22 +08:00
cqy123456
aoiasd
Ted Xu
congqixia
SimFG
foxspy
smellthemoon
cai.zhang
wt
wayblink
Chun Han
OxalisCu
sre-ci-robot
liliu-z
wei liu
Bingyi Sun
XuanYang-cn
sre-ci-robot
Buqian Zheng
Linkwei
yihao.dai
nico
yellow-shine
yanliang567