milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
20,727 Commits
127 Branches
113 Tags
553 MiB
Commit Graph

20727 Commits (b9c037f5583a487d7da3937de6b3d01419a191a3)

Author SHA1 Message Date
Ted Xu b9c037f558
feat: adding cache to expression parse (#36185) 
See #36122

This PR improves the proxy node performance by adding cache to
expression parse.

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-09-13 10:03:09 +08:00
sre-ci-robot 1833f2d516 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-09-13 00:01:28 +00:00
cai.zhang 8395c8a8db
enhance: Update stats task to optional (#35947) 
issue: #33744

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-09-12 20:37:08 +08:00
XuanYang-cn e8840a1b41
enhance: Add metrics for Delete entries num of L0seg (#36175) 
- Add metrics *DataCoordL0DeleteEntriesNum*
- Remove metrics *DataCoordRateStoredL0Segment*

See also: #36147

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-09-12 18:07:08 +08:00
binbin b5ff348906
test: Add part of test cases for default and null support (#36186) 
issue : #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
 2024-09-12 17:29:13 +08:00
zhagnlu 5e5e87cc2f
enhance: rename some params and reduce default bitmapCardinalityLimit… (#36138) 
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-09-12 12:09:08 +08:00
congqixia 11dbe1e755
enhance: Add L0 forward policy to support remote load (#36189) 
Related to #35303

This PR add a param item to support change l0 forward behavior from bf
filtering and forward to remote load.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-12 12:01:08 +08:00
aoiasd da227ff9a1
feat: Support create collection with functions (#35973) 
relate: https://github.com/milvus-io/milvus/issues/35853
Support create collection with functions. Prepare for support bm25
function.

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-09-12 10:43:06 +08:00
congqixia 08e681174a
fix: [GoSDK] Sync empty result set handling (#36177) 
Related to #33952

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-12 10:27:08 +08:00
congqixia 700a443e85
enhance: Check ANN field loaded in proxy (#36172) 
Related to #35415

Currently ANN field is loaded in Parital load is not performed in proxy,
this cause error message not clear and no error returned when collection
is empty

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-11 17:43:06 +08:00
Ted Xu e7ea1d7a04
enhance: improve log encoding performance on proxy nodes (#36123) 
See #36122

This PR is designed to enhance log performance through two improvements:

1. Optimize JSON encoding by switching JSON serializer to
`json-iterator`.
2. Adding support of lazy initialization `WithLazy`.

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
 2024-09-11 14:51:07 +08:00
smellthemoon 3f75bf1f20
fix: clustering compact not support null (#36152) 
#36055

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-09-11 14:49:06 +08:00
XuanYang-cn f31264c02c
fix: Change l0SegmentsRowCount limits to a reasonable value (#36014) 
See also: #36028

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-09-11 11:55:08 +08:00
binbin d9c8d1ea90
test: Add part of test cases for default and null support (#36130) 
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
 2024-09-11 10:59:07 +08:00
wt dbe03a6151
test: add check after altering cardinality limit (#36158) 
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2024-09-11 10:49:06 +08:00
nico 42eef4900d
test: update test cases and sdk version (#36148) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-09-11 10:45:07 +08:00
sre-ci-robot 229f969560 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-09-10 12:00:39 +00:00
yellow-shine 2694551757
enhance: fix a security vulnerability (#36154) 
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-09-10 19:09:27 +08:00
yellow-shine 1d22de140a
enhance: [pipeline] list all status before uninstall helm release (#36040) 
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-09-10 19:07:08 +08:00
CharlesFeng 29ddb10e8c
enhance: timer instead of ticker (#36133) 
https://github.com/milvus-io/milvus/issues/36132

Signed-off-by: fengjun2016 <jornfeng@gmail.com>
 2024-09-10 15:23:06 +08:00
yellow-shine 9d10f378e1
enhance: [skip e2e] [mergify]replace branch by reference (#36139) 
https://github.com/milvus-io/milvus/issues/36140

Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-09-10 15:21:07 +08:00
yellow-shine 9aa01c69f8
enhance: add a env (#36142) 
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-09-10 15:19:13 +08:00
Jiquan Long 89bf226f0b
feat: support keyword text match (#35923) 
fix: #35922

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2024-09-10 15:11:08 +08:00
cai.zhang b0939fd20e
enhance: Reset task check interval configuration (#36121) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-09-10 14:35:08 +08:00
Chun Han 9d0aa5c202
fix: empty result when having only one subReq(#36098) (#36128) 
related: #36098

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-10 14:25:07 +08:00
Bingyi Sun 53a8a24554
fix: fix empty indices of sparse float (#35403) 
https://github.com/milvus-io/milvus/issues/35401

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-09-10 14:23:07 +08:00
CharlesFeng 4850641943
fix: BinlogDeserializeReader leak (#36087) 
https://github.com/milvus-io/milvus/issues/36086

Signed-off-by: fengjun2016 <jornfeng@gmail.com>
 2024-09-10 12:43:07 +08:00
zhuwenxing f4d0c589fa
test: set bulk insert test fail fast (#36039) 
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-09-10 11:43:06 +08:00
yellow-shine c9f7512e6d
enhance: [skip e2e] fix mergify (#36135) 
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-09-10 10:59:06 +08:00
wt 53a87825f3
test: add more bitmap test cases (#36131) 
1. verified issues #36054 and #35971 
2. add mix scenes test cases for BITMAP index

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2024-09-10 10:55:07 +08:00
wei liu 5aedc169cd
fix: Revert "enhance: avoid the coexistence of the old coordinator and the new node/proxy (#35720)" (#36113) 
issue: #36100
This reverts commit 73be0ba941.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-09-10 10:23:07 +08:00
congqixia 851f3b9883
fix: Make legacy non-lexicographic branch break swtich (#36125) 
Related to #35941
Previous PR: #36034

This patch makes the switch branching logic correct and make the unit
test work for cases which does not select the whole dataset.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-10 10:15:07 +08:00
congqixia c0c12c6c5b
fix: Use SliceSetEqual to compare load field list (#36051) 
Related to #36037

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-10 10:13:14 +08:00
wt c916407f37
test: add query expr test cases (#36073) 
1. query with expr under different scalar index types
2. test framework supports preparing one piece of data and multiple
parameter queries

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2024-09-09 19:55:06 +08:00
OxalisCu 3a381bc247
enhance: Bulkinsert supports null in csv formats (#35912) 
see details in this issue
https://github.com/milvus-io/milvus/issues/35911

---------

Signed-off-by: OxalisCu <2127298698@qq.com>
 2024-09-09 19:17:07 +08:00
congqixia aaa8487590
enhance: Remove duplicated log in proxy service (#36110) 
Remove duplicate identical log in proxy/service.go

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-09 17:19:05 +08:00
congqixia 51870aef08
enhance: Use stats Handler to record request/response size metrics (#36107) 
Related to #36102

This PR use newly added `grpcSizeStatsHandler` to reduce calling
`proto.Size` since the request & response size info is recorded by grpc
framework.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-09 17:13:05 +08:00
yellow-shine 73a34a5a71
enhance: [skip e2e] add a rule to label ci-pass when go-sdk check status (#36105) 
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
 2024-09-09 16:17:06 +08:00
congqixia 3123093dd7
enhance: Use `MARISA_LABEL_ORDER` when building trie index (#36034) 
Related to #35941
Previous PR: #35943

This PR make `Trie` index using `MARISA_LABEL_ORDER`, which make
predictive search iterating in lexicographic order.

When trie index is build in label order, lexicographc could be utilized
accelerating `Range` operations.

However according to the official document, using `MARISA_LABEL_ORDER`
will make "exact match lookup, common prefix search, and predictive
search" slower.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-09-09 14:29:05 +08:00
XuanYang-cn 2687747278
fix: Set an empty segment if compaction deleted all inserts (#36044) 
See also: #36038

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-09-09 14:23:05 +08:00
SimFG 99817953eb
fix: delay to start the metric server port (#36080) 
- issue: #36083
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2024-09-09 14:01:05 +08:00
CharlesFeng 8787e65b1f
fix: lifeTime not released in time (#36093) 
https://github.com/milvus-io/milvus/issues/36092

Signed-off-by: fengjun2016 <jornfeng@gmail.com>
 2024-09-09 11:25:05 +08:00
wei liu 30a99b66c1
fix: Fix logic dead lock when delegator has high memory usage (#36065) 
issue: #36064
when delegator has high memory usage, load l0 segment will failed. and
balance segment task will blocked by load segment task, then delegator
cann't free memory by moving out some segment, causes a logic dead lock.

this PR remove the limit for balance, we permit segment and balance
execute in parallel. which won't cause side effect due to:
1. one segment can only has one task in qc's scheduler, and load/release
task will replace balance task if necessary
2. balance speed has been limited, and it won't block load segment task.

3. if collection has load task and balance task at same time, load task
will be scheduled first due to high proirity.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-09-09 10:21:06 +08:00
zhagnlu 208c8a2328
fix:support config index offsetcache and fix create same index again (#35985) 
#35971

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-09-08 18:23:05 +08:00
jaime 91d23ecbe1
fix: memory leak in proxy meta cache (#36075) 
issue: #36074

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-09-08 17:33:05 +08:00
yihao.dai 6b4ae0c65e
enhance: Log warn on delayed compaction tasks (#36049) 
/kind enhancement

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-09-08 17:23:05 +08:00
Chun Han e480b103bd
feat: supporing hybrid search group_by (#35982) 
related: #35096

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-09-08 17:09:04 +08:00
CharlesFeng 62f4a6a112
fix: binlog reader not released in time (#36078) 
https://github.com/milvus-io/milvus/issues/36077

Signed-off-by: fengjun2016 <jornfeng@gmail.com>
 2024-09-07 08:15:06 +08:00
sre-ci-robot 9a8ed9af5f Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-09-07 00:01:31 +00:00
Xiaofan f4263fee55
Update README.md 2024-09-06 16:01:31 -07:00