milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
21,286 Commits
133 Branches
115 Tags
559 MiB
Commit Graph

21286 Commits (7c5a8012cf8e04b426ceb21aa797e944873a28cb)

Author SHA1 Message Date
congqixia 97b7ebbc08
fix: Move init kafka pool into once (#37786) 
Introduced by #37744

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-18 21:24:48 +08:00
wei liu 351463b67e
fix: L0 segment has been loaded to worker during channel balance (#37748) 
issue: #37703

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-11-18 20:58:30 +08:00
MimoN 5065d778d6
fix: [skip e2e]update webui base api url (#37779) 
issue: #36621

- fix tools links in webui

Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
 2024-11-18 20:40:31 +08:00
sthuang 2d72ad33f2
enhance: RBAC built in privilege groups (#37720) 
issue: #37031

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
 2024-11-18 20:38:39 +08:00
aoiasd 16e206167c
enhance: analyzer length filter max should be close interval instead open interval (#37770) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-11-18 19:30:31 +08:00
MimoN 82d2ac6be0
fix: update webui base api url (#37775) 
issue: #36621

- fixed the base api url in webui

Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
 2024-11-18 19:10:33 +08:00
wei liu 261212ee4a
fix: Get shard client failed by client is closed (#37729) 
issue: #37718
This PR refine the shard client ref counter, dec ref counter won't
release client anymore, and only permit shard client manager to remove
client.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-11-18 18:22:30 +08:00
wei liu 226fe900e7
enhance: Use batch to speed up list collections from meta kv (#37742) 
issue: #36228

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-11-18 16:58:30 +08:00
yihao.dai 0fc0d1a888
fix: Limit the concurrency of channel tasks (#37740) 
Limit the maximum concurrency of channel tasks for each DataNode to
prevent excessive subscriptions from causing DataNode OOM.

issue: https://github.com/milvus-io/milvus/issues/37665

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-11-18 16:26:30 +08:00
aoiasd e9391acf80
fix: bm25 brute force search need index params k1 and b (#37721) 
relate: https://github.com/milvus-io/milvus/issues/35853

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-11-18 15:44:31 +08:00
jaime 12ed40e125
fix: some missing metrics for WebUI restful API (#37747) 
issue: #36621

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-11-18 14:00:32 +08:00
Chun Han f2a2fd6808
enhance: shallow copy search request for large nq cases(#37732) (#37749) 
related: #37732

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
 2024-11-18 13:06:32 +08:00
jaime 257ecab84b
enhance: remove collection queryable check from health check (#37712) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-11-18 10:50:38 +08:00
congqixia b0bd290a6e
enhance: Use internal json(sonic) to replace std json lib (#37708) 
Related to #35020

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-18 10:46:31 +08:00
congqixia 3743fc2b00
enhance: Use pool to limit kafka cgo thread number (#37744) 
Related to #37665

Thread number went rocket high when there is lots of kafka consumers on
datanode. Since the internal implementation is CGO, using which directly
will make cgo thread leaked.

This PR add a worker pool for kafka API utilzing CGO calls to limit
thread number.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-18 10:42:31 +08:00
Zhen Ye 3f1614e9d9
enhance: add trace_id into segcore logs (#37656) 
issue: #37655

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-11-18 10:20:30 +08:00
Buqian Zheng 00edec2ebd
fix: incorrect dim set when creating BM25 doc sparse array (#37717) 
issue: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-11-16 16:36:30 +08:00
zhuwenxing 3cdb485022
test: update timeout of load in ci cases (#37596) 
action for
https://github.com/milvus-io/milvus/issues/37166#issuecomment-2469502955

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-11-16 16:00:31 +08:00
congqixia 4820dd76e5
fix: Close leaked health check kafka producer (#37728) 
Related to #36822

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-16 11:20:32 +08:00
zhuwenxing 3f7352f3cf
test: remove xfail of fts test cases after fix (#37724) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2024-11-16 11:14:30 +08:00
jaime ead1e7f68c
enhance: The Restful API supports a single page web application. (#37719) 
issue: #36621

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-11-16 11:04:31 +08:00
yihao.dai 3df2c92bb0
enhance: Remove segment-level tag from monitoring metrics (#37696) 
When there are a large number of segments, the metrics consume a lot of
memory. This PR Remove segment-level tag from monitoring metrics.

issue: https://github.com/milvus-io/milvus/issues/37636

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-11-16 10:58:31 +08:00
MimoN f7c7ac51fc
fix: update webui and remove the unnecessary code (#37726) 
issue: #36621
* update webui top nav
* remove the unnecessary code in .gitignore file

Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
 2024-11-16 10:56:33 +08:00
aoiasd 3b5a0df159
enhance: Optimize chinese analyzer and support CnAlphaNumFilter (#37727) 
relate: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-11-16 10:28:30 +08:00
foxspy 0ba868ae64
enhance: update knowhere version (#37730) 
release note draft :
https://github.com/zilliztech/knowhere/releases/tag/v2.5.0

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
 2024-11-16 10:08:30 +08:00
sre-ci-robot 589b5b9786 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-11-15 12:00:46 +00:00
smellthemoon 7999367c0c
fix: use not retried err when get wrong parameter (#37707) 
#37508

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2024-11-15 19:14:30 +08:00
zhagnlu e4b6773d0a
fix: fix create text index dir conflict bug (#37693) 
#37623

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2024-11-15 18:26:30 +08:00
MimoN 0f0162f70a
feat: refactor web ui (#37698) 
issue: #36621

* The front-end code of the management interface has been refactored
using React.
* The interaction and display of the front-end interface have been
optimized.

Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
Co-authored-by: zilliz <zilliz@zillizdeMacBook-Pro.local>
 2024-11-15 17:20:30 +08:00
yihao.dai 18a64782dc
enhance: Remove unnecessary clone in SetState (#37697) 
issue: https://github.com/milvus-io/milvus/issues/37637

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2024-11-15 15:06:30 +08:00
Zhen Ye 81fa7dd52c
fix: add ddl and dcl concurrency to avoid competition (#37672) 
issue: #37166

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-11-15 15:04:31 +08:00
Bingyi Sun 65d3c6622a
enhance: Optimize GetChunkIDByOffset and add ut (#37704) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-11-15 14:16:31 +08:00
XuanYang-cn 5a23c80f20
fix: Change memoryCheck write lock to read lock (#37525) 
See also: milvus-io#37493

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-11-15 10:44:31 +08:00
Bingyi Sun d1596297d9
fix: Fix query failure with inverted index (#37686) 
https://github.com/milvus-io/milvus/issues/37649

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-11-15 10:28:31 +08:00
congqixia 5afcee6bfa
fix: Store default value if `ErrKeyNotFound` is returned (#37691) 
Related to #37690

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-15 10:24:32 +08:00
congqixia 66bf254437
enhance: Enable `RemoteLoad` l0 forward policy by default (#37678) 
Related to #35303

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-11-15 10:22:30 +08:00
wei liu a1b6be1253
fix: Delegator stuck at unserviceable status (#37694) 
issue: #37679

pr #36549 introduce the logic error which update current target when
only parts of channel is ready.

This PR fix the logic error and let dist handler keep pull distribution
on querynode until all delegator becomes serviceable.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-11-15 10:20:31 +08:00
cai.zhang b9357e4716
fix: Modify the batchsize of writer to timely flushing binlogs (#37692) 
issue: #37579 

If the schema includes large varchar fields, a few thousand rows can
reach hundreds of MB in size. Therefore, if the batch size of the
segment writer is large, it will produce relatively large `binlogs`,
which can cause datanode to run out of memory (OOM) during compaction.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2024-11-15 10:14:31 +08:00
foxspy 5ae347aba0
enhance: update knowhere version (#37688) 
issue: #37665 #37631 #37620 #37587 #36906 
knowhere has add default nlist value, so some invalid param test ut with
no nlist param will be valid.

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
 2024-11-15 10:10:31 +08:00
jaime 1d06d4324b
fix: Int64 overflow in JSON encoding (#37657) 
issue: ##36621

- For simple types in a struct, add "string" to the JSON tag for
automatic string conversion during JSON encoding.
- For complex types in a struct, replace "int64" with "string."

Signed-off-by: jaime <yun.zhang@zilliz.com>
 2024-11-14 22:52:30 +08:00
wei liu c5485bb1b1
fix: Make GetShardLeaders skip retry on not loaded collection (#37684) 
issue: #37532

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-11-14 21:14:40 +08:00
Bingyi Sun 1b4f7e3ac1
enhance: Add more expr ut for chunked segment (#37600) 
related pr: #37570

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2024-11-14 18:40:32 +08:00
nico 405c12f7d1
test: update test cases (#37660) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2024-11-14 17:46:39 +08:00
aoiasd 993051bb49
fix: brute force bm25 search lack avgdl param (#37650) 
relate: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-11-14 14:58:31 +08:00
XuanYang-cn 31a8d08bdd
fix: Correct varchar primarykey size calculation (#37617) 
See also: #37582

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-11-14 14:16:38 +08:00
Zhen Ye cd181e4c6d
fix: make NewDataSyncService idempotent of dispatcher (#37576) 
issue: #37547

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-11-14 11:00:31 +08:00
Buqian Zheng 0565300b7f
fix: Sparse to use CC index as growing/temp index (#37591) 
relate: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 2024-11-14 10:54:31 +08:00
aoiasd 1c5b5e1e3d
feat: Add chinese and english analyzer with refactor jieba tokenizer (#37494) 
relate: https://github.com/milvus-io/milvus/issues/35853

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2024-11-14 10:34:31 +08:00
wei liu 1304b40552
fix: Balance channel may stuck at increasing replica number case (#37641) 
issue: #37640
fix the pr #36549
cause balance channel will wait until new delegator becomes serviceable,
but new delegator need to sync target version then becomes serviceable,
and sync target version need to be wait all replica load done. so if
increasing replica number and balance channel happens at same time,
logic dead lock occurs.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-11-14 10:08:31 +08:00
sre-ci-robot caf207f5a0 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2024-11-13 12:00:38 +00:00