3743fc2b00
enhance: Use pool to limit kafka cgo thread number ( #37744 )
...
Related to #37665
Thread number went rocket high when there is lots of kafka consumers on
datanode. Since the internal implementation is CGO, using which directly
will make cgo thread leaked.
This PR add a worker pool for kafka API utilzing CGO calls to limit
thread number.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-18 10:42:31 +08:00
3f1614e9d9
enhance: add trace_id into segcore logs ( #37656 )
...
issue: #37655
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-18 10:20:30 +08:00
00edec2ebd
fix: incorrect dim set when creating BM25 doc sparse array ( #37717 )
...
issue: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-11-16 16:36:30 +08:00
3cdb485022
test: update timeout of load in ci cases ( #37596 )
...
action for
https://github.com/milvus-io/milvus/issues/37166#issuecomment-2469502955
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-16 16:00:31 +08:00
4820dd76e5
fix: Close leaked health check kafka producer ( #37728 )
...
Related to #36822
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-16 11:20:32 +08:00
3f7352f3cf
test: remove xfail of fts test cases after fix ( #37724 )
...
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-11-16 11:14:30 +08:00
ead1e7f68c
enhance: The Restful API supports a single page web application. ( #37719 )
...
issue: #36621
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-16 11:04:31 +08:00
3df2c92bb0
enhance: Remove segment-level tag from monitoring metrics ( #37696 )
...
When there are a large number of segments, the metrics consume a lot of
memory. This PR Remove segment-level tag from monitoring metrics.
issue: https://github.com/milvus-io/milvus/issues/37636
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-16 10:58:31 +08:00
f7c7ac51fc
fix: update webui and remove the unnecessary code ( #37726 )
...
issue: #36621
* update webui top nav
* remove the unnecessary code in .gitignore file
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-11-16 10:56:33 +08:00
3b5a0df159
enhance: Optimize chinese analyzer and support CnAlphaNumFilter ( #37727 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-16 10:28:30 +08:00
0ba868ae64
enhance: update knowhere version ( #37730 )
...
release note draft :
https://github.com/zilliztech/knowhere/releases/tag/v2.5.0
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-16 10:08:30 +08:00
589b5b9786
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-11-15 12:00:46 +00:00
7999367c0c
fix: use not retried err when get wrong parameter ( #37707 )
...
#37508
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-15 19:14:30 +08:00
e4b6773d0a
fix: fix create text index dir conflict bug ( #37693 )
...
#37623
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-11-15 18:26:30 +08:00
0f0162f70a
feat: refactor web ui ( #37698 )
...
issue: #36621
* The front-end code of the management interface has been refactored
using React.
* The interaction and display of the front-end interface have been
optimized.
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
Co-authored-by: zilliz <zilliz@zillizdeMacBook-Pro.local>
2024-11-15 17:20:30 +08:00
18a64782dc
enhance: Remove unnecessary clone in SetState ( #37697 )
...
issue: https://github.com/milvus-io/milvus/issues/37637
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-11-15 15:06:30 +08:00
81fa7dd52c
fix: add ddl and dcl concurrency to avoid competition ( #37672 )
...
issue: #37166
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-15 15:04:31 +08:00
65d3c6622a
enhance: Optimize GetChunkIDByOffset and add ut ( #37704 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-15 14:16:31 +08:00
5a23c80f20
fix: Change memoryCheck write lock to read lock ( #37525 )
...
See also: milvus-io#37493
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-15 10:44:31 +08:00
d1596297d9
fix: Fix query failure with inverted index ( #37686 )
...
https://github.com/milvus-io/milvus/issues/37649
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-15 10:28:31 +08:00
5afcee6bfa
fix: Store default value if `ErrKeyNotFound` is returned ( #37691 )
...
Related to #37690
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-15 10:24:32 +08:00
66bf254437
enhance: Enable `RemoteLoad` l0 forward policy by default ( #37678 )
...
Related to #35303
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-15 10:22:30 +08:00
a1b6be1253
fix: Delegator stuck at unserviceable status ( #37694 )
...
issue: #37679
pr #36549 introduce the logic error which update current target when
only parts of channel is ready.
This PR fix the logic error and let dist handler keep pull distribution
on querynode until all delegator becomes serviceable.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-15 10:20:31 +08:00
b9357e4716
fix: Modify the batchsize of writer to timely flushing binlogs ( #37692 )
...
issue: #37579
If the schema includes large varchar fields, a few thousand rows can
reach hundreds of MB in size. Therefore, if the batch size of the
segment writer is large, it will produce relatively large `binlogs`,
which can cause datanode to run out of memory (OOM) during compaction.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-15 10:14:31 +08:00
5ae347aba0
enhance: update knowhere version ( #37688 )
...
issue: #37665 #37631 #37620 #37587 #36906
knowhere has add default nlist value, so some invalid param test ut with
no nlist param will be valid.
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-15 10:10:31 +08:00
1d06d4324b
fix: Int64 overflow in JSON encoding ( #37657 )
...
issue: ##36621
- For simple types in a struct, add "string" to the JSON tag for
automatic string conversion during JSON encoding.
- For complex types in a struct, replace "int64" with "string."
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-14 22:52:30 +08:00
c5485bb1b1
fix: Make GetShardLeaders skip retry on not loaded collection ( #37684 )
...
issue: #37532
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-14 21:14:40 +08:00
1b4f7e3ac1
enhance: Add more expr ut for chunked segment ( #37600 )
...
related pr: #37570
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-11-14 18:40:32 +08:00
405c12f7d1
test: update test cases ( #37660 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-14 17:46:39 +08:00
993051bb49
fix: brute force bm25 search lack avgdl param ( #37650 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-14 14:58:31 +08:00
31a8d08bdd
fix: Correct varchar primarykey size calculation ( #37617 )
...
See also: #37582
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-14 14:16:38 +08:00
cd181e4c6d
fix: make NewDataSyncService idempotent of dispatcher ( #37576 )
...
issue: #37547
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-14 11:00:31 +08:00
0565300b7f
fix: Sparse to use CC index as growing/temp index ( #37591 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-11-14 10:54:31 +08:00
1c5b5e1e3d
feat: Add chinese and english analyzer with refactor jieba tokenizer ( #37494 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-11-14 10:34:31 +08:00
1304b40552
fix: Balance channel may stuck at increasing replica number case ( #37641 )
...
issue: #37640
fix the pr #36549
cause balance channel will wait until new delegator becomes serviceable,
but new delegator need to sync target version then becomes serviceable,
and sync target version need to be wait all replica load done. so if
increasing replica number and balance channel happens at same time,
logic dead lock occurs.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-14 10:08:31 +08:00
caf207f5a0
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-11-13 12:00:38 +00:00
e7fb87d491
enhance: Fix lb policy retry log after targetNode type changed ( #37646 )
...
Related to previous pr: #37371
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-13 19:52:29 +08:00
af433ffd68
test: Add expression template tests and use error msg only as assertion ( #37618 )
...
related issue: https://github.com/milvus-io/milvus/issues/37451
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-11-13 17:14:30 +08:00
a654487995
fix: some error in restful ( #37487 )
...
#37370
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-13 17:12:39 +08:00
cf883b114e
enhance: update knowhere version ( #37510 )
...
issue: #36925
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-11-13 16:36:27 +08:00
a9538f6e96
fix: config file validator missed the file tail validation ( #37608 )
...
See: #32168
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-11-13 15:14:30 +08:00
1a49da2cc0
fix: refuse schedule compaction tasks if there is no slot ( #37589 )
...
See #37621
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-11-13 15:12:30 +08:00
3389a6b500
enhance: support null in text match index ( #37517 )
...
#37508
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-13 11:08:29 +08:00
30d396f476
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-11-13 00:01:25 +00:00
cfdff464dd
doc: Rewrite readme file ( #37428 )
...
Signed-off-by: codingjaguar <jiang.chen@zilliz.com>
2024-11-12 13:29:57 -08:00
3c225e5c94
fix: data race when using fields_ ( #37612 )
...
issue: #37609
Signed-off-by: chyezh <chyezh@outlook.com>
2024-11-13 04:06:30 +08:00
5ab3c562da
enhance: tidy compaction logs ( #37595 )
...
Remove some annoying logs and lower a log level from warn to info
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-11-12 21:18:29 +08:00
14e007d6fb
fix: Fix the bug that retrieved from wrong field for L0 segments ( #37598 )
...
issue: #37574
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-12 20:58:29 +08:00
cef5b7fd60
enhance: [cpp-unit-test] abort previous build if it's a PR, otherwise… ( #37571 )
...
[cpp-unit-test] abort previous build if it's a PR, otherwise queue the
build
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-11-12 19:14:29 +08:00
2630717836
test: update sdk version and cases ( #37507 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-11-12 17:48:28 +08:00
congqixia
Zhen Ye
Buqian Zheng
zhuwenxing
jaime
yihao.dai
MimoN
aoiasd
foxspy
sre-ci-robot
smellthemoon
zhagnlu
MimoN
Bingyi Sun
XuanYang-cn
wei liu
cai.zhang
nico
yanliang567
Ted Xu
codingjaguar
yellow-shine