Commit Graph

1602 Commits (7b21032d19d04a979c4e82e078aa33f3114ea408)

Author SHA1 Message Date
foxspy 48b14c56f6
fix: add prometheus dependency for milvus_monitor (#35078)
add prometheus dependency for monitor module. Or else Some compilers may
report a compilation failure.
issue: #35077

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-07-31 10:15:48 +08:00
zhagnlu dd0c26cf58
enhance: redefine variable column block size (#35040)
#35013

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-30 19:23:50 +08:00
congqixia de8a266d8a
enhance: Enable linux code checker (#35084)
See also #34483

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 15:53:51 +08:00
zhagnlu a8a4779749
fix: fix parse plan proto failed for search type (#34944)
#25848

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-29 21:19:49 +08:00
zhagnlu 86322e0468
fix: fix query count(*) concurrently (#35007)
#34778
#34849
fix two problems:
1. count(*) incorrect, if growing insert duplicated (pk, timestamp)
pairs that pk and timestamp all same, need to keep just one pair.
2. count(*) may core dump, if get_real_count interface get snapshot and
do mvcc at not consistency status, mainly happens under concurrency.

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-29 19:53:50 +08:00
congqixia 972752258a
enhance: Support otlp http exporter (#35053)
See also #35052

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-29 17:43:49 +08:00
zhagnlu f77f5364b2
fix: disable use_index when some array expr (#34894)
#34797

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-29 00:17:46 +08:00
Aldrin 9463eeef2b
fix: Avoided dereferencing NULL pointer (#34836)
issue : https://github.com/milvus-io/milvus/issues/34835

Signed-off-by: Ald392 <imagesai32@gmail.com>
2024-07-27 17:27:52 +08:00
zhagnlu 804ec24c02
fix:fix retrieve raw data from bitmap array index (#34848)
#34795

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-27 01:53:47 +08:00
sre-ci-robot e93687cdd3
[automated] Update Knowhere Commit (#35039)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-27 00:59:46 +08:00
smellthemoon 7ec9d856f3
fix: access address was not malloc (#34971)
issue: #34972  
fix string type data use memcpy to fill cause segv for not malloc enough
memory in advance.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-25 02:31:44 +08:00
smellthemoon 5616b7e8d2
enhance: support null in c data_datacodec and load null value (#32183)
1. support read and write null in segcore
    will store valid_data(use uint8_t type to save memory) in fieldData.
2. support load null
binlog reader read and write data into column(sealed segment),
insertRecord(growing segment). In sealed segment, store valid_data
directly. In growing segment, considering prior implementation and easy
code reading, it covert uint8_t to fbvector<bool>, which may optimize in
future.
3.  retrieve valid_data.
    parse valid_data in search/query.
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-23 16:07:51 +08:00
Min Tian a4aed9b0b5
enhance: new knowhere param for range_search (#34686)
issue: #34685 
knowhere needs a new json param `range_search_k` for RangeSearch to
early terminate the iterator.

Signed-off-by: min.tian <min.tian.cn@gmail.com>
2024-07-23 11:45:43 +08:00
sre-ci-robot 12b49859c0
[automated] Update Knowhere Commit (#34909)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-23 00:53:43 +08:00
Chun Han 6c19f9baf8
enhance: optimize search reduce perf(#32507) (#34607)
related: #32507

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-22 20:57:43 +08:00
Chun Han ed057e6fce
fix: non-init seg_offset for growing raw-data when doing groupby (#34748)
related:  #34713

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-19 17:01:40 +08:00
foxspy 8e64bf929c
enhance: add scalar filtering and vector search latency metrics (#34785)
add scalar filtering and vector search latency metrics to distinguish
the cost of scalar filtering.
To add metrics in query chain, add a monitor module and move the metric
files from original storage module.
issue: #34780

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-07-19 14:01:39 +08:00
sre-ci-robot c61592dcea
[automated] Update Knowhere Commit (#34819)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-19 02:09:42 +08:00
zhagnlu f1b2f7b640
enhance: refactor bitmap index and internal hybrid index (#34450)
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-18 10:39:42 +08:00
Buqian Zheng aa5418a5a9
fix: correctly set search params when using knowhere iterator (#34731)
issue: #34730

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-07-17 15:13:41 +08:00
zhagnlu 804dd5409a
enhance: mark duplicated pk as deleted (#34586)
fix #34247

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-16 14:25:39 +08:00
zhagnlu bd9727a1f7
fix: fix bug that set incorrect info to columnbase (#34428)
#34427

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-14 22:27:46 +08:00
zhagnlu 18c83c6466
fix: fix auto merge error (#34661)
#33704

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-13 13:07:37 +08:00
Chun Han f00c529aea
feat: support group_size for search_group_by(#33544) (#33720)
related: #33544

mainly changes in three aspects:

1. enable setting group_size for group by function
2. separate normal reduce and group by reduce
3. eleminate uncessary padding in search result for reducing

Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-12 10:17:36 +08:00
Patrick Weizhi Xu 104d0966b7
feat: support partition key isolation (#34336)
issue: #34332

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-07-11 19:01:35 +08:00
congqixia 4850336ca3
fix: Write padding at end of mmap file not chunk (#34529)
Related to #34508

The padding bytes shall be written only at the end of the mmap file not
the chunk of each field data file.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-10 11:12:14 +08:00
sre-ci-robot 0b3f317b14
[automated] Update Knowhere Commit (#34549)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-10 01:26:14 +08:00
congqixia 6b4d977a10
fix: Write padding into mmap file in case of SIGBUS (#34443)
See also #34442

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-05 17:44:09 +08:00
sre-ci-robot d65b6895d7
[automated] Update Knowhere Commit (#34432)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-05 01:18:09 +08:00
Gao ec5db7a57d
enhance: update knowhere version (#34361)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-02 19:18:24 +08:00
yihao.dai 734415b8a2
fix: Reduce duplicate PKs in segcore (#34267)
issue: https://github.com/milvus-io/milvus/issues/34247

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-01 17:42:06 +08:00
smellthemoon ef3ced8138
fix: descriptor event in previous version not has nullable to parse error (#34235)
#34176

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-01 16:38:06 +08:00
congqixia 14e827dc6c
fix: Implement singleflight for segcore ChunkCache (#34250)
See also #34249

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-01 11:46:06 +08:00
zhagnlu 3030e4625e
enhance: refactor variable column to reduce memory cost (#33875)
#33874

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-30 20:16:06 +08:00
zhagnlu cc1bc07bfd
enhance: add log to bitmap index (#34197)
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-30 20:02:06 +08:00
Cai Yudong ad90360162
enhance: Update knowhere commit (#34223)
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-06-27 18:20:06 +08:00
Gao a933f6731b
fix: centroids file not removed when data skew in major compaction (#34050)
issue: https://github.com/milvus-io/milvus/issues/30633

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-06-26 10:48:04 +08:00
sre-ci-robot cc54ab2e47
[automated] Update Knowhere Commit (#34172)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-26 02:20:03 +08:00
sre-ci-robot 254b555d53
[automated] Update Knowhere Commit (#34119)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-25 01:22:02 +08:00
Jiquan Long aa36f9feed
fix: [ut] regex query under unsupported index (#34087)
/kind improvement
issue: https://github.com/milvus-io/milvus/issues/29988

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-06-24 14:08:03 +08:00
Patrick Weizhi Xu b961767005
enhance: support integral type for MV and skip MV if there is only one category (#33161)
issue: #29892

---------

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-06-24 10:20:01 +08:00
zhagnlu 03a3f50892
enhance: add skip using array index when some situation (#33947)
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-23 21:26:02 +08:00
zhagnlu 0d7ea8ec42
enhance: Enhance and correct exception module (#33705)
#33704

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-23 21:22:01 +08:00
chyezh 259a682673
enhance: async search and retrieve in cgo (#33228)
issue: #30926, #33132
related pr: #33133

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-22 09:38:02 +08:00
sre-ci-robot d9f986439d
[automated] Update Knowhere Commit (#34040)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-21 01:50:01 +08:00
presburger dac20d41c0
fix: fix milvus gpu compile error (#33995)
NVIDIA RAFT uses customized versions of spdlog and fmt, which conflict
with the versions provided by Conan. Therefore, it is necessary to
isolate the build environment for knowhere to avoid these conflicts.
issue: https://github.com/milvus-io/milvus/issues/33991

Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-06-20 20:30:01 +08:00
cqy123456 dc4437ff82
enhance: use segment id and type to register in MmapChunkManager and opt malloc in variableChunk (#33993)
issue: https://github.com/milvus-io/milvus/issues/32984

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-20 17:42:02 +08:00
smellthemoon 2a1356985d
enhance: support null in go payload (#32296)
#31728

---------

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-06-19 17:08:00 +08:00
Jiquan Long 7b9462c0d3
enhance: fix copying hits of inverted index twice (#33968)
issue: https://github.com/milvus-io/milvus/issues/29793
The custom `VecCollector` have already transformed the results into
vector of offsets, no need to copy them twice.

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-06-19 12:40:01 +08:00
cqy123456 298e50b834
enhance: check index with data type (#33880)
issue: https://github.com/milvus-io/milvus/issues/22837
related: https://github.com/milvus-io/milvus/pull/33878

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-19 10:15:59 +08:00
Gao 0d20303e54
fix: fix binary vector data size (#33750)
issue: https://github.com/milvus-io/milvus/issues/22837

- fix byte size wrong for binary vectors
- fix the expect/actual error msg

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-06-18 21:39:59 +08:00
cqy123456 b460862537
fix: can't find Chunk struct after growing support mmap (#33951)
issue: https://github.com/milvus-io/milvus/issues/32984

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-18 18:37:58 +08:00
congqixia 3fdaae8792
fix: Return record with largest timestamp for entires with same PK (#33936)
See also #33883

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-18 15:55:59 +08:00
cqy123456 32f685ff12
enhance: growing segment support mmap (#32633)
issue: https://github.com/milvus-io/milvus/issues/32984

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-18 14:42:00 +08:00
presburger e83ecd5074
enhance: fix not find arrow::parquet lib (#33928)
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-06-18 00:22:01 +08:00
zhagnlu d43ec4db0b
enhance: support array bitmap index (#33527)
#32900

---------

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-16 21:51:58 +08:00
zhagnlu e422168f09
fix: readd timestamp index because segment timestamp not ordered (#33856)
#33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-16 21:48:01 +08:00
Yinzuo Jiang 7d74edd6dd
fix: update clang-tidy and clang-format from 10 to 12 (#33141)
Default llvm toolchain version in Ubuntu 20.04 is 10, while Ubuntu 22.04
does not have `clang-tidy-10` or `clang-format-10` by default.

issue: #33142

Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-06-13 15:27:58 +08:00
Buqian Zheng 47b04ea167
enhance: support sparse cardinal hnsw index (#33656)
issue: #29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-06-12 16:57:55 +08:00
Buqian Zheng 8cb350598c
enhance: Improve GetVectorById of Sparse Float Vector (#33209)
issue: #29419

* sparse float vector to support raw data mmap

For get vector from chunk cache, I added a unit test but marking it as
skipped due to a known issue. I have tested it locally.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-06-12 10:09:55 +08:00
Jiquan Long ecf2bcee42
enhance: speed up array-equal operator via inverted index (#33633)
fix: #33632

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-06-11 14:13:54 +08:00
chyezh f53ab54c5d
enhance: async cgo utility (#33133)
issue: #30926, #33132

- implement future-based cgo utility.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2024-06-09 22:55:53 +08:00
zhagnlu 6ce9df913f
fix: clean vector memory (#33692)
#33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-07 10:37:54 +08:00
cai.zhang 27cc9f2630
enhance: Support analyze data (#33651)
issue: #30633

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: chasingegg <chao.gao@zilliz.com>
2024-06-06 17:37:51 +08:00
wei liu b69740c8f3
enhance: Remove unnecessary log info during load segment (#33663)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-06 14:13:50 +08:00
sre-ci-robot fd191dd7db
[automated] Update Knowhere Commit (#33655)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-06 01:53:50 +08:00
cqy123456 703fc73f71
enhance: disk index support binary vector (#33631)
issue:https://github.com/milvus-io/milvus/issues/22837
related https://github.com/milvus-io/milvus/pull/33575

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-05 19:37:57 +08:00
zhagnlu 8ad26093ba
fix: fix load failure (#33599)
issue: #33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-05 19:19:51 +08:00
Gao 545d4725fb
fix: correct get vector data size for bf16/fp16/binary vector (#33377)
related #22837

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-06-05 14:31:57 +08:00
congqixia 597f4c5e03
enhance: Make hasMoreResult accurate when hit number larger than limit (#33609)
See also milvus-io/milvus-sdk-go#756

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-05 11:51:51 +08:00
zhagnlu c6f8a73bb2
enhance: optimize some cache to reduce memory usage (#33534)
#33533

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-06-04 14:09:47 +08:00
sre-ci-robot d25c755480
[automated] Update Knowhere Commit (#33573)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-06-04 01:55:46 +08:00
congqixia 2b285e5573
fix: Wrap init segcore tracing with golang timeout (#33494)
See also #33483

Wrap `C.InitTrace` & `C.SetTrace` with timeout preventing otlp
initializtion hangs forever when endpoint is not set correctly

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-03 19:25:51 +08:00
Buqian Zheng 4171414222
enhance: update knowhere version (#33490)
issue: https://github.com/milvus-io/milvus/issues/33489
update knowhere version to latest. remove usage of `seed_ef` as it be
replaced by existing `ef`.

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-31 10:17:50 +08:00
Jiquan Long 0c5d8660aa
feat: support inverted index for array (#33452)
issue: https://github.com/milvus-io/milvus/issues/27704

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-31 09:47:47 +08:00
Chun Han 416a2cf507
fix: query iterator lack results(#33137) (#33422)
related: #33137 
adding has_more_result_tag for various level's reduce to rectify
reduce_stop_for_best

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-05-30 17:51:44 +08:00
zhagnlu 589d4dfd82
enhance: optimize bitmap index (#33358)
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-05-30 13:09:43 +08:00
Alexander Guzhva 9f9dff33f1
fix: make milvus compilable using gcc-13 (#30149)
add a missing header

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
2024-05-27 14:52:53 +08:00
yihao.dai 5cf4161394
fix: Fix exception info is missing (#33393)
Replace based std::exception to prevent "object slicing"

issue: https://github.com/milvus-io/milvus/issues/33392

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-27 14:33:41 +08:00
sre-ci-robot b391781a2e
[automated] Update Knowhere Commit (#33340)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-24 01:43:40 +08:00
foxspy 3bec2c4928
enhance: Update Knowhere version (#33309)
/kind branch-feature

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-05-23 16:57:41 +08:00
Buqian Zheng c5918ffbdb
enhance: mark sparse inverted index as mmap-able (#33281)
issue: #29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-23 14:11:42 +08:00
cai.zhang 32d3e22d7d
fix: Throw an exception after all the threads in thread pool finished (#32810)
issue: #32487

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-05-23 11:47:40 +08:00
cai.zhang be77ceba84
enhance: Use proto for passing info in cgo (#33184)
issue: #33183

---------

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-05-23 10:31:40 +08:00
congqixia 3c4df81261
enhance: Assert insert data length not overflow int (#33248)
When InsertData is too large for cpp proto unmarshalling, the error
message is confusing since the length is overflowed

This PR adds assertion for insert data length.

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-22 15:11:39 +08:00
Alexander Guzhva 648d5661ca
enhance: Upgrade bitset for ARM SVE (#32718)
issue: #32826 
improve ARM SVE performance for `internal/core/src/bitset`

Baseline timings for gcc 11.4 + Graviton 3 + manually enabled SVE:
https://gist.github.com/alexanderguzhva/a974b50134c8bb9255fb15f144e5ac83

Candidate timings for gcc 11.4 + Graviton 3 + manually enabled SVE:
https://gist.github.com/alexanderguzhva/19fc88f4ad3757e05e0f7feaf563b3d3

Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
2024-05-22 11:37:40 +08:00
sre-ci-robot c6e2dd05fc
[automated] Update Knowhere Commit (#33147)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-20 01:51:37 +08:00
zhagnlu d669fbcf46
enhance: support bitmap index for scalar type (#32902)
#32900

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-05-19 21:49:38 +08:00
Buqian Zheng 7c60d725cc
fix: validate sparse vector in search request (#32856)
issue: #32368

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-15 15:39:33 +08:00
Gao 3777eb2f68
enhance: update knowhere new version (#33043)
for milvus new release, knowhere update to latest release version

Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-05-14 20:45:32 +08:00
sre-ci-robot 09cca1fb51
[automated] Update Knowhere Commit (#33030)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-14 01:47:32 +08:00
foxspy f6777267e3
enhance: add score compute consistency config for knowhere (#32997)
issue: https://github.com/milvus-io/milvus/issues/32583
related: #32584

Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-05-13 14:21:31 +08:00
Gao f7d29118a5
enhance: update knowhere version (#32931)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-05-10 16:13:31 +08:00
Buqian Zheng bb7765cbd6
fix: fix Indexing.Iterator ut: build index with all data at once (#32844)
issue: #32843

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-05-10 11:31:30 +08:00
aoiasd 54a51b1236
enhance: Support dynamic config for opentelemetry trace (#32169)
relate: https://github.com/milvus-io/milvus/issues/31940

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-09 17:43:30 +08:00
sre-ci-robot 3d78b90fe7
[automated] Update Knowhere Commit (#32872)
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-05-09 01:57:30 +08:00
Jiquan Long 035a508722
fix: make sure inverted index has only one segment (#32858)
issue: #32717

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-08 21:25:30 +08:00
Chun Han 01c2684355
enhance: [skip e2e] disable unstable ut temporarily (#32836)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-05-08 12:17:29 +08:00
Jiquan Long 9837ad6a8d
enhance: remove deprecated api (#32808)
issue: #32728

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-07 20:03:30 +08:00
Jiquan Long 1f58cda957
enhance: add more trace for search & query (#32734)
issue: https://github.com/milvus-io/milvus/issues/32728

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-07 13:03:29 +08:00