d6206ad2de
fix: Remove duplicated Macro definition ( #39076 )
...
Issue: #39102
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2025-01-09 15:26:56 +08:00
8abf6c9149
fix: build text index when loading field data ( #39070 )
...
fix: https://github.com/milvus-io/milvus/issues/39053
may fix https://github.com/milvus-io/milvus/issues/38644 which could be
caused by https://github.com/milvus-io/milvus/issues/39053
---------
Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
2025-01-09 15:24:56 +08:00
aceb972963
enhance: update the expr static web page ( #39082 )
...
issue: #39083
/kind improvement
Three new functions of the static web page:
1. The input box can be expanded and scrolled if it exceeds the maximum
size
2. Input history
3. It will simply check whether the quotation marks and brackets appear
in pairs
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-01-08 21:14:57 +08:00
5a4bafccc6
enhance: Add resource group api on Restful ( #38885 )
...
issue: #38739
enable to manage resource group by Restful
- CreateResourceGroup
- DescribeResourceGroup
- UpdateResourceGroup
- ListResourceGroup
- DropResourceGroup
- TransferReplica
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-01-08 19:40:57 +08:00
a89ac861da
fix: not add index name in alter index properties ( #38979 )
...
#38967
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-08 15:02:55 +08:00
f0dae81494
fix: set iterative filter hint to false when no expr specified ( #39033 )
...
issue: https://github.com/milvus-io/milvus/issues/39013
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-08 12:56:56 +08:00
134952b6c5
enhance: refactor the streaming node log ( #39052 )
...
issue: #38399
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-08 11:30:56 +08:00
47e7ea241e
enhance: Add log for case which target not update as expected ( #38944 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-01-07 17:45:03 +08:00
c5a7000a92
enhance: move streaming coord from datacoord to rootcoord ( #39007 )
...
issue: #38399
We want to support broadcast operation for both streaming and msgstream.
But msgstream can be only sent message from rootcoord and proxy.
So this pr move the streamingcoord to rootcoord to make easier
implementation.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2025-01-07 17:42:57 +08:00
3dc95153b7
fix: build break under debug mode ( #38790 )
...
See #38435
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2025-01-07 17:36:56 +08:00
182cac03e5
enhance: Use bitset or instead of bitwise set ( #39037 )
...
Related to #39003
Copying bitset value bit by bit is slow and CPU heavy, this PR utilizes
bitset operator "|=" to accelerate this procedure
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-07 15:02:56 +08:00
84f8047a86
fix: Fix Milvus build error ( #39008 )
...
Issue: #39005
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2025-01-07 14:22:56 +08:00
3739446a33
enhance: refine array view to optimize memory usage( #38736 ) ( #38808 )
...
related: #38736
700m data, array_length=10
non-mmap_offsets_uint64: 2.0G
mmap_offsets_uint64: 1.1G
mmap_offsets_uint32: 880MB
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2025-01-07 13:26:55 +08:00
72f5b85c05
enhance: Accelerate `find_first` by utilizing bitset simd methods ( #39004 )
...
Related to #39003
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-07 10:34:54 +08:00
8165044b6d
fix: fix query incorrect in case of concurrent delete ( #38991 )
...
#38961
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2025-01-06 15:14:54 +08:00
f0cddfd160
fix: Fix panic caused by removing directory ( #38622 )
...
https://github.com/milvus-io/milvus/issues/38604
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-06 10:54:54 +08:00
7128e36eb0
enhance: Use mvcc timestamp as guarantee ts if set ( #38980 )
...
When MvccTimestamp is set, it could be used as guarantee timestamp
directly instead of new ts allocated by scheduler reducing the waiting
time when delegator has tsafe lag
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2025-01-05 19:02:54 +08:00
bd280d6f98
fix: expression with capital AND and OR are not recognized ( #38927 )
...
fix https://github.com/milvus-io/milvus/issues/38864
AND, OR and NOT is not recognized in milvus parser
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-01-05 18:55:00 +08:00
9672eee78a
enhance: Increase the buffer capacity of notifyIndexChan to support concurrency ( #38957 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-01-05 18:53:01 +08:00
11bfc93683
[automated] Update Knowhere Commit ( #38993 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-01-04 01:16:53 +08:00
c6a7f48123
enhance: Fix using wrong pool for warmup ( #38941 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-03 17:50:54 +08:00
af08b5b311
enhance: Update Knowhere version ( #38942 )
...
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2025-01-03 14:28:53 +08:00
4245c5bed1
fix: text match panics when enable_match is set be false ( #38950 )
...
fix: https://github.com/milvus-io/milvus/issues/38949
---------
Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
2025-01-03 14:20:55 +08:00
d7623ab635
enhance: extract the task step execution process ( #38323 )
...
- issue: #38322
Signed-off-by: SimFG <bang.fu@zilliz.com>
2025-01-03 12:36:54 +08:00
d9ebb70b10
fix: search req transfer correctly ( #38955 )
...
issue: #37899
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2025-01-03 10:04:54 +08:00
aa0a87eda7
fix: Block warmup submit if pool full in sync mode ( #38690 )
...
https://github.com/milvus-io/milvus/issues/38692
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2025-01-02 15:04:58 +08:00
d3a5282eaa
enhance: add param for tuning max VARCHAR length and restore limit to 65535 ( #38884 )
...
issue: #38882
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 8e740e004151dd2c11918aad2857c1c8d1bd98f5)
2025-01-02 14:42:53 +08:00
907fc24f85
enhance: support null expr ( #38772 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2025-01-02 14:16:54 +08:00
accfacf012
fix: [skip e2e]update webui logic ( #38755 )
...
issue: #36621
* update collection detail page logic
* fix data page segments list
* remove useless js file
2024-12-31 18:25:00 +08:00
11f8f4a378
enhance: add intent ctx to rootcoord init ( #38439 )
...
issue: #35917
add intent ctx to rootcoord initialization process
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-31 18:22:53 +08:00
6b105837b4
enhance: add some apis in Restful ( #38733 )
...
add some apis in Restful #38709
- alter/drop collection properties
- alter/drop index properties
- alter collection field properties
- refresh load
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-31 11:28:51 +08:00
3822819942
enhance: Remove an undefined behavior in index writer ( #38657 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-31 10:42:52 +08:00
af0315f190
fix: Release compaction task lock when return function ( #38856 )
...
issue: #38851
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-30 22:44:49 +08:00
cb6eca8e91
fix: drop partition can not be successful if load failed ( #38793 )
...
fix #38649
when partition load failed, the partition drop will also fail due to the
wrong error message
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-12-30 19:42:52 +08:00
ba3c2e6fb1
fix: Only generate the index_null_offset file when the field support null value ( #38833 )
...
issue: #38832
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2024-12-30 18:02:52 +08:00
07c1f43d95
fix: the retrieve plan on heap is used after free when reduce ( #38840 )
...
issue: #38731
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-30 17:50:51 +08:00
56c5b66619
fix: the close operation of rmq consumer is not sync ( #38734 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-30 16:04:51 +08:00
dd81a13821
fix: used wrong label for query received_nq metrics ( #37987 )
...
issue: #37982
Signed-off-by: Reid Chan <reidchan@apache.org>
2024-12-30 11:10:52 +08:00
2557e3f2a9
enhance: Initialize field id to avoid negative number ( #38789 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-27 18:00:50 +08:00
4ba0ed3178
enhance: speed up streaming barrier timetick ( #38787 )
...
issue: #38399
- use last allocate but not last confirmed id to make barrier.
- move the barrier logic into the timetick allocator.
- try to sync up local allocator and remote allocator when first barrier
check not pass to speed up.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-27 15:08:50 +08:00
19052ef3e5
enhance: Add buffered writer to reduce fwrite syscall ( #38570 )
...
Related to previous PR #38157
If mmapped row is too small, frequent fwrite call still cost too much
cpu time for context switching. This PR add buffered write to avoid this
bad case with extra buffer per variable field.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-27 12:20:50 +08:00
4df444ef25
fix: Aviod add negative missing count ( #38748 )
...
See also: #34665
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-26 18:58:56 +08:00
94955e5292
fix: expand privilege group when list policy in rootcoord ( #38758 )
...
related: https://github.com/milvus-io/milvus/issues/38757
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-26 15:38:50 +08:00
85f462be1a
enhance: speed up search iterator stage 1 ( #37947 )
...
issue: #37548
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-12-26 10:32:49 +08:00
f49d618382
fix: Querycoord will trigger unexpected balance task after restart ( #38630 )
...
issue: #38606
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 19:30:48 +08:00
bc15ad24f2
fix: sealed segment get empty index params when brute force search for bm25 ( #38707 )
...
relate: https://github.com/milvus-io/milvus/issues/38236
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-25 19:06:51 +08:00
363d7f31ef
fix: report error when hints not supported ( #38717 )
...
issue: #38705
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-12-25 19:02:56 +08:00
c7ea09a8be
enhance: return exception type name when segcore return unkonwn exception ( #38326 )
...
relate: https://github.com/milvus-io/milvus/issues/38265
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-25 18:58:50 +08:00
25f0c82ceb
fix: Fix update loading collection's load config doesn't work ( #38595 )
...
issue: #38594
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 18:02:51 +08:00
6bc799061e
fix: fix privilege group list and list collections ( #38684 )
...
related: #37031
* built-in privilege group privileges in listPrivilegeGroups() should be
the same as in milvus.yaml
* collections granted by collection level built-in privilege group
should be list in showCollections()
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-25 18:00:51 +08:00
9c3f59dbbe
fix: Prevent balancer from overloading the same QueryNode ( #38719 )
...
issue: #38718
The balancer calculates the workload of executing tasks as an ongoing
score for target nodes. However, a logic issue arises when
GetSegmentTaskDelta or GetChannelTaskDelta is called with
collectionID=-1, which incorrectly returns zero.
Due to the incorrect global score, the executing task's workload is not
properly reflected for each collection. Consequently, each collection
submits its own balance task, leading to the balancer assigning
excessive tasks to the same QueryNode.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-25 16:36:49 +08:00
acc8fb7af6
enhance: eliminate compile warnings (part2) ( #38535 )
...
See #38435
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-25 15:30:50 +08:00
5395ec19ad
enhance: add mem size for index file and fallback to multiply with serialized size ( #38716 )
...
issue: #38715
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 14:46:49 +08:00
c731357538
enhance: Add missing delete metrics ( #38634 )
...
Add 2 counter metrics:
- Total delete entries from deltalog:
milvus_datanode_compaction_delete_count
- Total missing deletes: milvus_datanode_compaction_missing_delete_count
See also: #34665
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-25 10:24:50 +08:00
285289d5b0
enhance: implement kafka for wal ( #38598 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 10:22:50 +08:00
05d9f8562f
fix: fix complie error generated by conflict pr ( #38729 )
...
issue: #38399
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-25 10:14:49 +08:00
69a9fd6ead
enhance: enable rmq for streaming ( #38669 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-24 20:24:48 +08:00
2b53b0905e
fix: Fix 0 read count during import ( #38694 )
...
issue: https://github.com/milvus-io/milvus/issues/38693
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-24 20:06:49 +08:00
c458dec206
fix: [skip e2e]update webui collection and query pages ( #38700 )
...
issue: #36621
* enhance collection detail page list columns
* enhance query page segments state detail logic
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-12-24 17:36:48 +08:00
636e107e44
fix: Remove sync task after finished ( #38681 )
...
Related to #38680
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-24 13:34:48 +08:00
d718bbb4de
fix: fix altercollectionfield interceptor dbname ( #38662 )
...
fix altercollectionfield interceptor dbname
issue: https://github.com/milvus-io/milvus/issues/37436
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-23 18:24:48 +08:00
bb5f38e574
enhance: Return collection not loaded rather than not found on querynode ( #38593 )
...
issue: #38586
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-23 14:36:48 +08:00
2356425530
fix: Support hybrid search with expression template ( #38623 )
...
issue: #38616
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-23 14:32:51 +08:00
205231b9c7
fix: Decode unicode for json key in expression ( #38651 )
...
issue: #38626
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-23 13:40:49 +08:00
5afd0c0a2b
fix: Revert "Expose metrics of stanby coordinators ( #27698 )" ( #38620 )
...
issue: #38608
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-23 11:46:57 +08:00
984a605d47
enhance: use lazy initializing client for streaming node ( #38400 )
...
issue: #38399
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-23 11:36:47 +08:00
f499ca47a0
fix: [skip e2e]update webui ( #36621 ) ( #38633 )
...
issue: #36621
* add collection detail page
* enhance query page segments & channels state display
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-12-23 11:00:49 +08:00
407035c994
[automated] Update Knowhere Commit ( #38641 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-22 00:44:46 +08:00
cce25ecdbc
[automated] Update Knowhere Commit ( #38635 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-21 00:50:46 +08:00
90de37e60a
enhance: field stringtype maxlength raise to 1M ( #38592 )
...
enhance: field stringtype maxlength raise to 1M
issue: https://github.com/milvus-io/milvus/issues/37436
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-20 14:32:45 +08:00
ca7ec23198
enhance: Use partitionID when delete by partitionKey ( #38231 )
...
When delete by partition_key, Milvus will generates L0 segments
globally. During L0 Compaction, those L0 segments will touch all
partitions collection wise. Due to the false-positive rate of segment
bloomfilters, L0 compactions will append false deltalogs to completed
irrelevant partitions, which causes *partition deletion amplification.
This PR uses partition_key to set targeted partitionID when producing
deleteMsgs into MsgStreams. This'll narrow down L0 segments scope to
partition level, and remove the false-positive influence
collection-wise.
However, due to DeleteMsg structure, we can only label one partition to
one deleteMsg, so this enhancement fails if user wants to delete over 2
partition_keys in one deletion.
See also: #34665
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-20 11:18:46 +08:00
c0b855dc75
fix: ChannelManager concurret Release and Watch bug ( #38590 )
...
See also: #38589
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-12-19 22:50:47 +08:00
8fcb33c21d
fix:fix delete record assert failed ( #38580 )
...
#38472
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-19 18:22:47 +08:00
3d360c0624
fix: SyncSegments rpc always failed ( #38578 )
...
miss the patch due to code branching
previous pr: #38032
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
2024-12-19 15:40:45 +08:00
b537a72309
fix: interted index out of range ( #38577 )
...
issue: #38546 , #38486
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-19 15:20:47 +08:00
306e5e6898
enhance: clean compaction task in compactionHandler ( #38170 )
...
issue: #35711
---------
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2024-12-19 12:36:47 +08:00
78438ef41e
fix: revert optimize CPU usage for CheckHealth requests ( #35589 ) ( #38555 )
...
issue: #35563
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-19 00:38:45 +08:00
134d06f7e6
fix: need pass parameter to listPrivilegeGroups ( #38562 )
...
issue: https://github.com/milvus-io/milvus/issues/38563
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-18 20:28:51 +08:00
de8858931a
fix: rbac revoke check if there is same privilege in other privilege groups granted ( #38558 )
...
related: https://github.com/milvus-io/milvus/issues/38557
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-18 18:46:45 +08:00
a348122758
fix: Support get segments from current segments view ( #38512 )
...
issue: #38511
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-18 18:00:54 +08:00
06d410b70f
enhance: update knowhere version ( #38544 )
...
related: #37730
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-12-18 16:52:45 +08:00
87056be748
fix: fix snapshot or size when query ( #38549 )
...
#38472
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-18 16:42:45 +08:00
894c203f27
enhance: allow hyphen in partition name ( #38494 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-18 15:04:45 +08:00
6ee94d00b9
fix:fix calculate arrow nest type and add ut ( #38527 )
...
#37767
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-18 11:54:44 +08:00
ffd3c5d2f5
[automated] Update Knowhere Commit ( #38542 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-18 01:24:47 +08:00
7a05b5bbea
fix: Fix stats task wrong RootPath when upload binlog ( #38539 )
...
issue: #38336
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-18 00:18:45 +08:00
1aa31e2a9b
fix: Fix inaccurate general count ( #38524 )
...
Checking general count should only count healthy collections and
partitions.
issue: https://github.com/milvus-io/milvus/issues/37630
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 23:24:45 +08:00
4d5572d7ed
enhance: [skip e2e] refine display format ( #38537 )
...
issue: #36621
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-17 22:30:44 +08:00
decdfdae10
fix: growing-groupby-crush( #38533 ) ( #38538 )
...
related: #38533
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-12-17 21:05:12 +08:00
f0096ec292
fix: Fix IsMmapSupported for scalar index ( #38135 )
...
https://github.com/milvus-io/milvus/issues/38134
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-17 20:30:44 +08:00
9afcc5bc5c
fix:fix incorrect dir operations when create or load inverted index ( #38359 )
...
#37944
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-17 20:06:45 +08:00
93fba1d5ab
fix: access log remove new log not the old and skip rotate empty log ( #38309 )
...
relate: https://github.com/milvus-io/milvus/issues/38293
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-17 19:16:44 +08:00
29e620fa6d
fix: sync task still running after DataNode has stopped ( #38377 )
...
issue: #38319
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-17 18:06:44 +08:00
d0a7e98a27
fix:remove incorrect assert for delete query ( #38509 )
...
#38472
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-17 17:48:44 +08:00
8916fbf122
fix: use new path for streamingnode recovery info ( #38516 )
...
issue: #38399
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-17 17:10:45 +08:00
d3c174b0f1
enhance: Accelerate observe collection ( #38028 )
...
1. A collection should observe the channel only once.
2. A collection should check the CollectionLoadPercent for updates only
once.
3. Skip saving coll/partition meta if there are no changes, primarily to
accelerate collection observation after recovery.
issue: https://github.com/milvus-io/milvus/issues/37630
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 14:14:45 +08:00
d4dab3c62f
enhance: Reduce segmentManager lock granularity ( #37836 )
...
Use a channel level key lock for segments in segmentManager.
issue: https://github.com/milvus-io/milvus/issues/37633 ,
https://github.com/milvus-io/milvus/issues/37630
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 14:12:52 +08:00
dd4f33ae19
fix: Fix chunked segment can not warmup using mmap ( #38492 )
...
issue: #38410
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-17 13:42:45 +08:00
33aecb0655
fix: build break on target test_cpp under OSX ( #38479 )
...
See: #38434
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-17 13:38:45 +08:00
01fa2f8da5
fix: Fix compile error ( #38510 )
...
/kind improvement
related: #37559
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-17 12:16:19 +08:00
afac153c26
enhance: move the lifetime implementation out of server level lifetime ( #38442 )
...
issue: #38399
- move the lifetime implementation of common code out of the server
level lifetime implementation
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-17 11:42:44 +08:00
28fdbc4e30
enhance: optimize CPU usage for CheckHealth requests ( #35589 )
...
issue: #35563
1. Use an internal health checker to monitor the cluster's health state,
storing the latest state on the coordinator node. The CheckHealth
request retrieves the cluster's health from this latest state on the
proxy sides, which enhances cluster stability.
2. Each health check will assess all collections and channels, with
detailed failure messages temporarily saved in the latest state.
3. Use CheckHealth request instead of the heavy GetMetrics request on
the querynode and datanode
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-17 11:02:45 +08:00
2afe2eaf3e
feat: support to replicate collection when the services contains the system tt msg ( #37559 )
...
- issue: #37105
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-17 09:08:46 +08:00
3e2a2f278b
enhance: Handle rust error in c++ ( #38113 )
...
https://github.com/milvus-io/milvus/issues/37930
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-16 19:40:45 +08:00
659847c11f
enhance: Remove load task limit in one round ( #38436 )
...
the task limit in assignSegment/assignChannel will works for both load
task and balance task.
this PR remove the load task limit, only limit balance task num in one
round.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-16 19:30:43 +08:00
9c8c1b3bb7
enhance: Remove levelZeroMut totally ( #38473 )
...
The level zero mutex could be remove since all operations are guarded by
segment manager mutex
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-16 14:38:45 +08:00
40f9db491e
fix: Fix SyncDistribution may cost too much time on retry ( #38454 )
...
issue: #38428
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-16 11:38:44 +08:00
4919ccf543
enhance: eliminate compile warnings ( #38420 )
...
See: #38435
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-16 09:58:43 +08:00
c3edc85359
fix: fix wrong size of arrow array for zero-copy mode ( #38449 )
...
#37767
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-15 18:14:43 +08:00
01de0afc4e
enhance: refactor delete mvcc function ( #38066 )
...
#37413
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-15 18:02:43 +08:00
6ea15265e1
enhance: add file info log when mmap failed. ( #38386 )
...
#37944
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-15 17:36:43 +08:00
fa8ac09550
fix: the issue of replicate message exception when the ttMsgEnable config is changed dynamically ( #38178 )
...
- issue: #38177
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-14 23:24:51 +08:00
27229f7907
enhance: refine exists log print with ctx ( #38080 )
...
issue: #35917
Refines exists log print with ctx
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-14 22:36:44 +08:00
1e274384cd
[automated] Update Knowhere Commit ( #38458 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-14 00:54:43 +08:00
b416ff8c6c
enhance: altercollectionfield support mmap_enable convert to mmap.enable ( #38448 )
...
altercollectionfield support mmap_enable convert to mmap.enable
issue: https://github.com/milvus-io/milvus/issues/37436
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-13 20:02:43 +08:00
c1f9158996
fix: search-group-by failed to get data from multi-chunked-segment(##… ( #38383 )
...
related: #38343
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-12-13 16:54:43 +08:00
3038383e36
fix: UT compile broken under osx ( #38432 )
...
See: #38434
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-13 16:24:43 +08:00
efbfa1cc3e
fix:fix ut failed for debug ( #38384 )
...
#38382
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-13 14:38:43 +08:00
833c74aa66
enhance: add detail, replica count for resource group ( #38314 )
...
issue: #30647
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-13 14:14:50 +08:00
10460ed3f8
enhance: Simplify querynode tsafe & reduce goroutine number ( #38416 )
...
Related to #37630
TSafe manager is too complex for current implementation and each
delegator need one goroutine waiting for tsafe update event.
Tsafe updating could be executed in pipeline. This PR remove tsafe
manager and simplify the entire logic of tsafe updating.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-13 10:56:43 +08:00
6ffc57c8dc
fix: Fix sorting buffer in clustering compaction ( #38417 )
...
issue: #28410
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-13 10:12:49 +08:00
e404123e3e
[automated] Update Knowhere Commit ( #38422 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-13 02:40:44 +08:00
f2d79b3d3a
enhance: altercollection validation logic ( #38419 )
...
altercollection validation logic
issue: https://github.com/milvus-io/milvus/issues/37436
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-12 21:42:42 +08:00
d0a8110a7a
enhance: alterdatabase support delete property ( #38380 )
...
alterdatabase support delete property
issue: https://github.com/milvus-io/milvus/issues/38379
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-12 17:04:43 +08:00
75e64b993f
enhance: add metrics for counting number of nun-zeros/tokens of sparse/FTS search ( #38329 )
...
sparse vectors may have arbitrary number of non zeros and it is hard to
optimize without knowing the actual distribution of nnz. this PR adds a
metric for analyzing that.
issue: https://github.com/milvus-io/milvus/issues/35853
comparing with https://github.com/milvus-io/milvus/pull/38328 , this
includes also metric for FTS in query node delegator
also fixed a bug of sparse when searching by pk
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-12-12 16:22:43 +08:00
b14a0c4bf5
fix:GrowingDataGetter get the wrong string data ( #38015 )
...
issue: https://github.com/milvus-io/milvus/issues/37994
2.4 pr: https://github.com/milvus-io/milvus/pull/37995
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-12-12 14:50:42 +08:00
a514f839b2
enhance: Speed up meta recovery ( #38285 )
...
Increase the batchSize in WalkWithPrefix operations to 10000.
issue: https://github.com/milvus-io/milvus/issues/37630
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-12 10:58:43 +08:00
304cdc7783
fix: Check target partition load state only when dropping partition ( #38375 )
...
Related to #38372
This PR make drop partition only check target partition load states only
in case of concurrent releasing other partition in same collection.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-11 21:06:42 +08:00
8977454311
enhance: support recall estimation ( #38017 )
...
issue: #37899
Only `search` api will be supported
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-12-11 20:40:48 +08:00
dc85d8e968
enhance: improve mix compaction performance by removing max segment limitations ( #38344 )
...
See #37234
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-11 20:38:42 +08:00
e279ccf109
enhance: Enable score based balance channel policy ( #38143 )
...
issue: #38142
current balance channel policy only consider current collection's
distribution, so if all collections has 1 channel, and all channels has
been loaded on same querynode, after querynode num increase, balance
channel won't be triggered.
This PR enable score based balance channel policy, to achieve:
1. distribute all channels evenly across multiple querynodes
2. distribute each collection's channel evenly across multiple
querynodes.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-11 17:20:43 +08:00
0d7a89a4f8
fix: Use the correct RootPath when decompressing binlog in stats task ( #38341 )
...
issue: #38336
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-11 16:16:42 +08:00
d3ae8e9232
fix: delay the wait other coord logic in query coord after query coord change into standby state ( #38259 )
...
issue: https://github.com/milvus-io/milvus/issues/37764
- After removing rpc layer from mixcoord, the querycoord at standby mode
will be blocked forever of deployment rolling
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-11 15:48:42 +08:00
43e0e2b7ed
fix: Fix empty import task result ( #38316 )
...
Ensure the idempotency of import tasks to prevent duplicate tasks in
DataNode.
issue: https://github.com/milvus-io/milvus/issues/38313
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-11 15:42:49 +08:00
9be106dedf
enhance: Refine task scheduler logs ( #38334 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-11 15:00:44 +08:00
7c55649585
enhance: Refine querynode collection number metrics ( #38350 )
...
Related to #37630
Previously the loaded collection metrics was calculated via scanning all
loaded segment in segment manager, which is slow and buggy
implementation.
This PR:
- Move collection num metrics to collection manager
- Remove deprecated loaded partition metrics update logic
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-11 11:36:43 +08:00
994fc544e7
enhance: support iterative filter execution ( #37363 )
...
issue: #37360
---------
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-12-11 11:32:44 +08:00
a118ca14a7
fix: Fix role be dropped when grant still exist. ( #38342 )
...
issue: #38325
the old impl only to check grant in default db before drop role, which
may cause role be dropped when grant still exist.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-11 11:24:42 +08:00
e4a9a473ee
enhance: keep the latest snapshot key if the origin key is existed ( #38333 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-12-11 10:36:43 +08:00
9ef76971ce
fix:add more info to local chunk manager log ( #38357 )
...
#37944
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-11 10:24:49 +08:00
db05d4f976
enhance: alterindex & altercollection supports altering properties ( #37437 )
...
enhance :
1. alterindex delete properties
We have introduced a new parameter deleteKeys to the alterindex
functionality, which allows for the deletion of properties within an
index. This enhancement provides users with the flexibility to manage
index properties more effectively by removing specific keys as needed.
2. altercollection delete properties
We have introduced a new parameter deleteKeys to the altercollection
functionality, which allows for the deletion of properties within an
collection. This enhancement provides users with the flexibility to
manage collection properties more effectively by removing specific keys
as needed.
3.support altercollectionfield
We currently support modifying the fieldparams of a field in a
collection using altercollectionfield, which only allows changes to the
max-length attribute.
Key Points:
- New Parameter - deleteKeys: This new parameter enables the deletion of
specified properties from an index. By passing a list of keys to
deleteKeys, users can remove the corresponding properties from the
index.
- Mutual Exclusivity: The deleteKeys parameter cannot be used in
conjunction with the extraParams parameter. Users must choose one
parameter to pass based on their requirement. If deleteKeys is provided,
it indicates an intent to delete properties; if extraParams is provided,
it signifies the addition or update of properties.
issue: https://github.com/milvus-io/milvus/issues/37436
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-11 10:20:42 +08:00
950203aba0
enhance: Optimize save colelction target latency ( #38345 )
...
issue: #38237
this PR only use better compression level for proto msg which is larger
than 1MB, and use a lighter compression level for smaller proto msg,
which could get a better latency in most case.
this PR could reduce the latency from 22.7s to 4.7s with 10000
collctions and each collections has 1000 segments.
before this PR:
BenchmarkTargetManager-8 1 22781536357 ns/op 566407275088 B/op 11188282
allocs/op
after this PR:
BenchmarkTargetManager-8 1 4729566944 ns/op 36713248864 B/op 10963615
allocs/op
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-11 10:12:43 +08:00
7ea9c983d2
enhance: Add mockery package config for QC&QN ( #38340 )
...
Related to #38339
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-10 19:18:42 +08:00
856e2aad7d
fix: Leader task stuck and retry again and again ( #38202 )
...
issue: #38201
leader task require to update delegator's distribution, and only success
after the distribution change has been applyed to delegator. but the
delegator will reject the distribution change if it's version is older
than current version in delegator. which cause the leader task stuck and
retry forever.
this PR remove the leader task finish check.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-10 19:16:42 +08:00
b8e3795374
enhance: Add secondary index for querynode segment manager ( #38311 )
...
Related to #37630
Add secondary index with vchannel to reduce `GetBy` rlock holding time
when segment number is large.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-10 18:18:42 +08:00
fec31fedce
enhance: Move `SyncCreatedPartition` step to proxy ( #38296 )
...
Related to #38275
This PR move sync created partition step to proxy to avoid potential
logic deadlock when create partition happens with target segment change.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-10 12:30:43 +08:00
f04986fceb
enhance: Remove constraint on release segment task ( #38297 )
...
issue: #38305
after we disable balance segment and balance channel happens at same
time, the constriant which require release segment must happens on
serviceable shard leader is unnessary.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-10 11:18:49 +08:00
7a5aea116d
fix: Fix init rootcoord meta timeout ( #38248 )
...
issue: https://github.com/milvus-io/milvus/issues/37630
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-08 18:12:47 +08:00
41b19c6b1d
enhance: Determine the number of buffers based on the resource limits of the DataNode ( #38209 )
...
issue: #28410
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-08 18:02:40 +08:00
32f575be0f
enhance: change bitmap index mmap mode to view mode ( #38179 )
...
#38138
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-08 17:26:41 +08:00
c522ce84b4
fix:remove unnecessary error logs ( #38245 )
...
#38241
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-12-08 17:20:40 +08:00
28c5189c6d
enhance: Refine the error msg for filter node ( #38278 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-06 18:22:40 +08:00
2035575941
fix: Datanode stop progress stuck at writer buffer memory check ( #38274 )
...
issue: #38273
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-06 18:20:39 +08:00
8ed019735c
enhance: add disk stats within system metrics ( #38033 )
...
issue: ##36621
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-06 16:32:41 +08:00
60dd55f292
enhance: add end time stats for compaction and index build tasks ( #38048 )
...
issue: #36621
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-06 16:28:40 +08:00
0d4073bdd8
enhance: use require dbname for some db req ( #38267 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-06 16:26:46 +08:00
36946cc9ce
enhance: Set loaded collection/partition number to metrics ( #38271 )
...
Related to #36456
Previous PR: #38471 #38233
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-06 16:18:40 +08:00
f16232f834
fix: sparse inc mast less than uint32 max ( #38250 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-06 15:14:41 +08:00
e1aebe8fce
enhance: Support collection name for ProxyDeleteVectors as well ( #38247 )
...
issue: https://github.com/milvus-io/milvus/issues/38242
Signed-off-by: Reid Chan <reidchan@apache.org>
2024-12-06 15:00:40 +08:00
18bef5e062
fix: fix crash when enable standby and streaming ( #38239 )
...
issue: #38125
- connect kv at standby mode.
- make balancer initialization lazy.
Signed-off-by: chyezh <chyezh@outlook.com>
2024-12-06 11:52:40 +08:00
6ff19481f0
enhance: Resolve compilation error due to PR conflict ( #38252 )
...
Related pr: #38233 #38059
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-05 19:26:40 +08:00
c91bb5a655
enhance: report restful info to hook ( #38112 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-05 19:02:41 +08:00
051bc280dd
enhance: Make dynamic load/release partition follow targets ( #38059 )
...
Related to #37849
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-05 16:24:40 +08:00
32645fc28a
enhance: Unify querycoord meta metrics ( #38233 )
...
Related to #36456
Unify collection/partition number metrics to collection manager in case
of unwant missing modification
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-05 15:48:39 +08:00
7944538ade
enhance: Add ctx param to KV operation interfaces ( #38154 )
...
issue: #35917
Refine KV operation interfaces by adding a ctx param
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-05 15:16:41 +08:00
618f0cb728
enhance: Put release segment and other misc cgo call into pool ( #38186 )
...
Related to #30273
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-05 11:04:40 +08:00
af288a9c21
fix: compilation failed due to CI conflicts ( #38222 )
...
issue: #38224
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-12-05 10:59:38 +08:00
3c6a373a75
enhance: Fill version for load delta request ( #38212 )
...
Version is needed for load delta request in case of false alarm warning
about version go backward
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-05 10:54:39 +08:00
6d0a4fdb31
fix: Fix bug for Search fails with filter expression contains underscore ( #38085 )
...
Enhance the matching for elements within the UnaryRangeArray
https://github.com/milvus-io/milvus/issues/38068
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
2024-12-05 10:18:40 +08:00
5768dbbb5d
enhance: refine pular related mq interfaces ( #38007 )
...
issue: #35917
Refines the pulsar-related mq APIs to allow the ctx to be passed down
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-04 20:50:39 +08:00
73aa95f596
fix: Add version to the proxy cache to resolve concurrency issues ( #38067 )
...
issue: #36989
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-04 18:06:39 +08:00
a65d395ecd
fix: grant v2 collection name and privilege error typo ( #38195 )
...
issue: https://github.com/milvus-io/milvus/issues/38042
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-04 16:14:38 +08:00
262f6db3d8
enhance: Add mmap file usage metric ( #38193 )
...
issue: #38156 Add mmap file usage metric
Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>
2024-12-04 16:12:47 +08:00
87aa9a0f2d
fix: empty analyzer params not use standard tokenizer ( #38148 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-12-04 14:58:39 +08:00
1f66b9ebfb
feat: add config field to set internal tls sni ( #38124 )
...
/cc @xiaofan-luan @jaime0815 @nish112022
part of https://github.com/milvus-io/milvus/issues/36864
Signed-off-by: haorenfsa <haorenfsa@gmail.com>
2024-12-04 14:56:47 +08:00
cbba29677a
fix: update web ui ( #38137 )
...
issue: #36621
- add filters to most of lists
- add more options to the existing list filter
- fix the error list column in channels and resource group list
Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>
2024-12-04 14:32:39 +08:00
f211e9970f
enhance: enable limiter for restful v1 ( #38160 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-04 10:42:38 +08:00
3445b8bd44
[automated] Update Knowhere Commit ( #38192 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-04 02:30:39 +08:00
d822edb9a8
enhance: add list aliases privilege into public role ( #38176 )
...
related issue: https://github.com/milvus-io/milvus/issues/37031
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-03 23:42:39 +08:00
db2ed2c044
fix: Fix inaccurate partition num metric ( #37996 )
...
The partition number has already been incremented in
`ChangePartitionState`, so there is no need to increment it again in
`AddPartition`.
issue: https://github.com/milvus-io/milvus/issues/37630
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-03 21:24:40 +08:00
e359725530
enhance: support db request in Restful api ( #38140 )
...
#38077
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-03 20:04:41 +08:00
4537e2cc46
fix: Fix timeout when listing meta ( #38151 )
...
When there are too many key-value pairs, the etcd list operation may
times out. This PR replaces `LoadWithPrefix` in list operations, which
could involve many keys, with `WalkWithPrefix`.
issue: https://github.com/milvus-io/milvus/issues/37917
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-03 16:28:39 +08:00
767b7e6218
enhance: Use fdopen, fwrite to reduce direct syscall ( #38157 )
...
`File.Write` and `File.WriteInt` use `write`, which may be just direct
syscall in some systems. When mappding field data and write line by
line, this could cost lost of CPU time when the row number is large.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-12-03 15:24:39 +08:00
28d39399e2
enhance: Refine clustering compaction log ( #38100 )
...
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-03 14:58:39 +08:00
9e008685ef
fix: Fix datacoord metrics ( #38163 )
...
issue: https://github.com/milvus-io/milvus/issues/38162
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-03 14:10:45 +08:00
e09f431891
fix: grant ManualCompact api doesn't work ( #38096 )
...
issue: #38086
cause ManualCompact api pass collection id in request, but RBAC requires
to check collection name, so grant ManualCompact api doesn't work.
This PR refine the ManualCompact api to accpet collection name in
request.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-12-03 10:36:38 +08:00
a5e0a56a8e
fix: move grant/revoke v2 params check from rootcoord to proxy ( #38130 )
...
related issue: https://github.com/milvus-io/milvus/issues/37031
fixed issues #38042 : The interface "grant_v2" does not support empty
collectionName while the error says it does
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-02 19:48:37 +08:00
db453c063d
enhance: no need database check in restful sdk ( #38078 )
...
#38077
remove the check for two reason
1. server will do the same to make sure use the correct database;
2. each req has an additional overhead of calling the proxy to check
database.
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-02 17:04:38 +08:00
cb926688f7
enhance: support templates for expression in Restful api ( #38040 )
...
#36672
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-12-02 16:54:38 +08:00
3a7a8c7944
enhance: try compact small segments first if they may compose a full segment ( #37709 )
...
See #37234
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-12-02 16:12:38 +08:00
90064cd47b
fix: Fix variable redeclaration in term filter ( #38045 )
...
https://github.com/milvus-io/milvus/issues/38046
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-12-02 15:10:38 +08:00
23dc313c44
fix: fix grant/revoke v2 meta and unclear error messages ( #38110 )
...
related issue: https://github.com/milvus-io/milvus/issues/37031
fixed issues:
#37974 : better error messages for grant v2 interface
#37903 : fix meta built-in privilege group object name
#37843 : better error messages for custom privilege group interface
#38002 : fix built-in privilege group meta to pass proxy interceptor
check
#38008 : fix revoke v2 to support revoking v1 granted privileges
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-12-02 11:36:39 +08:00
7fb0c281f2
fix: Fix checkGeneralCapacity slowly ( #37976 )
...
Cache the general count to speed up `checkGeneralCapacity`.
issue: https://github.com/milvus-io/milvus/issues/37630
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-12-01 18:28:38 +08:00
2319018fcb
fix: Check if the dynamic fields contain any static fields ( #38025 )
...
issue: #38024
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-12-01 18:06:37 +08:00
108434969f
enhance: Add collection id to search request count metrics ( #38069 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-11-29 17:28:38 +08:00
61f73d552f
fix: Return thread watcher goroutine after closed ( #38091 )
...
Resolves #38090
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-29 15:10:36 +08:00
a9080d8d96
enhance: the actual number of databases should equal the config value ( #38006 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-11-29 14:44:36 +08:00
48ed586167
enhance: pre allocate space to prevent slice grow ( #38065 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-11-29 14:26:43 +08:00
9d3035c8ab
enhance: Fix conflict complication error ( #38087 )
...
Related to #37957 #37989
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-29 12:00:36 +08:00
5e152767a3
fix: Handle the error of the compaction queue being full ( #37989 )
...
issue: #37988
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-11-29 11:10:36 +08:00
Cai Yudong
Spade A
SimFG
wei liu
smellthemoon
Gao
Zhen Ye
Ted Xu
congqixia
Chun Han
zhagnlu
Bingyi Sun
cai.zhang
sre-ci-robot
foxspy
Patrick Weizhi Xu
MimoN
tinswzy
Xiaofan
Reid Chan
XuanYang-cn
sthuang
aoiasd
yihao.dai
Xianhui Lin
jaime
Buqian Zheng
cqy123456
shaoyue