7859faf8ea
fix: Change deltalog memory estimation factor to one ( #36033 )
...
See also: #36031
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-09-06 18:21:05 +08:00
a103dd5eb3
enhance: Fix SearchOnSealed clang-format lint ( #36056 )
...
Related to #36008
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-06 16:47:04 +08:00
21b135c7c2
fix: not append valid data when transfer to insert record ( #36027 )
...
fix not append valid data when transfer to insert record and add a tiny
check when in groupBy field.
#35924
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-06 14:53:04 +08:00
5247631289
fix: fill the metric type field in the LoadMetaInfo object ( #35962 )
...
- issue: #35960
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-09-05 20:50:23 -07:00
9871841fa0
fix: datacoord broker use querycoord broker config and get segment info timeout ( #36026 )
...
relate: https://github.com/milvus-io/milvus/issues/36025
Will case channel with large segment num watch failed.
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-09-06 08:15:05 +08:00
11325d9ed5
fix: binary arith expression on inverted index ( #35945 )
...
issue: https://github.com/milvus-io/milvus/issues/35946
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-09-05 20:01:05 +08:00
75676fbd11
fix: Fix dynamic release partition may fail search/query request ( #35919 )
...
issue: #33550
cause concurrent issue may occur between remove parition in target
manager and sync segment list to delegator. when it happens, some
segment may be released in delegator, and those segment may also be
synced to delegator, which cause delegator become unserviceable due to
lack of necessary segments, then search/query fails.
this PR make sure that all write access to target_manager will be
executed in serial to avoid the concurrent issues.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-05 18:47:03 +08:00
560e8e70b0
enhance: reduce mmap_rss after chunkcache warmup ( #35974 )
...
related pr: https://github.com/milvus-io/milvus/pull/35965
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-09-05 18:07:05 +08:00
8b043f58dc
fix: Fix DB limiter nodes are mistakenly cleaned up ( #35991 )
...
This issue only occurs for a short time right after a table is created.
To avoid this, we simply reduce the frequency of cleaning up invalid
limiter nodes.
issue: https://github.com/milvus-io/milvus/issues/35933
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-05 17:35:04 +08:00
f985173da0
fix: Fill load field list from old version load info ( #35993 )
...
See also #35959
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-05 16:57:05 +08:00
c61eea737b
enhance: Fix trace.cpp lint format issue ( #36004 )
...
Introduced by #35928
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-05 16:33:04 +08:00
32e55a02ea
fix: Fix privilege group hasn't been register for validate ( #35937 )
...
issue: #35471
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-05 15:35:04 +08:00
7b21032d19
fix: Check all values for `trie.predictive_search` ( #35943 )
...
Related to #35941
For marisa trie `predictive_search` default behavior, it value iterated
is not in lexicographic order.
This PR is a brute force fix to make range operator returns correct
values.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-05 15:01:04 +08:00
9e96ed4873
fix: Fix tracing config update logic ( #35928 )
...
Related to #35927
There are serveral issue this PR addresses:
- Use `ResetTraceConfig` method instead init one in update event handler
- Implement dynamic stats.Handler to receive tracing config update event
- Update `enable_trace` flag when `ResetTraceConfig` is invoked
- Change `enable_trace` to `std::atomic<bool>` in case of data race
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-05 14:27:04 +08:00
5e3f700e5d
enhance: Remove too frequent logs in Delete ( #35980 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-09-05 14:25:03 +08:00
8593c4580a
enhance: Add delete buffer related quota logic ( #35918 )
...
See also #35303
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-05 11:39:03 +08:00
80a9efd96d
fix: not check nullable and default value in pk field ( #35987 )
...
#35926
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-05 11:27:04 +08:00
66ed289a85
enhance: Fix typo of clustering key not loaded msg ( #35948 )
...
Related to #35415
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-05 10:49:03 +08:00
90bdb171ab
fix: Fix data race for clustering compaction writer ( #35957 )
...
issue: #35950
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-05 04:07:10 +08:00
6eb8b3f745
fix: err degenerated to a new variable ( #35891 )
...
https://github.com/milvus-io/milvus/issues/35890
Signed-off-by: fengjun2016 <jornfeng@gmail.com>
2024-09-04 14:57:04 +08:00
24fb10114b
enhance: remove cooling off in rate limiter for read requests ( #35935 )
...
issue: #35934
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-04 14:39:10 +08:00
ea36d13ab0
feat: add static view for the expr ( #35887 )
...
- issue: #35886
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-09-04 11:09:04 +08:00
cc02dc0a55
fix: Handle Input/Output Errors in vsnprintf and snprintf ( #35898 )
...
Fix for Issue: #35897
2024-09-04 08:15:04 +08:00
9da86529a7
enhance: Add disk filemananger parallel load control to reduce the memory consumption ( #35281 )
...
issue: #35280
add parallel control to limit the memory consumption during index file
loading
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-09-03 18:01:03 +08:00
f68df9a11e
fix: SkipIndex cause segment fault ( #35907 )
...
issue: #35882
Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-03 17:15:03 +08:00
74048ce34f
fix:rename mmap file path to avoid directory conflict ( #35810 )
...
#35784
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-03 16:05:03 +08:00
c84ea5465c
fix: Fix some replicas don't participate in the query after the failure recovery ( #35850 )
...
issue: #35846
querycoord will notify proxy to update shard leader cache after
delegator location changes, but during querynode's failure recovery,
some delegator may become unserviceable due to lacking of segments, and
back to serviceable after segment loaded, so we also need to notify
proxy to invalidate shard leader cache when delegator serviceable state
changes.
This PR will maintain querynode's serviceable state during heartbeat,
and notify proxy to invalidate shard leader cache if serviceable state
changes.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-03 15:39:03 +08:00
6130a85444
enhance: Remove bf from streaming node ( #35902 )
...
Remove bf from streaming node:
1. When watching vchannels, skip loading bloom filters for segments.
2. Bypass bloom filter checks for delete messages, directly writing to
L0 segments.
3. Remove flushed segments proactively after flush.
issue: https://github.com/milvus-io/milvus/issues/33285 ,
https://github.com/milvus-io/milvus/issues/34585
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-03 14:17:02 +08:00
325f1987d9
enhance: rewrite index params for compatibility ( #35788 )
...
#32900
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-02 19:41:03 +08:00
3698c53a72
enhance: Check load fields for previous loaded collection ( #35905 )
...
Related to #35415
This PR make querycoord report error when load request tries to update
load fields list, which is currently not supported.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-02 18:39:03 +08:00
4641fd9195
enhance: make search groupby stop when reaching topk groups ( #35814 )
...
related: #33544
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-02 18:25:03 +08:00
b2eb9fe2a7
fix: memory leak in unittest and open the USE_ASAN option when build unittest ( #35855 )
...
issue: #35854
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-02 15:59:04 +08:00
6fd33285e1
fix: Fix compile error ( #35901 )
...
/kind improvement
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-02 14:50:35 +08:00
2c9bb4dfa3
feat: Support stats task to sort segment by PK ( #35054 )
...
issue: #33744
This PR includes the following changes:
1. Added a new task type to the task scheduler in datacoord: stats task,
which sorts segments by primary key.
2. Implemented segment sorting in indexnode.
3. Added a new field `FieldStatsLog` to SegmentInfo to store token index
information.
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-02 14:19:03 +08:00
9d80137698
fix: Check clustering key skip load behavior ( #35865 )
...
feature issue: #35415
See also #35861
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-02 11:17:02 +08:00
576ac2bbed
fix: Fix the reference to a variable after it has been moved ( #35875 )
...
#35607
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-02 10:05:02 +08:00
5ea2454fdf
feat: tantivy tokenizer binding ( #35801 )
...
fix : #35800
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-09-01 17:13:03 +08:00
1413ffe9b1
enhance: Rename preAllocatedSegments ( #35871 )
...
Rename `preAllocatedSegments` to `preAllocatedSegmentIDs` to avoid
confusion.
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-01 17:09:01 +08:00
2e090b2426
enhance: Refine flusher channel management ( #35870 )
...
Change the ChannelTask to ChannelLifetime, only removing it upon
unregistering.
issue: https://github.com/milvus-io/milvus/issues/33285
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-01 17:07:10 +08:00
407fc933e7
fix: bump bytedance/sonic to v1.12.2 to fix compilation error with go 1.23.0 ( #35879 )
...
fixes : #35878
Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>
2024-09-01 13:43:01 +08:00
311f860676
enhance: support to drop the role which is related the privilege list ( #35727 )
...
- issue: #35545
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-30 15:17:00 +08:00
323400c190
enhance: Enable to write multiple segments in mix compactor ( #35705 )
...
Prevent segments to be written larger than maxSize * expansionRate
See also: #35584
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-30 11:29:01 +08:00
1c3abd1aea
fix: Fix unstable TestFlusher ut ( #35831 )
...
issue: https://github.com/milvus-io/milvus/issues/35830
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-30 10:25:01 +08:00
f9be01ce89
fix: mockery config ( #35834 )
...
issue: #33285
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-30 09:49:00 +08:00
671112d17b
enhance: add more info to hybrid index log ( #35808 )
...
#32900
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-29 21:07:04 +08:00
a3f2f044d6
fix: not set nullable when stream writer write headers ( #35799 )
...
#35802
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-29 20:59:00 +08:00
3c8941a88e
enhance: Print log only when rate limit updates ( #35806 )
...
The debug log for "RateLimiter register for rateType" is too frequent
and in e2e cases, the may print 18M times in one run.
This PR make the log be printed only when rate limit is updated.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 15:07:00 +08:00
09ef3f1b4f
fix: Make sure querycoord observers started once ( #35811 )
...
Related to #35809
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 14:45:00 +08:00
b3089b5bdc
feat: support range search pagination retains order ( #35738 )
...
issue: #35464
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-08-29 14:09:00 +08:00
b0ac04d104
enhance: set database properties to restrict read access ( #35745 )
...
issue: #35744
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-29 13:17:01 +08:00
b51b4a2838
fix: try get not exist file after upgrade ( #35740 )
...
https://github.com/milvus-io/milvus/issues/35741
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-29 11:09:01 +08:00
86691656f3
enhance: Change frequent balancer debug log to rated one ( #35749 )
...
"skip balance" log is too frequent in debug level. This PR changes it
into rated on.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 10:07:00 +08:00
985d84d3ee
enhance: Add skip load validation for create collection task ( #35737 )
...
Related to #35415
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 10:05:08 +08:00
99dff06391
enhance: using streaming service in insert/upsert/flush/delete/querynode ( #35406 )
...
issue: #33285
- using streaming service in insert/upsert/flush/delete/querynode
- fixup flusher bugs and refactor the flush operation
- enable streaming service for dml and ddl
- pass the e2e when enabling streaming service
- pass the integration tst when enabling streaming service
---------
Signed-off-by: chyezh <chyezh@outlook.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-29 10:03:08 +08:00
c5c9ef9876
fix: Check response size before add to counter ( #35778 )
...
Related to #35767
prometheus counter cannot add negative value
when response is not written(say timeout/network broken) panicking may
happen if not check
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 00:05:00 +08:00
9b96841ae9
fix: wrong construction in evalctx ( #35772 )
...
issue: #35771
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-28 19:21:00 +08:00
91223deb91
fix: Use `SafeChan` as TxnManager.closed ( #35763 )
...
Resolves : #35762
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-28 19:06:59 +08:00
56b3490a98
fix: Fix rate wasn't limited to the expected value ( #35699 )
...
Each time the rate is reset, the token bucket is fully refilled, causing
the rate wasn't limited to the expected value. This PR addresses the
issue by preventing the token reset.
issue: https://github.com/milvus-io/milvus/issues/35675 ,
https://github.com/milvus-io/milvus/issues/35702
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-28 14:13:00 +08:00
a52ba3d09d
enhance: allow many segments for inverted index ( #35616 )
...
fix: https://github.com/milvus-io/milvus/issues/35615
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-08-28 11:30:59 +08:00
d8aa01bc1a
fix: inconsistent meta view causes rate limit invalid ( #35665 )
...
issue: #35663
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-28 11:21:04 +08:00
b7ea1defd3
fix: mistaken deletions may occur during GC channel checkpoints ( #35707 )
...
issue: #35706
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-28 10:11:05 +08:00
98866205fa
fix: munmap deallocate too much memory ( #35725 )
...
issue: #35693
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-27 17:18:59 +08:00
9868fe4e6c
fix: Fix panic due to empty candidate import segments ( #35673 )
...
issue: https://github.com/milvus-io/milvus/issues/35662
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-27 17:08:59 +08:00
4d2f96c760
enhance: support bitmap mmap ( #35399 )
...
#32900
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-27 16:34:59 +08:00
bfd9d86fe9
feat: support groupby size on go-layer( #33544 ) ( #33845 )
...
related: #33544
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-27 14:21:00 +08:00
fe83805d56
fix: loss data bug for deprecated querynode DoubleBuffer ( #35128 )
...
relate: https://github.com/milvus-io/milvus/issues/31548
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-08-27 14:10:59 +08:00
f12e368a76
fix: Fill nil schema so that Milvus can watch channel for those upgraded from 2.2 to 2.4 #35695 ( #35694 )
...
See also: [#35701 ](https://github.com/milvus-io/milvus/issues/35701 )
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-27 10:36:59 +08:00
3e1052f889
enhance: use the msg position obj when getting replicate channel position ( #35606 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-27 10:28:59 +08:00
7d3c0d748c
fix: the incorrect number of partitions in rootcoord meta ( #35600 )
...
issue: #35698
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-26 15:20:58 +08:00
82743c5c50
fix: Clear channelcp meta and metrics ASAP ( #35658 )
...
See also: #35588
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-26 12:22:57 +08:00
6ddfd02f01
[automated] Update Knowhere Commit ( #35688 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-26 01:04:57 +08:00
4236c3bafb
enhance: enable MV on binary vectors ( #35668 )
...
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-08-25 16:27:09 +08:00
098c10922b
enhance: add some check when create collection ( #35629 )
...
not support nullable==true in vector type. check it when create
collection.
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-25 16:23:05 +08:00
0dc5e89007
enhance: reduce the log level of frequent log ( #35652 )
...
fix #35651
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-08-25 16:20:57 +08:00
615a653988
fix: Fix offset out of range for creating Trie index ( #35553 )
...
issue: #35550
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-25 15:50:57 +08:00
f2b83d316b
enhance: Support memory mode chunk cache ( #35347 )
...
Chunk cache supports loading raw vectors into memory.
issue: https://github.com/milvus-io/milvus/issues/35273
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-25 15:42:58 +08:00
50fcfe8ef1
enhance: add nan and inf check ( #35683 )
...
fix #35594
add float check on files
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-08-25 15:22:57 +08:00
42f7800b5b
enhance: add bitmap offset cache to speed up retrieve raw data ( #35498 )
...
#35458
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-24 01:40:58 +08:00
75da36d1aa
enhance: enable asan for milvus ( #35627 )
...
issue: #35626
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 21:06:58 +08:00
59387f0b3a
fix: Use correct release func when datanode unwatch a channel ( #35655 )
...
See also #35654
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-23 14:09:03 +08:00
e8e3544a11
enhance: add hit segment num metrics for queryHook ( #35577 )
...
issue: #35576
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-08-23 12:49:02 +08:00
a773836b89
enhance: optimize milvus core building ( #35610 )
...
issue: #35549,#35611,#35633
- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 12:35:02 +08:00
ab532ae199
enhance: Add back BF lazy load logic for datanode watch channel ( #35646 )
...
Add back lazy loading statslog when watch dml channel on datanode.
Related to #22994 #27675
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 19:42:57 +08:00
5c245d51c4
enhance: Refresh proxy cache after restore rbac meta ( #35635 )
...
issue: #35443
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-22 19:09:01 +08:00
9fff07936a
fix: Re-read value after `once` initialization ( #35642 )
...
Related to #35641
See also #35271
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 17:24:58 +08:00
582d2eec79
enhance: Move datanode/indexnode manager to session pkg ( #35634 )
...
Related to #28861
Move session manager, worker manager to session package. Also renaming
each manager to corresponding node name(datanode, indexnode).
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 16:02:56 +08:00
3107701fe8
enhance: optimize retrieve on dynamic field ( #35580 )
...
#35514
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-08-22 14:24:56 +08:00
c992a61a23
enhance: Separate allocator pkg in datacoord ( #35622 )
...
Related to #28861
Move allocator interface and implementation into separate package. Also
update some unittest logic.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 10:06:56 +08:00
024eccbde0
enhance: add pkg-config for knowhere ( #35433 )
...
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-08-22 09:56:56 +08:00
ed4eaffc9d
enhance: add csv support for bulkinsert ( #34938 )
...
See this issue for details: #34937
---------
Signed-off-by: OxalisCu <2127298698@qq.com>
2024-08-21 17:47:01 +08:00
731d45abbe
enhance: provide more general configuration to control mmap behavior ( #35359 )
...
- issue: #35273
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-21 00:22:54 +08:00
41646c8439
feat: integrate new deltalog format ( #35522 )
...
See #34123
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-08-20 19:06:56 +08:00
bc2bbdc82f
enhance: Forbid access log print error message with line break ( #34491 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-08-20 17:36:55 +08:00
80a7c78f28
enhance: import supports null in parquet and json formats ( #35558 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-20 16:50:55 +08:00
2fbc628994
feat: Support field partial load collection ( #35416 )
...
Related to #35415
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-20 16:49:02 +08:00
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2( #35350 ) ( #35470 )
...
related: #35350
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-20 16:16:55 +08:00
3491608256
fix: Match int8_t and int16_t in Array::get_data ( #35579 )
...
Related to #35578
Previously int16/int8 bitmap index may read int32 array as int16, which
may cause build index with half of the data(if array is full) and half
zeros. This causes BITMAP index lost information.
This PR matches int8_t & int16_t while `get_data` when building index.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-20 16:10:56 +08:00
e09dc3be58
enhance: Mark query node as read only after suspend ( #35492 )
...
issue: #34985 #35493
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 14:02:54 +08:00
3948bd4e79
fix: Added check for validating varchar,array max length ( #35499 )
...
issue : https://github.com/milvus-io/milvus/issues/34150
This is for numpy,parquet,json readers.
---------
Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com>
2024-08-20 11:42:55 +08:00
22ced010cd
enhance: make configure load param feature be compatible with old sdk ( #35520 )
...
issue: #31570 #35521
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 10:30:55 +08:00
XuanYang-cn
congqixia
smellthemoon
SimFG
aoiasd
Jiquan Long
wei liu
cqy123456
yihao.dai
cai.zhang
CharlesFeng
jaime
Abdullah Ahmed
foxspy
Zhen Ye
zhagnlu
Chun Han
Yinzuo Jiang
Patrick Weizhi Xu
sre-ci-robot
Xiaofan
Gao
presburger
OxalisCu
Ted Xu
nish112022