enhance: Add schema update time verification for insert and upsert to
use cache
issue: https://github.com/milvus-io/milvus/issues/39093
---------
Signed-off-by: Xianhui.Lin <xianhui.lin@zilliz.com>
issue: #36621#39417
1. Adjust the server-side cache size.
2. Add source information for configurations.
3. Add node ID for compaction and indexing tasks.
4. Resolve localhost access issues to fix health check failures for
etcd.
Signed-off-by: jaime <yun.zhang@zilliz.com>
issue: #38399
- broadcast message can carry multi resource key now.
- implement event-based notification for broadcast messages
- broadcast message use broadcast id as a unique identifier in message
- broadcasted message on vchannels keep the broadcasted vchannel now.
- broadcasted message and broadcast message have a common broadcast
header now.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #38399
- Make the wal scanner interface same with streaming scanner.
- Use wal if the wal is located at current node.
- Otherwise fallback the old logic.
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #38399
- The stats may be kept after wal closing if the growing segment is not
dirty.
- Change the error handling of wal open to avoid redundant manager api
call.
Signed-off-by: chyezh <chyezh@outlook.com>
Related to #39596
When updating the build param configuration, the `Formatter` could be
used to do so and completed avoid touching the `overlay` config items
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
The dsl fields are separated in sub request structs and cannot be
easiliy printed before. This PR adds a log helper to print the dsl
expressions of HybridSearchRequest.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #38399
- Embed the query node into streaming node to make delegator available
at streaming node.
- The embedded query node has a special server label
`QUERYNODE_STREAMING-EMBEDDED`.
- Change the balance strategy to make the channel assigned to streaming
node as much as possible.
Signed-off-by: chyezh <chyezh@outlook.com>
issue: https://github.com/milvus-io/milvus/issues/37630
Reduce the frequency updating metrics to avoid holding the mutex for
long periods.
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
The error message was malformated or missing some meta info, say field
name. This PR recitfies some message format and add field name in error
message when type param check fails.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Previously the grep with regex does not work and failed to match lots of
.cpp files
This PR:
- use "-E" flag to use regex match
- commit the fixed result of current cpp code
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
When there are many segment tasks in the querycoord scheduler, the
traversal in `GetSegmentTaskDelta` checks becomes time-consuming. This
PR adds caching for segment deltas.
issue: https://github.com/milvus-io/milvus/issues/37630
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Read metadata such as segments, binlogs, and partitions concurrently at
the collection level.
issue: https://github.com/milvus-io/milvus/issues/37630
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
1. Limit the maximum number of restored segments to 1024.
2. Fail the import job if saving binlog fails.
3. Fail the import job if saving the import task fails to prevent
repeatedly generating dirty importing segments.
issue: https://github.com/milvus-io/milvus/issues/39331
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Related to #39339
Extra indexes can be ignored for most cases since sorted pk column
already provided indexing features
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Xianhui Lin
jaime
Zhen Ye
aoiasd
Ted Xu
Gao
cai.zhang
wei liu
yihao.dai
congqixia
sthuang
Cai Yudong
zhagnlu
junjiejiangjjj
smellthemoon
Spade A
SimFG
XuanYang-cn
Bingyi Sun
zhenshan.cao