github.com/gogo/protobuf is deprecated and could be error prune after
upgrade protobuf message to v2.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
related: #33544
mainly changes in three aspects:
1. enable setting group_size for group by function
2. separate normal reduce and group by reduce
3. eleminate uncessary padding in search result for reducing
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
issue: # #34545
Print warn log instead of check health fail if orphan channel cp meta is
found in health check request.
Signed-off-by: jaime <yun.zhang@zilliz.com>
When the main dispatcher has not yet consumed data, curTs is 0. During
this time, merging dispatchers should not be allowed; otherwise, the
data of the solo dispatcher will be skipped.
issue: https://github.com/milvus-io/milvus/issues/34255
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Related to #34508
The padding bytes shall be written only at the end of the mmap file not
the chunk of each field data file.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #34500
cause the sort in `GetLevel0Deletions` will broken the corresponed order
between pks and tss, then the pks and tss will be sorted in
segment.Delete() interface.
This PR remove this uncessary and incorrect sort progress to avoid query
may return deleted records.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
See also #34483
Some lint issues are introduced due to lack of static check run. This PR
fixes these problems.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #33285
- implement producing and consuming server of message
- implement management operation for streaming node server
---------
Signed-off-by: chyezh <chyezh@outlook.com>
See also #33785
When config item is not present in paramtable, CAS fails due to
GetConfig returns error.
This PR make this returned err instance of ErrKeyNotFound and check
error type in \`CASCachedValue\` methods.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
related: #30376
fix: paritionIDs lost when no setting paritions
enhance: refine metrics for segment prune
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
issue: #34304
cosine is more widely used in float vectors, and cosine and hamming
distance are 'metrics' which have good geometric properties
Signed-off-by: chasingegg <chao.gao@zilliz.com>
The import is dependent on syncTask, which in turn relies on the
allocator. This PR pre-allocate the necessary IDs for import syncTask.
issue: https://github.com/milvus-io/milvus/issues/33957
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
issue: #34123
Benchmark case: The benchmark run the go benchmark function
`BenchmarkDeltalogFormat` which is put in the Files changed. It tests
the performance of serializing and deserializing from two different data
formats under a 10 million delete log dataset.
Metrics: The benchmarks measure the average time taken per operation
(ns/op), memory allocated per operation (MB/op), and the number of
memory allocations per operation (allocs/op).
| Test Name | Avg Time (ns/op) | Time Comparison | Memory Allocation
(MB/op) | Memory Comparison | Allocation Count (allocs/op) | Allocation
Comparison |
|---------------------------------|------------------|-----------------|---------------------------|-------------------|------------------------------|------------------------|
| one_string_format_reader | 2,781,990,000 | Baseline | 2,422 | Baseline
| 20,336,539 | Baseline |
| pk_ts_separate_format_reader | 480,682,639 | -82.72% | 1,765 | -27.14%
| 20,396,958 | +0.30% |
| one_string_format_writer | 5,483,436,041 | Baseline | 13,900 |
Baseline | 70,057,473 | Baseline |
| pk_and_ts_separate_format_writer| 798,591,584 | -85.43% | 2,178 |
-84.34% | 30,270,488 | -56.78% |
Both read and write operations show significant improvements in both
speed and memory allocation.
Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
Check if the segment exists during FlushSegments and add some key logs
in write path.
issue: https://github.com/milvus-io/milvus/issues/34255
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Added retry method and unit test cases for retrying etcd server start.
New to open source and Go language. Please reject if this is not the
requirement/ specify the modifications needed in code.
issue : #17569
Signed-off-by: Charles Kakumanu <charles_kakumanu@apple.com>
Co-authored-by: Charles Kakumanu <charles_kakumanu@apple.com>
SimFG
congqixia
Chun Han
aoiasd
wayblink
jaime
XuanYang-cn
Patrick Weizhi Xu
smellthemoon
ThreadDao
chyezh
yihao.dai
wei liu
sre-ci-robot
sre-ci-robot
Ted Xu
Gao
Aldrin
elstic
shaoting-huang
charleskakumanu