## What does this PR do?
Fix a flaky test in `test_milvus_client_hybrid_search_v2.py`.
## Why?
`test_hybrid_search_as_search` asserts that hybrid search (single field
+ WeightedRanker) returns the same ordered result list as a regular
search. When two results have tied scores, tie-breaking order is
non-deterministic across architectures and runs due to floating-point
differences — this causes spurious failures on ARM e2e CI.
Comparing ID sets instead of ordered lists preserves the intent of the
test (same result candidates returned) while being robust to ordering of
ties.
---------
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
## Summary
- Migrate 15 partition key test cases from
`testcases/test_partition_key.py` (v1 ORM style) to
`milvus_client/test_milvus_client_partition_key.py` (v2 MilvusClient
style)
- Migrate 4 partition key isolation test cases from
`testcases/test_partition_key_isolation.py` to
`milvus_client/test_milvus_client_partition_key_isolation.py`
- Optimize isolation test parameters to prevent CI timeout: data_size
10000→1000, dim 768→128, HNSW M=30→16, efConstruction=360→64
- Remove migrated v1 test files
## Related Issues
#48048
## Test plan
- [x] All 19 migrated test cases (46 parametrized combinations) pass on
cluster (19531)
- [x] Partition key isolation tests (4 cases) all pass (~68s total, down
from >360s CI timeout)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: lyyyuna <yiyang.li@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
- Migrate 50+ partition test cases from `testcases/test_partition.py`
(v1 ORM style) to `milvus_client/test_milvus_client_partition.py` (v2
MilvusClient style)
- Add `gen_default_rows_data_for_upsert` helper in `common_func.py` for
row-based upsert data generation
- Enhance existing v2 tests with partition name validation and `None`
partition name coverage
- Remove migrated test cases from v1 file, keeping only v2-incompatible
cases (description params, dataframe insert)
## Related Issues
#48048
## Test plan
- [x] v1 remaining tests pass on standalone (19530) and cluster (19531)
- [x] v2 full test suite passes on standalone (148 passed, 3 failed -
replica tests need cluster)
- [x] v2 full test suite passes on cluster (151 passed, 0 failed)
- [x] v2 tests pass with 8-worker concurrency (`-n 8`) on both
environments
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: lyyyuna <yiyang.li@zilliz.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
Fixes#47316
- **GroupingSet::outputRowCount()**: Add null check for `lookup_` to
prevent SIGSEGV when filter matches zero rows and no input data arrives
- **ExecPlanNodeVisitor::setupRetrieveResult()**: When query result is
nullptr, build empty field_data arrays with correct schema (0 rows, N
columns) so downstream receives proper column structure instead of
missing columns
- Enable previously skipped E2E test `test_empty_result_aggregation`
## Test plan
- [x] E2E test `test_empty_result_aggregation` passes (GROUP BY +
aggregation with filter matching 0 rows returns empty result `[]`)
- [ ] Existing aggregation E2E tests still pass
- [ ] CI green
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
Add 4 test cases to `TestQueryAggregationSharedV2` covering GROUP BY on
nullable fields, verifying the fix for #47350 (PR #47445).
**Related issue**: #47350
### Changes
**Schema additions** to the shared collection:
- `c10_nullable_varchar`: VARCHAR (nullable, 7 unique values + ~15%
NULL)
- `c11_nullable_int16`: INT16 (nullable, 5 unique values + ~15% NULL)
**New tests:**
| Test | What it verifies |
|------|-----------------|
| `test_group_by_nullable_varchar_field` | GROUP BY nullable VARCHAR
returns actual values (not all NULL) + aggregation correctness |
| `test_group_by_nullable_int16_field` | GROUP BY nullable INT16 returns
actual values + aggregation correctness |
| `test_multi_column_group_by_with_nullable_field` | Multi-column GROUP
BY with one nullable column |
| `test_search_group_by_nullable_field` | Search with GROUP BY on
nullable field returns actual group values |
### Test Plan
- [x] All 4 new tests passed against Milvus 2.6.6
- [x] All 28 existing tests still pass (0 regressions)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
## Summary
- Enable search by ids tests that were commented out pending fix of
#47065
- Fix syntax error in test: `for i in len()` → `for i in range(len())`
- Update test expectations to match the fixed behavior:
- Search on null vectors now returns empty results instead of error
- Fix assertion to use correct `num_entities_with_not_null_vector` count
## Test plan
- [x] `test_milvus_client_add_nullable_vector_field_search` - verified
passing
- [x] `test_milvus_client_collection_null_vector_field_search` -
verified passing
issue: #47065🤖 Generated with [Claude Code](https://claude.ai/code)
---------
Signed-off-by: Yan Liang <yanliang@zilliz.com>
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
## Summary
- Add mock TEI server utility (`common/mock_tei_server.py`) for testing
text embedding functions
- Add test cases for PR #46984 fix (alter function when other function
is invalid)
## Test Cases
1. `test_alter_function_when_other_function_is_invalid` - Verify that
altering a valid function succeeds even when another function in the
collection is invalid
2. `test_alter_invalid_function_to_valid_endpoint` - Verify that users
can fix an invalid function by altering it to a valid endpoint
## Related Issues
- Issue: https://github.com/milvus-io/milvus/issues/46949
- Fix PR: https://github.com/milvus-io/milvus/pull/46984
## Test Plan
- [x] Verified test fails with unfixed Milvus
(`master-20260112-7bcd3b10`)
- [x] Verified test passes with fixed Milvus (`master-20260112-b39ecb0`)
---------
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
/kind improvement
/assign @yanliang567
**PR Summary**
Migrate advanced upsert valid test cases from ORM-based implementation.
- concurrent upserts on the same primary key
- multiple upsert operations
- dynamic field support
- default and None value handling
- sparse vector upsert cases
Signed-off-by: zilliz <jiaming.li@zilliz.com>
/kind improvement
/assign @yanliang567
**PR Summary**
Migrate core upsert valid test cases from ORM-based implementation.
- upsert with non-existing / existing primary keys
- upsert with auto_id enabled
- upsert with string primary key
- upsert with binary vector data
- upsert with data identical to inserted data
- upsert in specific or mismatched partitions
---------
Signed-off-by: zilliz <jiaming.li@zilliz.com>
/kind improvement
/assign @yanliang567
**PR Summary**
- Fix mismatched expectations in the insert resource-limit test.
- Migrate async insert test cases from ORM-based cases, including normal
insert, large insert, invalid data, and invalid partition scenarios.
---------
Signed-off-by: zilliz <jiaming.li@zilliz.com>
\kind improvement
\assign @yanliang567
- Skip duplicated partition insert tests:
- test_insert_default_partition (covered by
test_milvus_client_insert_partition)
- test_insert_partition_not_existed (covered by
test_milvus_client_insert_not_exist_partition_name)
- Fix the docstring of
test_milvus_client_insert_not_exist_partition_name to correctly describe
insert behavior with a non-existent partition.
- Strengthen assertions for partition insert tests:
- Verify total entity count after inserting into multiple partitions.
- Validate returned primary key IDs when inserting with explicit IDs.
- Fix mismatched test expectations in
`test_insert_auto_id_false_same_values` by aligning the docstring with
actual behavior.
- Add test_insert_all_datatype_collection to validate insert behavior
for collections containing all supported data types using row-based
client insert.
Signed-off-by: zilliz <jiaming.li@zilliz.com>
/kind improvement
/assign @yanliang567
**PR Summary**
- Skip duplicated client insert tests already covered by existing cases
(e.g. insert after client closed, missing vector field, auto-ID
scenarios).
- Fix test docstrings to accurately describe vector field missing and
vector data type mismatch behaviors.
- Migrate a client insert test (`test_insert_field_name_not_match`).
- Fix CaseLabel tagging to keep consistent with the original ORM-based
testcases.
Signed-off-by: zilliz <jiaming.li@zilliz.com>
/kind improvement
/assign @yanliang567
**PR Summary**
- migrate Binary vector insert with schema dimension mismatch.
- migrate insert with None data.
- add `test_milvus_client_insert_binary_default` to cover BINARY_VECTOR,
which was not included in `test_milvus_client_insert_default` (covered
FLOAT_VECTOR, FLOAT16_VECTOR, BFLOAT16_VECTOR, and INT8_VECTOR
Signed-off-by: zilliz <jiaming.li@zilliz.com>
issue: #42937
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
- Core invariant: callers must explicitly close output streams (call
Close()) instead of relying on RemoteOutputStream's destructor to
perform closure.
- Logic removed/simplified: RemoteOutputStream's destructor no longer
closes or asserts on the underlying arrow::io::OutputStream; an explicit
public Close() method was added and closure responsibility moved to that
code path.
- Why this is safe (no data loss/regression): callers now invoke Close()
before reading or destroying streams (e.g.,
DiskFileManagerTest::ReadAndWriteWithStream calls os->Close() before
opening the input stream). Write paths remain unchanged
(RemoteOutputStream::Write -> output_stream_->Write), and Close invokes
output_stream_->Close() with status assertion, ensuring
flush/confirmation via the same API and preserving data integrity;
removing destructor-side asserts prevents unexpected failures during
object destruction without changing write/close semantics.
- Chore: updated third-party pins — internal/core/thirdparty/knowhere
CMakeLists.txt: KNOWHERE_VERSION -> a59816e;
internal/core/thirdparty/milvus-common CMakeLists.txt:
MILVUS-COMMON-VERSION -> b6629f7.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
related: #36380
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
- Core invariant: aggregation is centralized and schema-aware — all
aggregate functions are created via the exec Aggregate registry
(milvus::exec::Aggregate) and validated by ValidateAggFieldType, use a
single in-memory accumulator layout (Accumulator/RowContainer) and
grouping primitives (GroupingSet, HashTable, VectorHasher), ensuring
consistent typing, null semantics and offsets across planner → exec →
reducer conversion paths (toAggregateInfo, Aggregate::create,
GroupingSet, AggResult converters).
- Removed / simplified logic: removed ad‑hoc count/group-by and reducer
code (CountNode/PhyCountNode, GroupByNode/PhyGroupByNode, cntReducer and
its tests) and consolidated into a unified AggregationNode →
PhyAggregationNode + GroupingSet + HashTable execution path and
centralized reducers (MilvusAggReducer, InternalAggReducer,
SegcoreAggReducer). AVG now implemented compositionally (SUM + COUNT)
rather than a bespoke operator, eliminating duplicate implementations.
- Why this does NOT cause data loss or regressions: existing data-access
and serialization paths are preserved and explicitly validated —
bulk_subscript / bulk_script_field_data and FieldData creation are used
for output materialization; converters (InternalResult2AggResult ↔
AggResult2internalResult, SegcoreResults2AggResult ↔
AggResult2segcoreResult) enforce shape/type/row-count validation; proxy
and plan-level checks (MatchAggregationExpression,
translateOutputFields, ValidateAggFieldType, translateGroupByFieldIds)
reject unsupported inputs (ARRAY/JSON, unsupported datatypes) early.
Empty-result generation and explicit error returns guard against silent
corruption.
- New capability and scope: end-to-end GROUP BY and aggregation support
added across the stack — proto (plan.proto, RetrieveRequest fields
group_by_field_ids/aggregates), planner nodes (AggregationNode,
ProjectNode, SearchGroupByNode), exec operators (PhyAggregationNode,
PhyProjectNode) and aggregation core (Aggregate implementations:
Sum/Count/Min/Max, SimpleNumericAggregate, RowContainer, GroupingSet,
HashTable) plus proxy/querynode reducers and tests — enabling grouped
and global aggregation (sum, count, min, max, avg via sum+count) with
schema-aware validation and reduction.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
/kind improvement
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
- Core invariant: index parameter validation and test expectations for
the HNSW-family must be explicit, consistent, and deterministic — this
PR enforces that by adding exhaustive parameter matrices for HNSW_PRQ
(tests/python_client/testcases/indexes/{idx_hnsw_prq.py,
test_hnsw_prq.py}) and normalizing expectations in idx_hnsw_pq.py via a
shared success variable.
- Logic removed / simplified: brittle, ad-hoc string expectations were
consolidated — literal "success" occurrences were replaced with a single
success variable and ambiguous short error messages were replaced by the
canonical descriptive error text; this reduces duplicated assertion
logic in tests and removes dependence on fragile, truncated messages.
- Bug fix (tests): corrected HNSW_PQ test expectations to assert the
full, authoritative error for invalid PQ m ("The dimension of the vector
(dim) should be a multiple of the number of subquantizers (m).") and
aligned HNSW_PRQ test matrices (idx_hnsw_prq.py) to the same explicit
expectations — the change targets test assertions only and fixes false
negatives caused by mismatched messages.
- No data loss or behavior regression: only test code is added/modified
(tests/python_client/testcases/indexes/*). Production code paths remain
unmodified — collection creation, insert/flush, client.create_index,
wait_for_index_ready, load_collection, search, and client.describe_index
are invoked by tests but not changed; therefore persisted data, index
artifacts, and runtime behavior are unaffected.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: zilliz <jiaming.li@zilliz.com>
related issue: #46616
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
- Core invariant: these tests assume the v2 group-by search
implementation (TestMilvusClientV2Base and pymilvus v2 APIs such as
AnnSearchRequest/WeightedRanker) is functionally correct; the PR extends
coverage to validate group-by semantics when using JSON fields and
dynamic fields (see
tests/python_client/milvus_client_v2/test_milvus_client_search_group_by.py
— TestGroupSearch.setup_class and parametrized group_by_field cases).
- Logic removed/simplified: legacy v1 test scaffolding and duplicated
parametrized fixtures/test permutations were consolidated into
v2-focused suites (TestGroupSearch now inherits TestMilvusClientV2Base;
old TestGroupSearch/TestcaseBase patterns and large blocks in
test_mix_scenes were removed) to avoid redundant fixture permutations
and duplicate assertions while reusing shared helpers in common_func
(e.g., gen_scalar_field, gen_row_data_by_schema) and common_type
constants.
- Why this does NOT introduce data loss or behavior regression: only
test code, test helpers, and test imports were changed — no
production/server code altered. Test helper changes are
backward-compatible (gen_scalar_field forces primary key nullable=False
and only affects test data generation paths in
tests/python_client/common/common_func.py; get_field_dtype_by_field_name
now accepts schema dicts/ORM schemas and is used only by tests to choose
vector generation) and collection creation/insertion in tests use the
same CollectionSchema/FieldSchema paths, so production
storage/serialization logic is untouched.
- New capability (test addition): adds v2 test coverage for group-by
search over JSON and dynamic fields plus related scenarios — pagination,
strict/non-strict group_size, min/max group constraints, multi-field
group-bys and binary vector cases — implemented in
tests/python_client/milvus_client_v2/test_milvus_client_search_group_by.py
to address issue #46616.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
- Updates e2e test error code and message after pymilvus update
see #46677
Signed-off-by: silas.jiang <silas.jiang@zilliz.com>
Co-authored-by: silas.jiang <silas.jiang@zilliz.com>
/kind improvement
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
- Core invariant: test infrastructure treats insertion granularity as
orthogonal to data semantics—bulk generation
gen_row_data_by_schema(nb=2000, start=0, random_pk=False) yields the
same sequential PKs and vector payloads as prior multi-batch inserts, so
tests relying on collection lifecycle, flush, index build, load and
search behave identically.
- What changed / simplified: added a full HNSW_PQ parameterized test
suite (tests/python_client/testcases/indexes/idx_hnsw_pq.py and
test_hnsw_pq.py) and simplified HNSW_SQ test insertion by replacing
looped per-batch generation+insert with a single bulk
gen_row_data_by_schema(...) + insert. The per-batch PK sequencing and
repeated vector generation were redundant for correctness and were
removed to reduce complexity.
- Why this does NOT cause data loss or behavior regression: the
post-insert code paths remain unchanged—tests still call client.flush(),
create_index(...), util.wait_for_index_ready(), collection.load(), and
perform searches that assert describe_index and search outputs. Because
start=0 and random_pk=False reproduce identical sequential PKs (0..1999)
and the same vectors, index creation and search validation operate on
identical data and index parameters, preserving previous assertions and
outcomes.
- New capability: comprehensive HNSW_PQ coverage (build params: M,
efConstruction, m, nbits, refine, refine_type; search params: ef,
refine_k) across vector types (FLOAT_VECTOR, FLOAT16_VECTOR,
BFLOAT16_VECTOR, INT8_VECTOR) and metrics (L2, IP, COSINE), implemented
as data-driven tests to validate success and failure/error messages for
boundary, type-mismatch and inter-parameter constraints.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: zilliz <jiaming.li@zilliz.com>
issue: https://github.com/milvus-io/milvus/issues/45525
see added README.md for added optimizations
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added query expression optimization feature with a new `optimizeExpr`
configuration flag to enable automatic simplification of filter
predicates, including range predicate optimization, merging of IN/NOT IN
conditions, and flattening of nested logical operators.
* **Bug Fixes**
* Adjusted delete operation behavior to correctly handle expression
evaluation.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
Issue #46015
<test>: <add timestamptz to more bulk writer>
On branch feature/timestamps
Changes to be committed:
modified: testcases/test_bulk_insert.py
Signed-off-by: Eric Hou <eric.hou@zilliz.com>
Co-authored-by: Eric Hou <eric.hou@zilliz.com>
Issue: #45756
1. add bulk insert scenario
2. fix small issue in e2e cases
3. add search group by test case
4. add timestampstz to gen_all_datatype_collection_schema
5. modify partial update testcase to ensure correct result from
timestamptz field
On branch feature/timestamps
Changes to be committed:
modified: common/bulk_insert_data.py
modified: common/common_func.py
modified: common/common_type.py
modified: milvus_client/test_milvus_client_partial_update.py
modified: milvus_client/test_milvus_client_timestamptz.py
modified: pytest.ini
modified: testcases/test_bulk_insert.py
Signed-off-by: Eric Hou <eric.hou@zilliz.com>
Co-authored-by: Eric Hou <eric.hou@zilliz.com>
issue: #44320
This change adds deduplication logic to handle duplicate primary keys
within a single upsert batch, keeping the last occurrence of each
primary key.
Key changes:
- Add DeduplicateFieldData function to remove duplicate PKs from field
data, supporting both Int64 and VarChar primary keys
- Refactor fillFieldPropertiesBySchema into two separate functions:
validateFieldDataColumns for validation and fillFieldPropertiesOnly for
property filling, improving code clarity and reusability
- Integrate deduplication logic in upsertTask.PreExecute to
automatically deduplicate data before processing
- Add comprehensive unit tests for deduplication with various PK types
(Int64, VarChar) and field types (scalar, vector)
- Add Python integration tests to verify end-to-end behavior
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #45403, #45463
- fix the Nightly E2E failures.
- fix the wrong update timetick of altering collection to fix the
related load failure.
Signed-off-by: chyezh <chyezh@outlook.com>
issue: #43897
- Alter collection/database is implemented by WAL-based DDL framework
now.
- Support AlterCollection/AlterDatabase in wal now.
- Alter operation can be synced by new CDC now.
- Refactor some UT for alter DDL.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
Xiaofan
Li Yiyang
Li Yiyang
Chun Han
yanliang567
zhuwenxing
pymilvus-bot
jiamingli-maker
Feilong Hou
Li Liu
foxspy
jac
Buqian Zheng
nico
wei liu
Spade A
Zhen Ye