Commit Graph

75 Commits (17532517c611cafe5ec7a79bda47c9f296e82682)

Author SHA1 Message Date
yanliang567 17532517c6
test: Fix sparse/group_by/text_match/diskann/invalid/by_pk (#48347)
## Summary
- GPU→L1, zip() for positional index, add metric_type, fix spelling
- IVF_SQ8→DISKANN, skip DISKANN mmap, fix INT8 metric, filter assertions
- Fix false-pass bypass in check_search_results for empty hits

## Code Review Fix (0e4ad86c13)

### check_search_results: remove empty-hits bypass

`check_search_results` in `func_check.py` had a `if len(hits) == 0:
continue` guard that silently skipped all verification (including limit
check) when search returned no results. This caused tests with filters
matching zero rows to false-pass instead of failing.

Removed the 2-line guard entirely. The existing `assert len(hits) ==
check_items["limit"]` already handles this correctly — when limit is
specified and hits is empty, the assertion properly fails.

**Impact:** `test_search_with_scalar_field` in
`test_milvus_client_search_diskann.py` filters `int64 in [1, 2, 3, 4]`
on randomly-generated INT64 data (non-PK field uses full-range random
values). This matches 0 rows, and previously false-passed due to the
bypass. After the fix, it correctly fails. The filter in the test should
also be updated to use values that exist in the data.

Verified: ran all 6 PR test files (L0+L1, `-n 4`) — 88 passed, 1
expected failure (`test_search_with_scalar_field`), 9 skipped.

issue: #48048
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 10:51:30 +08:00
yanliang567 61e929c2cf
test: Fix json/string/iterator/pagination verification (#48346)
## Summary
- Add check_task/filter assertions, fix hardcoded COSINE, add pk_range
to func_check
- Reduce L0 iterator combos, fix bare field names, increase
overlap_ratio

issue: #48048
🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 10:47:29 +08:00
yanliang567 4bbc01324b
fix: handle NaN vs None comparison in output field value check for nullable fields (#47841)
## Summary

- Fix `output_field_value_check` in `param_check.py` to handle NaN vs
None comparison for nullable fields
- When nullable fields contain `None` values, `pandas.DataFrame`
converts them to `NaN`. The check function then fails comparing `NaN`
(from DataFrame) with `None` (from Milvus search results). This fix
treats both as equivalent null values.
- Remove resolved issue reference comments (`#47065`)

**Related issue**: #47065

### Changes

**Modified Files:**
- `tests/python_client/check/param_check.py`: Add NaN/None equivalence
check before asserting output field values
- `tests/python_client/milvus_client/test_add_field_feature.py`: Remove
resolved `#47065` comment
- `tests/python_client/milvus_client/test_milvus_client_collection.py`:
Remove resolved `#47065` comment

### Test Plan

- [x]
`test_search_by_pk_with_output_fields_and_consistency_level[Strong/Session/Bounded/Eventually]`
- 4 previously failing tests now PASS
- [x] All 98 search_by_pk tests: 93 passed + 4 fixed + 1 skipped
- [x] 13 null/nullable vector related tests all PASS

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: yanliang567 <82361606+yanliang567@users.noreply.github.com>
2026-02-25 16:14:46 +08:00
yanliang567 9ef7e25778
test: enable search by ids on null vectors tests for #47065 (#47559)
## Summary
- Enable search by ids tests that were commented out pending fix of
#47065
- Fix syntax error in test: `for i in len()` → `for i in range(len())`
- Update test expectations to match the fixed behavior:
  - Search on null vectors now returns empty results instead of error
- Fix assertion to use correct `num_entities_with_not_null_vector` count

## Test plan
- [x] `test_milvus_client_add_nullable_vector_field_search` - verified
passing
- [x] `test_milvus_client_collection_null_vector_field_search` -
verified passing

issue: #47065

🤖 Generated with [Claude Code](https://claude.ai/code)

---------

Signed-off-by: Yan Liang <yanliang@zilliz.com>
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 14:17:49 +08:00
zhuwenxing a80e003731
test: add snapshot API test cases (#47309)
## Summary
- Add comprehensive test cases for Milvus snapshot API functionality
- Cover snapshot create, list, describe, drop, restore operations
- Test various data types, index types, partition scenarios
- Add concurrent operation and data integrity tests
- Update pymilvus to 2.7.0rc122 for snapshot API support

## Test Coverage
- L0 smoke tests: basic snapshot lifecycle
- L1 tests: data types, partitions, indexes
- L2 tests: boundary conditions, negative cases, concurrency

## Related Issue
https://github.com/milvus-io/milvus/issues/44358

## Test plan
- [x] Run tests with `-n 6` parallel execution
- [x] Verify all tests pass (67 tests: 62 passed, 4 xfail)

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2026-01-28 19:41:33 +08:00
yanliang567 bdb56f6053
enhance: Optimize Python test logging from 60MB to 3-5MB per run (#47253)
## Summary

This PR optimizes Python test logging to reduce log file size from 60MB+
to 3-5MB per test run (90%+ reduction) while preserving full debugging
information for failed tests.

related issue:  #47256

### Key Improvements

- **Conditional Logging**: PASSED tests save only metadata, FAILED tests
preserve complete logs
- **Memory Buffering**: Eliminates runtime disk I/O (no file writes
during test execution)
- **Unified Reports**: Generates 2 formats
  - `test_report.json`: AI-friendly structured data
  - `test_report.html`: Human-readable with color-coded logs
- **Worker Parallelism**: Full support for `pytest -n` with data merging

### Performance Impact

| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| **Log Size** (15K tests) | 60 MB | 3-5 MB | **-90%+** |
| **Memory Usage** (6 workers) | ~50 MB | ~450 MB | +400 MB |
| **CPU Overhead** | N/A | 0.0004% | Negligible |
| **Runtime I/O** | 75 seconds | 75 ms | **-99.9%** |

### Changes

**New Files:**
- `plugin/log_filter.py`: Conditional log handler with per-test memory
buffers
- `plugin/__init__.py`: Plugin package initialization

**Modified Files:**
- `config/log_config.py`: Simplified config (only JSON/HTML paths)
- `conftest.py`: Register log filter plugin
- `utils/util_log.py`: Removed redundant file handlers
- `utils/api_request.py`: Optimized API request logging
- `check/param_check.py`: Truncate long lists in error messages
- `common/common_func.py`: Removed verbose logging
- `pytest.ini`: Added log filter plugin

**Cleanup:**
- `milvus_client/test_add_field_feature.py`: Removed unused variable

### Test Plan

- [x] Syntax validation: All Python files pass `py_compile`
- [x] Module imports: All modules load successfully
- [x] Data structures: Verified serialization without report/buffer
objects
- [x] Log configuration: Confirmed only necessary attributes present
- [x] Small-scale test (17 tests):  Generated correct JSON + HTML
reports
- [x] Performance analysis: Memory/CPU profiling for 15K test scenario

### Example Output

**test_report.html** (human-readable):
- Beautiful color-coded UI
- Failed tests with full error traceback
- Expandable log sections by level (debug/info/warning/error)

**test_report.json** (AI-friendly):
```json
{
  "metadata": { "total_tests": 15000, ... },
  "summary": { "passed": 13500, "failed": 1500, ... },
  "tests": {
    "failed": [
      {
        "id": "test.py::test_func",
        "error": { "type": "AssertionError", ... },
        "logs": { "debug": [...], "error": [...] }
      }
    ]
  }
}
```

### Memory Safety

For the standard scenario (15K tests, 10% failure, 6 workers):
- Worker memory: ~53 MB each (318 MB total)
- Main process peak: ~112 MB
- Total system: **< 450 MB**  Safe

High-risk scenarios (>30% failure or >50K tests) may require optional
streaming optimizations (documented in performance analysis).

### Backward Compatibility

-  No breaking changes to external APIs
-  Existing test commands work unchanged
-  Compatible with all pytest plugins

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Claude Opus 4.5 <noreply@anthropic.com>
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2026-01-27 16:37:33 +08:00
marcelo-cjl 9e726e82d9
fix: improve error messages for nullable vector field operations (#46798)
issue: #46779 
related: #45993 

Return clear error when using is null/is not null filter on vector
fields
Return clear error when search by IDs with all null vectors
Fix nq mismatch when search by IDs with mixed null/valid vectors

Signed-off-by: marcelo-cjl <marcelo.chen@zilliz.com>
2026-01-08 17:45:25 +08:00
Feilong Hou 624147740b
test: fix timestamptz e2e case failure on Jenkins Weekly (#46210)
Issue: #46188 

Bug was caused by inconsistent version of tzdata as well as wrong month
assignment in convert_timestamptz function.
Also fix when debug_mode=True the compare function can correctly return
True or False.

---------

Signed-off-by: Eric Hou <eric.hou@zilliz.com>
Co-authored-by: Eric Hou <eric.hou@zilliz.com>
2025-12-09 16:09:15 +08:00
nico 43fe215787
test: update sdk version and skip some debug log (#46040)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-12-04 10:33:11 +08:00
yanliang567 13a52016ac
test: Update hybrid search tests with milvus client (#46003)
related issue: https://github.com/milvus-io/milvus/issues/45326

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-12-02 18:11:10 +08:00
yanliang567 1da75c0ee2
test: Update hybrid search tests to milvus client style (#45772)
related issue: #45326

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-11-24 17:55:07 +08:00
zhuwenxing 1e130683be
test: add geometry datatype in checker (#44794)
/kind improvement

---------

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2025-10-24 11:28:04 +08:00
yanliang567 d45274512c
test: Refactor search tests and remove useless common functions (#43608)
related issue: #40698

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-08-05 11:15:39 +08:00
yanliang567 abb3aeacdf
test: Refactor diskann and hsnw index, and update gen data functions (#43452)
related issue #40698
1. add diskann and hnsw index test
2. update gen_row_data and gen_column_data functions

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-07-23 22:04:54 +08:00
yanliang567 e8011908ac
test: Add tests for partition key filter issue and ttl eventually search (#43052)
related issue: #42918
1. add tests for ttl eventually search
2. add tests for partition key filter 
3. improve check query results for output fields 
4. verify some fix for rabitq index and update the test accordingly
5. update gen random float vector in (-1, 1) instead of (0,1)

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-07-02 11:02:43 +08:00
qixuan 3b2ed5815f
test:add test cases for add field (#42472)
issue: #42126

---------

Signed-off-by: qixuan <673771573@qq.com>
2025-06-11 17:06:39 +08:00
nico c827f4b948
test: update sdk version and cases (#42077)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-06-03 14:16:32 +08:00
binbin eea6b50fbb
test: add test cases for int8 vector (#41957)
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2025-05-23 09:24:28 +08:00
yanliang567 ee659d50db
test: [E2e Refactor] update search basic tests and add a pk_name instead of hard code (#41669)
related issue: https://github.com/milvus-io/milvus/issues/40698

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-05-09 21:58:54 +08:00
nico 5c11c4add1
enhance: update pymilvus version (#41695)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2025-05-08 10:04:54 +08:00
yanliang567 70b311735b
test: [E2e Refactor] use vector datatype instead of hard code dataype names (#41497)
related issue: #40698 
1. use vector datat types instead of hard code datatpe names
2. update search pagination tests
3. remove checking distances in search results checking, for knowhere
customize the distances for different metrics and indexes. Now only
assert the distances are sorted correct.

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-04-25 10:46:38 +08:00
yanliang567 6a9ecb760f
test: Update binary pagination search assertion for more stable (#41370)
related issue: #41362

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-04-17 14:16:32 +08:00
binbin 345538d10a
test: Add test cases for part of json path index (#40445)
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2025-03-10 14:30:07 +08:00
yanliang567 4437dae479
test: Add search iterator with external filter func neg tests (#40303)
related issue: https://github.com/milvus-io/milvus/issues/39985

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-03-05 10:58:00 +08:00
yanliang567 afbe02009e
test: Add test for search iterator external filter function (#39986)
related issue: #39985
1. add some test for external filter function
2. combine search iterator tests into one test file

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-02-21 11:25:53 +08:00
laurazhao0611 a5217e9f36
test: add database milvus client testcases (#39505)
/kind improvement

---------

Signed-off-by: laurazhao0611 <laurazhao@zilliz.com>
Co-authored-by: laurazhao0611 <laurazhao@zilliz.com>
2025-01-23 18:45:07 +08:00
qixuan 228bff5e7b
test: modify search iterator V2 case about alias/db/rbac for milvus client (#39400)
issue: #37548

Signed-off-by: qixuan <673771573@qq.com>
2025-01-20 10:25:03 +08:00
yanliang567 731e882d22
test: Refactor pymilvus client v2 testbase and add a test for search hint (#38939)
issue: https://github.com/milvus-io/milvus/issues/38877
1. refactor pymilvus client v2 testcasebase
2. add a test for search hint
3. update pymilvus to 2.6

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2025-01-06 11:24:55 +08:00
ThreadDao ea339c13c6
test: add cases for async milvus client (#38699)
issue: #38697

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-12-25 16:54:50 +08:00
yanliang567 6884319d03
test: [CP] add alter tests (#38659)
related issue: https://github.com/milvus-io/milvus/issues/38471

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-12-23 19:42:49 +08:00
yanliang567 af433ffd68
test: Add expression template tests and use error msg only as assertion (#37618)
related issue: https://github.com/milvus-io/milvus/issues/37451

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-11-13 17:14:30 +08:00
zhuwenxing c8dd665bf6
test: supplementing case for text match (#36693)
/kind improvement

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-10-28 10:31:40 +08:00
yanliang567 0d376f1a8f
test: Add query iterator test and seperate a new test file (#37085)
related issue: #37084

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-10-24 11:47:29 +08:00
binbin 0d57ff01a6
test: add null and default test cases (#36612)
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-10-08 17:25:24 +08:00
wt 14be89df29
test: add case to enable offset cache and mmap at the same time (#36505)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-25 19:59:13 +08:00
binbin 5ca4d5977a
test: Add bulk insert related test cases for default and null support (#36219)
issue: #36129

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-18 19:33:17 +08:00
yanliang567 2e434b2358
test: Add group size tests (#36240)
related issue: #36146

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-09-13 16:13:08 +08:00
wt cb49b32358
test: add bitmap index cases (#35909)
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-03 16:49:03 +08:00
yanliang567 019d76c79d
test: Enable group by tests in e2e (#31903)
enable groupby tests in e2e

---------

Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-04-08 10:25:16 +08:00
binbin a556671119
test: add hybrid search cases (#29830)
issue: #29799

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-02-20 11:58:51 +08:00
nico 168260cba3
test: update test cases about rbac (#29088)
issue: #29087
RBAC cases fail a lot.
1. some cases are out of date, for example, the default value of db_name
has changed from "default" to "" in some apis
2. add time sleep after the action of grant or revoke, for it costs time
to take effect

Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-01-30 11:33:02 +08:00
binbin a847d56ac0
test: add part of milvus client test cases (#30222)
issue: #30221

Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-01-26 09:25:01 +08:00
zhuwenxing 24bbb12166
test: add assert message if assert failed (#30186)
add assert message if assert failed

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2024-01-23 16:07:01 +08:00
nico aad3d47a06
update test cases (#26771)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2023-08-31 18:55:00 +08:00
MrPresent-Han f31c6786ab
fix e2e test for iterator due to interface change(#26552) (#26565)
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-28 11:32:26 +08:00
nico ec23b81f23
Add test cases of query iterator empty expression (#26294)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2023-08-17 16:42:18 +08:00
binbin 4ba922876e
Add test cases support for random primary keys (#25840)
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2023-07-25 19:39:12 +08:00
nico 9b64f12a6c
Add output fields value check (#24255)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2023-07-07 16:58:25 +08:00
nico 626516bb90
Add test cases of search iterator (#25039)
Signed-off-by: nico <cheng.yuan@zilliz.com>
2023-06-28 19:38:23 +08:00
binbin 31122a6858
Update high level api test cases (#25118)
Signed-off-by: Binbin Lv <binbin.lv@zilliz.com>
2023-06-28 14:18:51 +08:00