mirror of https://github.com/milvus-io/milvus.git
6b4171e7ac
design doc: https://github.com/milvus-io/milvus-design-docs/blob/main/design_docs/20260105-external_table.md issue: #45881 This change introduces manual refresh capability for external collections, allowing users to trigger on-demand data synchronization from external sources. It replaces the legacy update mechanism with a more robust job-task hierarchy and persistent state management. Key changes: - Add RefreshExternalCollection, GetRefreshExternalCollectionProgress, and ListRefreshExternalCollectionJobs APIs across Client, Proxy, and DataCoord - Implement ExternalCollectionRefreshManager to manage refresh jobs with a 1:N Job-Task hierarchy - Add ExternalCollectionRefreshMeta for persistent storage of jobs and tasks in the metastore - Add ExternalCollectionRefreshChecker for task state management and worker assignment - Implement ExternalCollectionRefreshInspector for periodic job cleanup - Use WAL Broadcast mechanism for distributed consistency and idempotency - Replace legacy external_collection_inspector and update tasks with the new refresh-based implementation - Add comprehensive unit tests for refresh job lifecycle and state transitions design doc: https://github.com/milvus-io/milvus-design-docs/blob/main/design_docs/20260105-external_table.md --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com> |
||
|---|---|---|
| .. | ||
| alias.go | ||
| analyzer.go | ||
| collection.go | ||
| collection_attr.go | ||
| collection_attr_test.go | ||
| common.go | ||
| database.go | ||
| external_table.go | ||
| external_table_test.go | ||
| field.go | ||
| field_test.go | ||
| function.go | ||
| function_test.go | ||
| load_state.go | ||
| privilege_group.go | ||
| rbac.go | ||
| resource_group.go | ||
| schema.go | ||
| schema_example_test.go | ||
| schema_test.go | ||
| segment.go | ||
| sparse.go | ||
| sparse_test.go | ||
| vectors.go | ||
| vectors_test.go | ||