mirror of https://github.com/milvus-io/milvus.git
6b4171e7ac
design doc: https://github.com/milvus-io/milvus-design-docs/blob/main/design_docs/20260105-external_table.md issue: #45881 This change introduces manual refresh capability for external collections, allowing users to trigger on-demand data synchronization from external sources. It replaces the legacy update mechanism with a more robust job-task hierarchy and persistent state management. Key changes: - Add RefreshExternalCollection, GetRefreshExternalCollectionProgress, and ListRefreshExternalCollectionJobs APIs across Client, Proxy, and DataCoord - Implement ExternalCollectionRefreshManager to manage refresh jobs with a 1:N Job-Task hierarchy - Add ExternalCollectionRefreshMeta for persistent storage of jobs and tasks in the metastore - Add ExternalCollectionRefreshChecker for task state management and worker assignment - Implement ExternalCollectionRefreshInspector for periodic job cleanup - Use WAL Broadcast mechanism for distributed consistency and idempotency - Replace legacy external_collection_inspector and update tasks with the new refresh-based implementation - Add comprehensive unit tests for refresh job lifecycle and state transitions design doc: https://github.com/milvus-io/milvus-design-docs/blob/main/design_docs/20260105-external_table.md --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com> |
||
|---|---|---|
| .. | ||
| properties.go | ||
| properties_test.go | ||
| state.go | ||
| time.go | ||
| type.go | ||