milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
milvus/internal/datanode/importv2
History
yihao.dai 9cbd194c6b
fix: Prevent import from generating small binlogs (#43132) 
- Introduce dynamic buffer sizing to avoid generating small binlogs
during import
- Refactor import slot calculation based on CPU and memory constraints
- Implement dynamic pool sizing for sync manager and import tasks
according to CPU core count

issue: https://github.com/milvus-io/milvus/issues/43131

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 		2025-07-07 21:32:47 +08:00
..
hash.go enhance: bulkinsert handles nullable/default (#42127) 2025-05-28 18:02:28 +08:00
hash_test.go fix: Fix no candidate segments error for small import (#41771) 2025-05-14 15:30:21 +08:00
memory_allocator.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
memory_allocator_test.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
mock_task.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
mock_task_manager.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
pool.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
pool_test.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
scheduler.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
scheduler_test.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
task.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
task_import.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
task_l0_import.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
task_l0_import_test.go enhance: ChunkManager is no longer created during datanode initialization (#42791) 2025-06-17 17:06:38 +08:00
task_l0_preimport.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
task_manager.go enhance: Use v2 package name for pkg module (#39990) 2025-02-22 23:15:58 +08:00
task_manager_test.go enhance: Use v2 package name for pkg module (#39990) 2025-02-22 23:15:58 +08:00
task_preimport.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
util.go fix: Prevent import from generating small binlogs (#43132) 2025-07-07 21:32:47 +08:00
util_test.go enhance: bulkinsert handle nullable/defaultValue/functionOutput fields (#42956) 2025-07-04 14:20:44 +08:00