Commit Graph

1474 Commits (3c1dcb6f0a3480a0df5feb7cacd2d45ee97b3755)

Author SHA1 Message Date
Yu Kun 30b437582d add GpuCacheMgr and unittest
Former-commit-id: 57e2a59b5039fca56e10c56ee5474472d87faa9a
2019-08-22 22:14:17 +08:00
yudong.cai d416d1c6fc opt metrics
Former-commit-id: 3455b1ffc9d401e8e689c5833fb5e1985b3dc7b5
2019-08-22 19:16:49 +08:00
quicksilver 58a49caf45 update Check_Last_Modify function
Former-commit-id: 9e5d2130bb38f34eb381372a67350deda346870d
2019-08-22 19:10:57 +08:00
peng.xu 86779397b1 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-402 Add dump implementation for TaskTableItem

See merge request megasearch/milvus!405

Former-commit-id: 2e0906cdbc8e4bae7c14e3ae25f71683a71344c0
2019-08-22 19:02:20 +08:00
wxyu bfa7f7c95f MS-402 Add dump implementation for TaskTableItem
Former-commit-id: f2957c7e7cf0c719b9b814f2e8ac16b42c396b2b
2019-08-22 19:01:44 +08:00
starlord 51475e3d37 fix index size bug
Former-commit-id: fdc398b873ffc25d25eeff5c05ae87ecb0872937
2019-08-22 18:39:06 +08:00
Yu Kun a6ef8becf7 Merge remote-tracking branch 'upstream/branch-0.4.0' into branch-0.4.0
Former-commit-id: 064eb796b95e9d1679e39708b64ce8ee5f0a2bc5
2019-08-22 15:59:10 +08:00
starlord f0d9b0f5e4 fix row count bug
Former-commit-id: d2a9faa6a20ebeb88f8ef2fdb77180a6c216625c
2019-08-22 15:59:06 +08:00
Yu Kun 3d79ccd97b add GpuCacheMgr
Former-commit-id: c95181ef545748516cad9d3c0561fb8f753c6f37
2019-08-22 15:59:01 +08:00
peng.xu 59ff9b57f3 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
refine code

See merge request megasearch/milvus!402

Former-commit-id: 1b896db5f16336e00d848fe600709b4a20bf0179
2019-08-22 12:31:58 +08:00
starlord 50a4de6d93 refine code
Former-commit-id: 7283855c5965c6e9fb95cfcc3d0146fc7dcefccf
2019-08-22 12:31:00 +08:00
peng.xu e66685c1fa Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-400 Add timestamp record in task state change function

See merge request megasearch/milvus!401

Former-commit-id: 574adedca801d53ece9360c542e07803cc57bac8
2019-08-22 11:58:28 +08:00
wxyu 4f5466dbe8 MS-400 Add timestamp record in task state change function
Former-commit-id: 062045c8e5cbd0b0c71a05e0f5650ea943e4b3fa
2019-08-22 11:56:45 +08:00
starlord 5b918a4028 MS-339 implement new api
Former-commit-id: 53e4a9309807b06dc43761755146d54d5ef63cf5
2019-08-22 11:50:18 +08:00
starlord e6d4b638b7 merge 0.3.1
Former-commit-id: fbb9a3a12fbfe4a596a383eaed833f2d366c17a3
2019-08-22 09:26:08 +08:00
starlord 45a777b4d5 merge 0.3.1
Former-commit-id: 6fa5ce5365752369865360d715659c56fc367f89
2019-08-22 09:21:15 +08:00
peng.xu 42cdf9a11d Merge branch 'branch-0.4.0' into 'branch-0.4.0'
add DeleteByRange interface

See merge request megasearch/milvus!396

Former-commit-id: f875eab044460d5f2ce81bc04acc9784e959e8d0
2019-08-21 19:25:42 +08:00
peng.xu f31ae64221 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-394 Update scheduler unittest

See merge request megasearch/milvus!397

Former-commit-id: 2ad26a7acb68a6928150131526ed78ed2e0d373e
2019-08-21 19:25:28 +08:00
wxyu 598f9f30dc MS-394 Update scheduler unittest
Former-commit-id: b4f0fe18d00b17c47a39e2a06cdd1119ae00d7fc
2019-08-21 19:22:38 +08:00
Yu Kun acbec2f657 fix merge bugd
Former-commit-id: c3ee55050faefd76433e807842c7873c91dc6aba
2019-08-21 19:08:31 +08:00
Yu Kun 7c8770b399 merge upstream and fix comflicts
Former-commit-id: 1fd44ef580cb92d519da8a57d13c6ef8982efd26
2019-08-21 19:05:13 +08:00
starlord d30e432e16 merge 0.3.1
Former-commit-id: a538685c673319581c756289a4c003fa5833d829
2019-08-21 18:35:57 +08:00
wxyu 8b7273dadb MS-391 Add PushTaskToNeighbourHasExecutor action
Former-commit-id: 11c3e1f9fce24225d16c2d3f500c2f76e46388e2
2019-08-21 18:27:05 +08:00
wxyu 1c89d40c94 MS-390 Update resource construct function
Former-commit-id: 1f87f3e285798b2bf3dda78537fdb2b0fc7cc6ef
2019-08-21 18:22:52 +08:00
Yu Kun e83c4278db add deletebyrange interface and unittest
Former-commit-id: 54a94b0924d2de5089a4f31810e31a765bceb591
2019-08-21 18:02:37 +08:00
wxyu a79017ef8f MS-389 Add clone interface in Task
Former-commit-id: bf4681eec6cc6e6b21087b82e5bb5676ec3418d9
2019-08-21 18:01:23 +08:00
starlord 223cf1f547 MS-338 NewAPI: refine code to support CreateIndex
Former-commit-id: b3a60c3de2da5522b2b9becd8ba382825a1c1e05
2019-08-21 16:53:31 +08:00
Yu Kun c1e82cb749 add vector ids check in unittest
Former-commit-id: 26c60e303c5ede2200c77efee11b17a782d0dc46
2019-08-20 16:49:04 +08:00
Yu Kun da6e37c8b8 Merge remote-tracking branch 'upstream/branch-0.4.0' into branch-0.4.0
Former-commit-id: af1332f7b1aa9fe5a504644581a3658665d50e2b
2019-08-20 15:58:57 +08:00
Yu Kun 9c6c7c31ec add vector_ids in Insert
Former-commit-id: d793fa40cfa71e910e34998e3962df61de6f40f3
2019-08-20 15:58:46 +08:00
peng.xu 3b31d63b53 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-384 Add global instance of ResourceMgr and Scheduler

See merge request megasearch/milvus!391

Former-commit-id: 1030c47545d1d972c64ab967188f0e6ccf7b287e
2019-08-20 15:14:06 +08:00
wxyu 6171d3bbe0 MS-384 Add global instance of ResourceMgr and Scheduler
Former-commit-id: f176c49d75dbb820b37c356fde99cd62cdc2c9c9
2019-08-20 11:10:52 +08:00
jinhai 7f77711938 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-383 Modify condition variable usage in scheduler

See merge request megasearch/milvus!390

Former-commit-id: 262de61e8cbc2430d7dfea90e5db0b8c8cbccd2b
2019-08-19 22:03:45 +08:00
wxyu 03b049b10b MS-383 Modify condition variable usage in scheduler
Former-commit-id: 5be1c8879d1a77d444a5722dbd806a5eb00973e5
2019-08-19 22:01:39 +08:00
Yu Kun f944df41bd Merge remote-tracking branch 'upstream/branch-0.4.0' into branch-0.4.0
Former-commit-id: 81ebc3a6ab38577ba4945b7db4022d9725586f19
2019-08-19 20:17:05 +08:00
peng.xu 8b3d867a43 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-380 Update resource loader and executor, work util all finished

See merge request megasearch/milvus!387

Former-commit-id: 2c942b172d24ffa494c756b5ce0eca9bad9f0e42
2019-08-19 11:49:48 +08:00
wxyu 842fa507fb MS-380 Update resource loader and executor, work util all finished
Former-commit-id: 712a0aceaa4c8d4ebbea40f5d18f524afeb38559
2019-08-19 11:48:45 +08:00
xj.lin 0d0b8eeb3f MS372 fix unittest
Former-commit-id: 78f0c12c86ed19158b982d1f460e4a2497bb0585
2019-08-19 11:11:28 +08:00
wxyu 41f3a2ac2b MS-379 Add Dump implementation in Resource
Former-commit-id: 0ad824b5d582fc7235a0e29919fdf5a2975bf534
2019-08-19 10:47:30 +08:00
jinhai 4f8ccf9d82 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-373 Add resource test

See merge request megasearch/milvus!383

Former-commit-id: 579f97477d802bd344d4a393cb51ea910f98b534
2019-08-18 20:19:27 +08:00
wxyu 08a0f545da MS-378 Debug and Update normal_test in scheduler unittest
Former-commit-id: 7b5add0a2d27185abeec2a1c5df51716aa9fc361
2019-08-18 20:10:32 +08:00
wxyu 07ec15f9cd MS-377 Improve process thread trigger in ResourceMgr, Scheduler and TaskTable
Former-commit-id: f121fba66ae395b03ae31c400216b5e9c301f0cf
2019-08-18 20:08:15 +08:00
wxyu 4ed498893d MS-376 Add loader and executor enable flag in Resource avoid diskresource execute task
Former-commit-id: 489f60f97444c963a3dbfd913672f79b8a9a330e
2019-08-18 20:04:54 +08:00
wxyu de6d68238f MS-375 Add Dump implementation for Event
Former-commit-id: c8518b48914da67bce45660866970c8c4eee915a
2019-08-18 20:01:44 +08:00
wxyu 40d8172671 MS-374 Add action definition
Former-commit-id: 68c221b2845cb3fd640433eea72dba4490ca977f
2019-08-18 19:59:56 +08:00
wxyu ac230a7ec3 MS-374 Add action definition
Former-commit-id: d3ed4be48d8c27f380b9d20d7eb1bff6abe7cee0
2019-08-18 19:59:15 +08:00
wxyu a591e965c9 MS-373 Add resource test
Former-commit-id: 64c86f4852c4d0cb6d31a679e63ad7555e03d088
2019-08-17 19:08:10 +08:00
wxyu 4cb3e103fc MS-371 Add TaskTableUpdatedEvent
Former-commit-id: f252d63664cb09b502c7280d7a82f0cb1fcac169
2019-08-17 19:05:58 +08:00
jinhai e78231f995 Merge branch 'ms-372' into 'branch-0.4.0'
MS372 dev clone index and get index deviceId

See merge request megasearch/milvus!381

Former-commit-id: 82df26e98dbc3e09e90b38fb3cc27dfc70a0f62e
2019-08-17 17:34:09 +08:00
jinhai 6fa0303a3f Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-371 Add TaskTableUpdatedEvent

See merge request megasearch/milvus!380

Former-commit-id: 5861036ac83035935f549939ab0d201aaf632c91
2019-08-17 17:34:02 +08:00
xj.lin b3cc7de2c0 MS372 dev clone index and get index deviceId
Former-commit-id: adc85e54cc20c644f9d39df823c8b1b539ec7a19
2019-08-17 17:24:09 +08:00
wxyu 2c9bb7224d MS-371 Add TaskTableUpdatedEvent
Former-commit-id: cfcd409967d2e9d8f1747432f3b99c5e75ded768
2019-08-17 17:10:18 +08:00
jinhai a1d37d6c4f Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-368 Implement cost.cpp

See merge request megasearch/milvus!378

Former-commit-id: 2eb14ab3ce37df22b781d88c64baa702d9e4b971
2019-08-16 19:58:17 +08:00
wxyu 2ea773b930 MS-368 Implement cost.cpp
Former-commit-id: 554f06b109662bb381bb06aa73ddab956fb8c47c
2019-08-16 17:51:05 +08:00
jinhai 5b21339b24 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-366 Implement TaskTable

See merge request megasearch/milvus!376

Former-commit-id: 8c306fc80d70ec2e799efe9a79ebe0b8f4f922e2
2019-08-16 17:46:17 +08:00
wxyu 484f255f5e MS-366 Implement TaskTable
Former-commit-id: e6d189c5c63b0477384353bf6f75482eeb196c7d
2019-08-16 16:09:02 +08:00
jinhai 4f642936e5 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-365 Use tasktableitemptr instead in event

See merge request megasearch/milvus!375

Former-commit-id: baba5e363f2d9cb1a674fe64b63cc063329e4fae
2019-08-16 15:58:04 +08:00
wxyu 404bc88b3f MS-365 Use tasktableitemptr instead in event
Former-commit-id: a2d8fe7456dc59da280f475ae489a63fa9c3d362
2019-08-16 15:36:56 +08:00
jinhai 90c5e47bfc Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-364 Modify tasktableitem in tasktable

See merge request megasearch/milvus!374

Former-commit-id: 045ac24b40f35cb5b36835efbb2b7489d8c3fae3
2019-08-16 15:31:08 +08:00
wxyu 8ffd68ec51 MS-364 Modify tasktableitem in tasktable
Former-commit-id: 79ac92b18cc3842d6ccaceb6bdd50ff636ca80ab
2019-08-16 15:29:35 +08:00
jinhai afe77c7015 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-361 Add event in resource

See merge request megasearch/milvus!373

Former-commit-id: aba409e3abf11a9435898a02b031cca13502f5f9
2019-08-16 15:04:22 +08:00
wxyu df382a1644 MS-361 Add event in resource
Former-commit-id: d5f18c13d7111a582ad0bc839aa49879c6bb7d32
2019-08-16 15:03:10 +08:00
Yu Kun 5b26d194f3 add unittest
Former-commit-id: 89139a24caab971fa16b27b814ca4731efd0ea20
2019-08-16 10:20:21 +08:00
jinhai 44846cc1d9 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-359 Add cost test in new scheduler

See merge request megasearch/milvus!370

Former-commit-id: 68f9f10ea3f70d068d01437ba267cd89c7db7f54
2019-08-15 19:56:07 +08:00
wxyu c3469fa561 MS-359 Add cost test in new scheduler
Former-commit-id: 5ec53217a216927e3b0ddf211f1a56bb1a44f2b4
2019-08-15 19:09:59 +08:00
peng.xu 145ee0f514 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
add nprobe in search and preloadtable unittest

See merge request megasearch/milvus!369

Former-commit-id: 4c32c4bc443b2ee61df2708bc47994ad39be1284
2019-08-15 18:31:03 +08:00
Yu Kun fabcbe9172 fix preloadtable unittest bug
Former-commit-id: 8d19429bb3d4d58a1fcc31700dcc74685188280e
2019-08-15 18:08:46 +08:00
xj.lin cbc734d83b MS-337 dev basic Resource
Former-commit-id: 7486415fa96cd3db77320aeb4b01dd78a2b4c878
2019-08-15 15:58:08 +08:00
wxyu 9400101582 MS-357 Add minimum schedule function
Former-commit-id: da78df540b934e3f1d881b16f023d749e4b37101
2019-08-15 15:57:12 +08:00
Yu Kun 536daeea16 Merge remote-tracking branch 'upstream/branch-0.4.0' into branch-0.4.0
Former-commit-id: 88fb929978ce4a016f8402c444668363aba35065
2019-08-15 15:14:59 +08:00
xj.lin c6f7423c79 MS-337 dev basic Resource
Former-commit-id: b2a6cc6eb4c8fb7509280bff4b3fccd77e73870f
2019-08-15 15:01:37 +08:00
Yu Kun 03b28a9bcd Merge remote-tracking branch 'upstream/branch-0.4.0' into branch-0.4.0
Former-commit-id: 2afda3c33fc5c4970005991d77937600904c312b
2019-08-15 10:32:43 +08:00
Yu Kun 608aa1b784 add nprobe and preloadtable unittest
Former-commit-id: dfaa1c4b36068c9de7499b48094ce39021d1213b
2019-08-15 10:32:36 +08:00
jinhai d5b6eceb66 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
MS-355 Add copy interface in ExcutionEngine

See merge request megasearch/milvus!363

Former-commit-id: 4f19f0cdd109e74fdec0d9d5b992a6b7c384b8d7
2019-08-14 21:26:57 +08:00
wxyu c11c58a555 MS-355 Add copy interface in ExcutionEngine
Former-commit-id: 9320dde8c2452982c552f604289d2661e5c1907e
2019-08-14 21:24:20 +08:00
Jeff dc1f80d2d0 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
add gperftools dependencies

See merge request megasearch/milvus!361

Former-commit-id: 1ce0bb9c3dc1ee3f4ad61258e1db3376c48f0323
2019-08-14 20:36:04 +08:00
quicksilver 7cc6e173b1 add gperftools dependencies
Former-commit-id: cc8e3f68f5fbe3393ac966083b607b54ccb50fe8
2019-08-14 20:35:21 +08:00
jinhai 0cd9a97487 Merge branch 'MS-351' into 'branch-0.4.0'
fix bug unittest

Closes MS-351

See merge request megasearch/milvus!360

Former-commit-id: f1fe91d674b212bc4297bcdba846a693de4606d0
2019-08-14 20:19:11 +08:00
xj.lin 4e20eb16a4 fix bug unittest
Former-commit-id: bb332dbf8545bc4e3528bdf699e38cf396be902f
2019-08-14 20:18:13 +08:00
wxyu 7cb9401716 MS-354 Add task class and interface in scheduler
Former-commit-id: b76186ae5ae4057a704279a6ed24bfd17878c445
2019-08-14 20:12:14 +08:00
wxyu 9d7302cd7c MS-354 Add task class and interface in scheduler
Former-commit-id: a8697bf20ed0db126cc6857c6d7719a4731bd87d
2019-08-14 20:10:31 +08:00
wxyu 4de6f675ec MS-354 Add task class and interface in scheduler
Former-commit-id: 0a61d095db7144006b80bc0824886e62be71a5d5
2019-08-14 20:09:00 +08:00
xj.lin 59b63524da Merge branch 'branch-0.4.0' into MS-351
Former-commit-id: 2c8203f1e953247ad3bab65052d2b44811cc94e3
2019-08-14 19:50:21 +08:00
xj.lin 387e15455f MS-351 enable index move between cpu<=>gpu
Former-commit-id: 40841d540f1ded276f8455ffd863789edc2a1796
2019-08-14 19:48:48 +08:00
peng.xu 4b887cf094 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
add PreloadTable and some parameters

See merge request megasearch/milvus!355

Former-commit-id: b9e0a21d137dfb22983d1ed4e71e1f91354bd49c
2019-08-14 19:45:10 +08:00
Yu Kun b230f19ce5 fix bugs with discessions
Former-commit-id: 54cd32977b70608898c6c4c61e5971d4716a8e08
2019-08-14 19:23:56 +08:00
quicksilver 027ee6920c update build.sh
Former-commit-id: 800a68080248c20b0c83b86360205bd395ee0371
2019-08-14 19:17:03 +08:00
Yu Kun d7936a6c60 add some parameters and merge upstream
Former-commit-id: 680349208d62108ceda4ad822c466107a840327c
2019-08-14 19:06:38 +08:00
Yu Kun 83ab3a640c Merge remote-tracking branch 'upstream/branch-0.4.0' into branch-0.4.0
Former-commit-id: 7cf01fff2fd7f33db79845d5fe6ca1972262975e
2019-08-14 19:04:47 +08:00
quicksilver 8b7581cad8 update build.sh
Former-commit-id: 1bf5c621573c58f865e996cad2ad61b488155722
2019-08-14 17:32:52 +08:00
starlord 7047472cf7 for jenkins build
Former-commit-id: e69f393d71c54489917100acc18d125ded248471
2019-08-14 17:13:02 +08:00
starlord 75e1fa90e1 for jenkins build
Former-commit-id: fa78b4d2201879fd6fe2619d2b0e498c5f9bf433
2019-08-14 17:09:33 +08:00
quicksilver 1246bfd89e build knowhere
Former-commit-id: e743650d98d8131ae50952f65500728c379c76bc
2019-08-14 15:58:30 +08:00
Yu Kun 5ef5c11140 Merge remote-tracking branch 'upstream/branch-0.4.0' into branch-0.4.0
Former-commit-id: 259fd94df3c95b445ccd8ff34be60f7343b873ee
2019-08-14 15:53:46 +08:00
Yu Kun ccec483e1e add preloadtable
Former-commit-id: f052a555777fce2a1d27478f7b2bb51d1e536c7c
2019-08-14 15:50:23 +08:00
starlord a7488c2478 milvus default version 0.4.0
Former-commit-id: a62048f862cb3c36187f59c19f14b31251291156
2019-08-14 15:25:50 +08:00
starlord 88d4020b4f fix conflict
Former-commit-id: 307526aa00a017b74c28b9a4d123b885631dd673
2019-08-14 14:46:30 +08:00
starlord d46d1e7f0d MS-350 remove submodule
Former-commit-id: 34c3bc3c966b666534229bed1ef03e30ad9c527b
2019-08-14 14:44:50 +08:00
starlord d9c8e22ac5 remove knowhere submodule
Former-commit-id: 949cb32d38482a60c357e8f95a84e0a546c5e98b
2019-08-14 10:37:38 +08:00
peng.xu 2e50a88f11 Merge branch 'branch-0.4.0' into 'branch-0.4.0'
default grpc

See merge request megasearch/milvus!344

Former-commit-id: 726e8e5f3041f11933b4c096e839065ca4627940
2019-08-14 10:18:33 +08:00