milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
15,223 Commits
169 Branches
152 Tags
621 MiB
Commit Graph

807 Commits (perf)

Author SHA1 Message Date
aoiasd 6e9c8f7cf3
Optimize large memory usage of InsertRecord (#19197 #19245) (#19421) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-09-27 11:50:56 +08:00
Cai Yudong e5f70676ac
Ignore cases when comparing metric type in segcore (#19436) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-09-26 17:54:53 +08:00
Jiquan Long a6c28cd713
Set default slice size for scalar index (#19326) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-09-22 11:36:51 +08:00
Cai Yudong ab30ebb1c2
Ugrade to knowhere-v1.2.0 (#18745) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-08-22 20:52:52 +08:00
congqixia 1e1b63d8fe
Fix DataType::bool cast to double (#18690) (#18693) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-17 22:14:54 +08:00
Ten Thousand Leaves f96ad7ec6c
Make DataNode not panic if stale segment is not found during flush (#18345) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-20 21:18:29 +08:00
zhenshan.cao 461a022e65
Return empty result in advance if all data filtered out (#18329) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-19 19:34:28 +08:00
Jeng.Gwan 774199669c
Fix leak of memory in ParseMallocInfo while assert false (#18311) (#18324) 
Signed-off-by: xaxys <zheng.guan@zilliz.com>

(cherry picked from commit db71c74cc7)
Signed-off-by: xaxys <zheng.guan@zilliz.com>
 2022-07-19 15:44:28 +08:00
xige-16 f6795dc722
Fix pk2offset occupy too much memory (#18269) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-07-14 12:00:25 +08:00
Jeng.Gwan 4ea7adae1d
Fix redundant string copy of PlaceholderGroup (#18236) (#18237) 
Signed-off-by: xaxys <zheng.guan@zilliz.com>
(cherry picked from commit a926a9eb66)
 2022-07-12 16:36:25 +08:00
xige-16 cb850e5540
Avoid invalid cal when multi repeated delete pks (#18225) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-07-12 10:32:25 +08:00
xige-16 85117efdc8
Fix query too slow when insert multi repeated pk data (#18189) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-07-11 10:26:24 +08:00
bigsheeper 507518f869
Get search field id from search plan and log if loaded index when search segments (#18184) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-08 21:58:21 +08:00
zhagnlu 0bad8f1bf8
Fix core dump when nq has no topk result (#17923) (#18051) (#18083) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Co-authored-by: zhagnlu <lu.zhang@zilliz.com>

Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-07-06 10:02:21 +08:00
Jiquan Long f6ce05590c
Fix search successfully with invalid metric type (#17977) (#18026) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-07-04 10:28:20 +08:00
bigsheeper ad2526a039
Return malloc_trim result and log if malloc_trim successfully (#17960) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-01 10:40:19 +08:00
xige-16 622a6952e7
Fix query not empty after delete string (#17949) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-30 15:20:22 +08:00
bigsheeper 34b855c57b
Remove temporary variable to prevent memory fragmentation (#17743) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-28 16:30:23 +08:00
Jiquan Long 90ca61d948
Fix double-freed global variable in easylogging lib (#17730) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-23 19:12:14 +08:00
Cai Yudong e78269f450
Optimize search related interface in segcore (#17568) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-06-23 18:16:13 +08:00
bigsheeper 1722431699
Checking for existence in map in segcore (#17700) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-22 18:16:14 +08:00
Jiquan Long fdccfb4273
Fix ut heap overflow (#17706) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-22 16:56:13 +08:00
bigsheeper 92d06b2e30
Purge memory by the memory state and try to purge after each search (#17565) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-17 17:46:10 +08:00
Ten Thousand Leaves 65ab847815
Add segment reference lock for bulk load (#17457) 
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-15 12:20:10 +08:00
Jiquan Long e751663fdc
Fix err msg when the dimension of query vectors mismatch field (#17553) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-15 10:54:09 +08:00
bigsheeper cdcdfa1ea5
Disable jemalloc and use malloc_trim instead (#17538) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-14 11:12:09 +08:00
xige-16 36ad989590
Fix segOffset grater than insert barrier when mark delete (#17444) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-10 20:02:08 +08:00
bigsheeper 2d9a52206d
Use jemalloc in QueryNode, DataNode and IndexNode (#17470) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-10 11:24:08 +08:00
xige-16 6d078ebe7a
Print more information when loading fields with different lines (#17448) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-08 20:16:06 +08:00
bigsheeper af108b5bda
Add malloc_trim after reduce search result (#17423) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-08 00:12:05 +08:00
xige-16 b5c11a216d
Alter varChar type params's name to max_length (#17409) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-07 15:58:06 +08:00
bigsheeper f38637c227
Pass PlaceholderGroup pointer to prevent memory copy in SegCore (#17389) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-06 21:34:05 +08:00
bigsheeper 9f786dd752
Remove unnecessary bitset flip in Search and Query (#17387) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-06 17:38:05 +08:00
Jiquan Long e88ffb8a57
Enable marisa trie ut on MacOS (#17316) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-02 10:48:03 +08:00
Xiaofan 9579a645c6
Support compile marisa on Macos (#17261) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-31 13:28:02 +08:00
SimFG 731870211a
Support role-based access control (only pb) (#16987) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-05-28 00:04:01 +08:00
Letian Jiang f2a27e0e64
Retry GetShardLeaders until service available or timeout (#17183) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-26 20:28:02 +08:00
xige-16 56778787be
Reverse data from scalar index (#17145) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-26 14:58:01 +08:00
xige-16 7d810ac159
Fix query empty when insert same pk after deletion (#17222) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-25 22:06:00 +08:00
Ten Thousand Leaves 5e1e7a6896
Fix an issue that causes flush hanging (#17169) 
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-25 14:34:00 +08:00
xige-16 a8829554eb
Merge multiple sets of expr's optypes in diff dirs (#17192) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-24 21:56:00 +08:00
zhenshan.cao d05c24a0ff
Delete unused code (#17182) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-24 14:49:59 +08:00
zhenshan.cao 08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-23 16:41:58 +08:00
Enwei Jiao d28a2db46c
move arrow from storage to core (#17061) 
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
 2022-05-22 20:03:58 +08:00
yah01 7746a5b742
Add NodeIds field for QuerySegmentInfo (#17121) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-20 18:03:58 +08:00
Jiquan Long 99b4042f8c
Disable segment small index (#16935) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-18 15:27:56 +08:00
Jiquan Long 77b0f290a8
Fix prefix match on sealed segment (#17050) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-17 20:15:55 +08:00
xige-16 935b729a0c
Fix multiple deletes not taking effect (#17033) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-16 18:39:56 +08:00
XuanYang-cn 43b36b4e5f
Make proxy update cache with NotShardLeader status code (#16940) 
See also: #16926

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-13 11:35:53 +08:00
Jiquan Long c5149c7e85
Replace text codec of pb with binary codec (#16955) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-13 09:53:53 +08:00