milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
18,435 Commits
152 Branches
122 Tags
588 MiB
Commit Graph

7765 Commits (13a5b9f64a4714746abe311bde4b3bb848413019)

Author SHA1 Message Date
congqixia af1c2044b9
Fix atomic.Int64 not found in go 1.18 (#28216) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-07 15:16:21 +08:00
groot 3f6b203018
Fix bulkinsert bug that segments are compacted after import (#28192) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-11-07 15:14:26 +08:00
wei liu 5b45a138b1
disable auto balance when old node exists (#28191) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-07 14:02:20 +08:00
wei liu da41a5b51e
fix check grpc error logic (#28182) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-07 11:54:18 +08:00
congqixia c41df18b6d
Add compaction meta in `SyncSegmentsRequest` (#28199) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-07 10:06:17 +08:00
XuanYang-cn 22c089894e
Add scheduler (#27938) 
See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-07 03:18:18 +08:00
yah01 ece592a42f
Deliver L0 segments delete records (#27722) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-07 01:44:18 +08:00
sre-ci-robot b1df3ead0e
Update knowhere commit (#28176) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-11-06 22:44:19 +08:00
XuanYang-cn 2bbc20c7b8
Update session manager (#27761) 
See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-06 22:10:19 +08:00
Xiaofan da19e49daf
Support purge old session for standalone (#28184) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-11-06 21:21:42 +08:00
yah01 863e26969a
Reduce 1x memory copy for retrieving data (#28106) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-06 17:48:19 +08:00
aoiasd 1d4be0d257
Adjust datacoord for L0 Delta (#28021) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-06 15:26:16 +08:00
chyezh 86973a8871
[Fixup] error root path for catalog when using tikv (#28163) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-11-06 12:06:19 +08:00
yah01 90e2c63d9e
Fix getting incorrect CPU num (#28146) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-11-06 06:02:16 +08:00
congqixia bf2f62c1e7
Add `WriteBuffer` to provide abstraction for delta policy (#27874) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-04 12:10:17 +08:00
cai.zhang 8011054a2a
Check length before comparing strings (#28110) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-04 10:04:29 +08:00
yihao.dai 8d080383b7
Use merr to prevent datanode panic (#28121) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-11-04 00:58:21 +08:00
congqixia 501cba90ce
Use IVFFLAT_CC instead of IVFFLAT for sealed segment tmp index (#28164) 
See also #28152

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-03 23:50:15 +08:00
wei liu 68a86471ba
fix grpc client retry on node server not match error (#28169) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-03 23:42:16 +08:00
wei liu 7485eeb689
fix sync distribution with wrong version (#28130) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-03 19:02:18 +08:00
Jiquan Long a21042dde7
Fix flushManager.isFull is too slow (#28141) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-11-03 14:42:17 +08:00
SimFG ef68680639
Fix the unstable resource manager testcase (#28033) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-03 11:00:17 +08:00
wayblink 00ae019ff0
Use go 1.20 csv_reader to keep milvus go=1.18 limitation (#28080) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-11-03 10:40:16 +08:00
aoiasd d2727cc0ff
Add write cache for access log (#27792) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-03 10:28:17 +08:00
sre-ci-robot 7f28e9d2f3
Update knowhere commit (#28087) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-11-03 10:24:16 +08:00
wei liu ecec5dfcfd
fix retry on offline node (#28079) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-03 10:14:16 +08:00
wei liu 86ec6f4832
fix load index for stopping node (#28047) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-03 07:58:18 +08:00
congqixia 1e51255c15
Implement `Injection` for SyncManager with block and meta transition (#28093) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-03 04:48:15 +08:00
cai.zhang fc2df9514f
Refine code for fixed-length types array (#28108) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-03 00:40:14 +08:00
yah01 dc89730a50
Support collection-level mmap control (#26901) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-02 23:52:16 +08:00
congqixia 5d2eba2c2f
Set qcv2 index task priority to Low (#28117) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-02 23:22:17 +08:00
wei liu 9b737b77a7
fix fromShardLeader typo (#28061) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-02 15:50:16 +08:00
Enwei Jiao f8dd589755
Refactor collection's cgo call (#28055) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-02 13:02:13 +08:00
yihao.dai 7bd44bd671
Fix unstable search task ut (#28101) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-11-02 12:52:15 +08:00
yihao.dai 873b29e226
Fix unstable cpp ut (#28083) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-11-02 00:58:16 +08:00
MrPresent-Han 854c0e8a16
fix precision for segcore reduce(#27325) (#28062) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-11-01 21:52:13 +08:00
Xiaofan af8ea0f3af
Revert "Remove Properties in rocksmq (#28000)" (#28071) 
This reverts commit 7c9d24cbb7.
 2023-11-01 21:44:20 +08:00
congqixia e4fdf5e68e
Refine offline segments logic in shard delegator (#28073) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-01 20:04:24 +08:00
groot abd5b199cc
Bulkinsert support pure list json (#27990) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-11-01 19:02:13 +08:00
PowderLi 0c0f012e03
add a custom http header: Accept-Type-Allow-Int64 (#27901) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-01 11:42:16 +08:00
wei liu 0c69f48ba4
refine stop order (#28016) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-01 11:40:15 +08:00
PowderLi 0252871d30
fix azure ListObjects (#27931) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-01 11:34:14 +08:00
aoiasd 2aa330146a
Delete by complex expression use partitionKey optimization (#27952) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-01 11:26:16 +08:00
Enwei Jiao 8ae9c947ae
Use OpenDAL to access object store (#25642) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-01 09:00:14 +08:00
cqy123456 4fbe3c9142
replace loaded binlog with binlog index for search performance (#27673) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-11-01 02:20:15 +08:00
KumaJie e88212ba4b
Add CSV file import function (#27149) 
Signed-off-by: kuma <675613722@qq.com>
Co-authored-by: kuma <675613722@qq.com>
 2023-10-31 22:47:23 +08:00
wei liu 0677d2623d
remove retry in getShards (#28011) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-31 16:16:14 +08:00
yah01 1b9b8a31a8
Fix QueryNode panic while upgrading (#28034) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 15:50:13 +08:00
yah01 ab6dbf7659
Limit max thread num for pool (#28018) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 13:00:13 +08:00
yah01 9658367a3c
Refine chunk manager errors (#27590) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 12:18:15 +08:00
Xiaofan 38aa53f523
Fix datacoord start timeout (#28026) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-31 10:04:12 +08:00
yah01 2af46d7333
Increase the ChunkManager request timeout (#28015) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 09:06:13 +08:00
congqixia 233bf90c55
Add SyncManager to replace flush manager (#27873) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-31 02:30:16 +08:00
Xiaofan 7c9d24cbb7
Remove Properties in rocksmq (#28000) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-31 00:44:14 +08:00
yah01 9b6eeb46f1
Cache segment row num, size, and insert count to reduce CGO calls (#28007) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-30 17:54:14 +08:00
yihao.dai ab6b0103a3
Get vector concurrently (#27838) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-30 15:44:12 +08:00
MrPresent-Han f93ad6471c
enable query reduce on querynodes for shortcut return(#27306) (#27837) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-10-30 15:08:11 +08:00
Ikko Eltociear Ashimine ed4f20b0ed
Fix typo in util.go (#27975) 
suppot -> support

Signed-off-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
 2023-10-30 14:40:27 +08:00
wei liu 7e0bda8b4c
remove binlog/delta log from getRecoveryInfoV2 (#27895) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-30 14:28:15 +08:00
congqixia 36ddeae2cc
Refine log level when request resource fail for loading segments (#28004) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-30 14:20:14 +08:00
Jiquan Long 6913283716
Fill output fields for count request (#27967) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-10-30 10:00:12 +08:00
congqixia 98e2aad752
Refine datanode metacache and implement CoW (#27985) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-28 11:12:11 +08:00
MrPresent-Han ccaec44930
enable parameter for dataCoord broker timeout(#26972) (#27825) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-10-28 06:14:11 +08:00
sre-ci-robot 1ae6e5d8c8
Update knowhere commit (#27993) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-10-28 04:24:10 +08:00
groot e6d4cde397
Fix duplicated segment id in bulkinsert task state (#27915) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-10-27 15:14:13 +08:00
yah01 f212158d61
Fix delete records timestamp may be reordered (#27941) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-27 10:24:10 +08:00
Enwei Jiao 4a33391b8f
rename createindex (#27903) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-10-27 10:12:14 +08:00
smellthemoon 403c6680cc
Add error description (#27959) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-10-27 10:10:13 +08:00
Filip Haltmayer 6b1a106a31
Moving etcd client into session (#27069) 
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
 2023-10-27 07:36:12 +08:00
congqixia 852be152de
Change task sourceID to stringer interface (#27965) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-27 01:08:12 +08:00
yihao.dai f9c630247d
Construct plan directly when search with vector output (#27928) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-26 19:30:10 +08:00
congqixia 13877a07ff
Add ctx parameter for tsafe pkg & NewDelegator method (#27877) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-26 19:14:10 +08:00
shaoyue 977e508a0f
Propagate task ctx done err for proxy (#27892) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-10-26 19:06:13 +08:00
jaime bf7f32b025
Refine Proxy stop (#27919) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-26 10:24:09 +08:00
wei liu 8041fc3c75
avoid add empty growing segment to delegator distribution (#27930) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-26 10:10:10 +08:00
yah01 a36a57bfec
Enable mmap for IVF index (#27904) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-26 01:48:12 +08:00
yah01 f79c7370f4
Fix panic while flushing dropped/compacted segment (#27927) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-25 22:10:13 +08:00
Gao c8415453a1
Support prepare config for index (#27920) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-10-25 19:54:12 +08:00
zhenshan.cao 6c3f29d003
Identify service providers based on addresses (#27907) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-10-25 17:28:10 +08:00
aoiasd 53246b1b38
Set accesslog default to close and use stdout (#27891) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-10-25 10:30:10 +08:00
wei liu e0222b2ce3
refine target manager code style (#27883) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-25 00:44:12 +08:00
jaime 4640928280
Fix initialization and backward compatibility issue for GRPC compression (#27894) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-24 22:14:14 +08:00
yihao.dai b9d5ef3599
Fix datanode ttNode goroutine leak (#27878) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-24 18:22:10 +08:00
Enwei Jiao 4faba61647
Upgrade conan to 1.61.0 (#27870) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-10-24 17:50:10 +08:00
jaime 6749957e71
Refine RPC call in unwatch drop channel (#27864) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-24 17:46:15 +08:00
congqixia 323fc107a7
Fix taskDispatcher add multiple tasks will ignore following ones (#27885) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-24 17:18:13 +08:00
aoiasd 9ba79a97e4
Add some log for delete task when it need query (#27317) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-10-24 14:54:12 +08:00
wei liu 178db7b0f0
check stopping node during start qc (#27859) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-24 12:20:11 +08:00
congqixia 93a877f55e
Make qcv2 target&leader observer execute in parallel (#27844) 
- Add `taskDispatcher` to submit and run task async safely
- Change `LeaderObeserver` and `TargetObserver` schedule and manual check action to submitting task into dispatcher
- Fix logic problem in collection observer when manual check return false

See also #27494

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-24 10:14:11 +08:00
sre-ci-robot 8c605ca858
Update knowhere commit (#27865) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-10-24 02:34:08 +08:00
Xiaofan 2ea7579dbb
Reduce rpc size for GetRecoveryInfoV2 (#27483) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-23 21:44:09 +08:00
zhagnlu 6060dd7ea8
Add chunk manager request timeout (#27692) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-10-23 20:08:08 +08:00
aoiasd 9091a27832
Add meta cache to datanode for L0 Delta (#27768) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-10-23 19:42:10 +08:00
jaime d2dbbbc11b
Reduce write lock scope in channel manager (#27823) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-21 07:58:16 +08:00
Jiquan Long babf4e37e4
Fix partition's gc (#27816) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-10-20 16:30:09 +08:00
SimFG 9b0ecbdca7
Support to replicate the mq message (#27240) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-20 14:26:09 +08:00
PowderLi dcb7a89601
[restful] new interface: upsert (#27787) 
interface: delete support expression

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-10-20 09:54:11 +08:00
wei liu eadd48ef7c
fix grpc error judge logic (#27800) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-20 04:28:09 +08:00
congqixia 49516d44b4
Add ctx parameter and log tracer for watch and selectNodes (#27809) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-20 04:22:11 +08:00
sre-ci-robot b6e07d6fe3
Update knowhere commit (#27812) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-10-20 03:54:09 +08:00