milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
18,382 Commits
139 Branches
115 Tags
576 MiB
Commit Graph

7728 Commits (e576271a240d5c8dce1e49dcbe15cf7f465c429b)

Author SHA1 Message Date
congqixia e4fdf5e68e
Refine offline segments logic in shard delegator (#28073) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-01 20:04:24 +08:00
groot abd5b199cc
Bulkinsert support pure list json (#27990) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-11-01 19:02:13 +08:00
PowderLi 0c0f012e03
add a custom http header: Accept-Type-Allow-Int64 (#27901) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-01 11:42:16 +08:00
wei liu 0c69f48ba4
refine stop order (#28016) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-01 11:40:15 +08:00
PowderLi 0252871d30
fix azure ListObjects (#27931) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-01 11:34:14 +08:00
aoiasd 2aa330146a
Delete by complex expression use partitionKey optimization (#27952) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-01 11:26:16 +08:00
Enwei Jiao 8ae9c947ae
Use OpenDAL to access object store (#25642) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-01 09:00:14 +08:00
cqy123456 4fbe3c9142
replace loaded binlog with binlog index for search performance (#27673) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-11-01 02:20:15 +08:00
KumaJie e88212ba4b
Add CSV file import function (#27149) 
Signed-off-by: kuma <675613722@qq.com>
Co-authored-by: kuma <675613722@qq.com>
 2023-10-31 22:47:23 +08:00
wei liu 0677d2623d
remove retry in getShards (#28011) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-31 16:16:14 +08:00
yah01 1b9b8a31a8
Fix QueryNode panic while upgrading (#28034) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 15:50:13 +08:00
yah01 ab6dbf7659
Limit max thread num for pool (#28018) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 13:00:13 +08:00
yah01 9658367a3c
Refine chunk manager errors (#27590) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 12:18:15 +08:00
Xiaofan 38aa53f523
Fix datacoord start timeout (#28026) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-31 10:04:12 +08:00
yah01 2af46d7333
Increase the ChunkManager request timeout (#28015) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 09:06:13 +08:00
congqixia 233bf90c55
Add SyncManager to replace flush manager (#27873) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-31 02:30:16 +08:00
Xiaofan 7c9d24cbb7
Remove Properties in rocksmq (#28000) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-31 00:44:14 +08:00
yah01 9b6eeb46f1
Cache segment row num, size, and insert count to reduce CGO calls (#28007) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-30 17:54:14 +08:00
yihao.dai ab6b0103a3
Get vector concurrently (#27838) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-30 15:44:12 +08:00
MrPresent-Han f93ad6471c
enable query reduce on querynodes for shortcut return(#27306) (#27837) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-10-30 15:08:11 +08:00
Ikko Eltociear Ashimine ed4f20b0ed
Fix typo in util.go (#27975) 
suppot -> support

Signed-off-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
 2023-10-30 14:40:27 +08:00
wei liu 7e0bda8b4c
remove binlog/delta log from getRecoveryInfoV2 (#27895) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-30 14:28:15 +08:00
congqixia 36ddeae2cc
Refine log level when request resource fail for loading segments (#28004) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-30 14:20:14 +08:00
Jiquan Long 6913283716
Fill output fields for count request (#27967) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-10-30 10:00:12 +08:00
congqixia 98e2aad752
Refine datanode metacache and implement CoW (#27985) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-28 11:12:11 +08:00
MrPresent-Han ccaec44930
enable parameter for dataCoord broker timeout(#26972) (#27825) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-10-28 06:14:11 +08:00
sre-ci-robot 1ae6e5d8c8
Update knowhere commit (#27993) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-10-28 04:24:10 +08:00
groot e6d4cde397
Fix duplicated segment id in bulkinsert task state (#27915) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-10-27 15:14:13 +08:00
yah01 f212158d61
Fix delete records timestamp may be reordered (#27941) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-27 10:24:10 +08:00
Enwei Jiao 4a33391b8f
rename createindex (#27903) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-10-27 10:12:14 +08:00
smellthemoon 403c6680cc
Add error description (#27959) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-10-27 10:10:13 +08:00
Filip Haltmayer 6b1a106a31
Moving etcd client into session (#27069) 
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
 2023-10-27 07:36:12 +08:00
congqixia 852be152de
Change task sourceID to stringer interface (#27965) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-27 01:08:12 +08:00
yihao.dai f9c630247d
Construct plan directly when search with vector output (#27928) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-26 19:30:10 +08:00
congqixia 13877a07ff
Add ctx parameter for tsafe pkg & NewDelegator method (#27877) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-26 19:14:10 +08:00
shaoyue 977e508a0f
Propagate task ctx done err for proxy (#27892) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-10-26 19:06:13 +08:00
jaime bf7f32b025
Refine Proxy stop (#27919) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-26 10:24:09 +08:00
wei liu 8041fc3c75
avoid add empty growing segment to delegator distribution (#27930) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-26 10:10:10 +08:00
yah01 a36a57bfec
Enable mmap for IVF index (#27904) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-26 01:48:12 +08:00
yah01 f79c7370f4
Fix panic while flushing dropped/compacted segment (#27927) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-25 22:10:13 +08:00
Gao c8415453a1
Support prepare config for index (#27920) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-10-25 19:54:12 +08:00
zhenshan.cao 6c3f29d003
Identify service providers based on addresses (#27907) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-10-25 17:28:10 +08:00
aoiasd 53246b1b38
Set accesslog default to close and use stdout (#27891) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-10-25 10:30:10 +08:00
wei liu e0222b2ce3
refine target manager code style (#27883) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-25 00:44:12 +08:00
jaime 4640928280
Fix initialization and backward compatibility issue for GRPC compression (#27894) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-24 22:14:14 +08:00
yihao.dai b9d5ef3599
Fix datanode ttNode goroutine leak (#27878) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-24 18:22:10 +08:00
Enwei Jiao 4faba61647
Upgrade conan to 1.61.0 (#27870) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-10-24 17:50:10 +08:00
jaime 6749957e71
Refine RPC call in unwatch drop channel (#27864) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-24 17:46:15 +08:00
congqixia 323fc107a7
Fix taskDispatcher add multiple tasks will ignore following ones (#27885) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-24 17:18:13 +08:00
aoiasd 9ba79a97e4
Add some log for delete task when it need query (#27317) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-10-24 14:54:12 +08:00
wei liu 178db7b0f0
check stopping node during start qc (#27859) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-24 12:20:11 +08:00
congqixia 93a877f55e
Make qcv2 target&leader observer execute in parallel (#27844) 
- Add `taskDispatcher` to submit and run task async safely
- Change `LeaderObeserver` and `TargetObserver` schedule and manual check action to submitting task into dispatcher
- Fix logic problem in collection observer when manual check return false

See also #27494

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-24 10:14:11 +08:00
sre-ci-robot 8c605ca858
Update knowhere commit (#27865) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-10-24 02:34:08 +08:00
Xiaofan 2ea7579dbb
Reduce rpc size for GetRecoveryInfoV2 (#27483) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-23 21:44:09 +08:00
zhagnlu 6060dd7ea8
Add chunk manager request timeout (#27692) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-10-23 20:08:08 +08:00
aoiasd 9091a27832
Add meta cache to datanode for L0 Delta (#27768) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-10-23 19:42:10 +08:00
jaime d2dbbbc11b
Reduce write lock scope in channel manager (#27823) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-21 07:58:16 +08:00
Jiquan Long babf4e37e4
Fix partition's gc (#27816) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-10-20 16:30:09 +08:00
SimFG 9b0ecbdca7
Support to replicate the mq message (#27240) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-20 14:26:09 +08:00
PowderLi dcb7a89601
[restful] new interface: upsert (#27787) 
interface: delete support expression

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-10-20 09:54:11 +08:00
wei liu eadd48ef7c
fix grpc error judge logic (#27800) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-20 04:28:09 +08:00
congqixia 49516d44b4
Add ctx parameter and log tracer for watch and selectNodes (#27809) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-20 04:22:11 +08:00
sre-ci-robot b6e07d6fe3
Update knowhere commit (#27812) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-10-20 03:54:09 +08:00
wei liu 55e5f80e24
update collection target after observer start (#27774) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-19 21:52:10 +08:00
Gao 9dd369dd99
Update knowhere version to v2.2.2 (#27810) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-10-19 21:34:07 +08:00
MrPresent-Han af61a80f73
connect traceID for retrieve (#27804) 
/kind improvement

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-10-19 19:22:10 +08:00
zhenshan.cao 020ad9a6bc
Rectify wrong exception messages associated with Array datatype (#27769) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-10-19 17:24:07 +08:00
yah01 635efdf170
Schedule loading L0 segments first (#27593) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-19 11:14:06 +08:00
congqixia bcbe98aba1
Add querynode client wrapper and avoid grpc in standalone mode (#27781) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-19 11:10:07 +08:00
smellthemoon 4b0ec156b3
Set channel work pool size in datanode (#27728) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-10-19 08:28:08 +08:00
yah01 9467de79c3
Use pool to block exceeded tasks (#27767) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-19 03:30:06 +08:00
jaime ac2d1bb5c2
Support receive signals from parent process (#27756) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-18 20:20:11 +08:00
zhenshan.cao c0edc22a6e
Limiting the idempotence check for index to user index params (#27615) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-10-18 20:18:24 +08:00
XuanYang-cn 7358c3527b
Add iterators (#27643) 
See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-10-18 19:34:08 +08:00
SimFG 630636c4ec
Support the apikey authentication for the restful api (#27758) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-18 16:36:12 +08:00
yihao.dai 49b3a12804
Return newly defined merr instead of grpc unimplemented err (#27751) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-18 15:32:11 +08:00
sre-ci-robot 6b79d2b7d6
Update knowhere commit (#27752) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-10-18 07:14:09 +08:00
Alexander Guzhva fbaec4c1a2
Fix non-removed 2 GB temporary file in test_utils.cpp (#27651) 
Signed-off-by: Alexandr Guzhva <alexanderguzhva@gmail.com>
 2023-10-18 02:24:09 +08:00
Enwei Jiao e98e56f75d
Fix SIGSEGV if run within gdb (#27736) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-10-18 02:16:11 +08:00
SimFG 84d05b939b
Support the apikey authentication (#27723) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-17 21:00:11 +08:00
congqixia 2f201c25e2
Remove deprecated io/ioutil usage (#27747) 
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
 2023-10-17 20:32:09 +08:00
PowderLi 09d8b76048
[restful] new context with grpc metadata (#27668) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-10-17 20:00:14 +08:00
congqixia 1f2a76d04d
Handle legacy querynode `Delete` Unimplemented (#27749) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-17 19:06:06 +08:00
XuanYang-cn 2f16339aac
Enhance InsertData and FieldData (#27436) 
1. Add NewInsertData
2. Add GetRowNum(), GetMemorySize(), and, Append() for InsertData
3. Add AppendRow() for FieldData for compaction

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-10-17 17:36:11 +08:00
congqixia efe8984eca
Fix Flush error message typo (#27729) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-17 17:18:09 +08:00
congqixia b7bfccaf21
Add channel name paramter in `FlushSegmentsRequest` (#27725) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-17 12:00:10 +08:00
Futao Wei 599012a340
Implement SearchByPks path for Search (#25882) 
Signed-off-by: unfode <forrest.futao.wei@gmail.com>
 2023-10-16 15:34:08 +08:00
wayblink e3f2122618
Expose metrics of stanby coordinators (#27698) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-10-16 15:04:09 +08:00
wei liu 7aa862c0ea
fix retry on unimplemented rpc error (#27639) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-16 10:50:09 +08:00
smellthemoon 5785756d31
Add some log to print size of search result (#27601) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-10-16 10:48:08 +08:00
jaime e386a62fae
Remove recollect segment stats during starting datacoord (#27410) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-16 10:26:09 +08:00
jaime ec1fe3549e
Add a stop hook to clean session (#27564) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-16 10:24:10 +08:00
Xiaofan 333a0c8a53
Fix RocksMQ seek speed (#27646) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-16 10:18:08 +08:00
yah01 50672f0d60
Skip empty segment while query/search (#27711) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-15 16:10:04 +08:00
xige-16 6cbb67832f
Compatible with scalar index types marisa-trie and Ascending (#27638) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-10-15 13:52:06 +08:00
zhagnlu c5ea31316d
Fix glog thread-safe bug for standalone mode (#27703) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-10-14 12:10:05 +08:00
sre-ci-robot 75343b2cb4
Update knowhere commit (#27706) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-10-14 03:03:37 +08:00
zhagnlu 2efc4ec703
Remove deprecated cmake parameter (#27695) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-10-14 02:57:35 +08:00
wei liu 310739d115
remove flush from ddl queue in proxy (#27688) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-13 18:33:36 +08:00
yihao.dai c882e9c87b
Fix compatibility of Flush (#27543) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-13 18:01:39 +08:00