milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
18,097 Commits
146 Branches
120 Tags
570 MiB
Commit Graph

18097 Commits (2b64cfc77209186a41f79edecb76e43825db2e00)

Author SHA1 Message Date
zhuwenxing 754500f1c8
[skip e2e]Increase the chaos duration for chaos test (#26990) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-09-11 17:57:18 +08:00
zhuwenxing 567fb23126
[test]Update health_checkers assertion for standby test (#26986) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-09-11 17:55:23 +08:00
congqixia c45c32fad4
Set task reason for collection released (#26962) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-10 15:15:17 +08:00
MrPresent-Han 2101f2d289
fix unstable checker id due to go map iteration(#26943) (#26944) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-09-10 10:11:16 +08:00
sre-ci-robot 49c8487986 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2023-09-10 00:01:20 +00:00
congqixia 2a5d574a0d
Fix querynodev2 concurrent load logic (#26959) 
Fix logic error from #26926
function `waitSegmentLoadDone` shall return error when context is done

Make delegator control concurrency for each same segment
Related to #26908

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-10 07:41:18 +08:00
Futao Wei ad1cc00a18
Fix typo in comment (#26951) 
Signed-off-by: unfode <forrest.futao.wei@gmail.com>
 2023-09-09 20:13:16 +08:00
sre-ci-robot e5dab53eb2 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2023-09-09 12:00:47 +00:00
congqixia c6116d1819
Remove segment to LocalSegment type assertion (#26931) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-09 10:35:16 +08:00
congqixia 758aad705d
Fix checker using default interval after manual check (#26953) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-09 08:29:16 +08:00
congqixia 44f308fff5
Handle nil status in `merr.OK` (#26940) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-08 18:43:16 +08:00
Jiquan Long 9dae896395
Fix indexnode log (#26927) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-08 16:57:16 +08:00
congqixia c8f9f22c4a
Fix segment loader return false success (#26926) 
`waitSegmentLoadDone` did not check waitCh result is success or failure
after load return without error, delegator will assume all segments are loaded

This PR changes waitCh to loadResult with `sync.Cond` with `atomic.Int32` to represent status

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-08 16:41:16 +08:00
yah01 22bb9c199b
Forbid returning merr directly (#26918) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-08 10:31:15 +08:00
zhuwenxing 0f4475e5e3
[test]Refine health checker in test (#26920) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-09-08 10:09:16 +08:00
Xu Tong 9166011c4a
Add float16 vector (#25852) 
Signed-off-by: Writer-X <1256866856@qq.com>
 2023-09-08 10:03:16 +08:00
sre-ci-robot eb3ea3a1df
Update Builder image changes (#26913) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-09-08 09:53:16 +08:00
XuanYang-cn 7f1ae35e72
Add timeout in dispatcher, AsConsumer and Seek (#26686) 
See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-08 09:51:17 +08:00
SimFG 0901b76732
Avoid the panic when the status of rpc response is nil (#26910) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-07 19:23:15 +08:00
zhuwenxing 3fd315f2b5
[skip e2e]Refine the script of getting image tag (#26919) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-09-07 19:05:15 +08:00
zhuwenxing e2528030bb
[skip e2e]Add python script to get image tag name (#26911) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-09-07 16:39:15 +08:00
Enwei Jiao 2a755baaa5
Fix amazonlinux cache not work (#26883) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-07 15:43:15 +08:00
congqixia 8dfa832fa0
Use same mergify rule for branch 2.3.0 (#26896) 
Since branch 2.3.0 share same action setup,
rules applied to master shall apply to branch 2.3.0 as well
For now, only branch 2.2.0 uses diferent set of rules

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-07 12:03:15 +08:00
congqixia af5c01082b
Refine delegator lifetime control (#26881) 
- Add SafeChan interface in lifetime package
- Embed SafeChan into  interface
- Replace private lifetime struct in delegator package with
- Refine delegator on-going task lifetime control and wait all accepted task done
- Fix potential goroutine leakage from  if delegator closed concurrently

/kind improvement

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-07 10:11:15 +08:00
sre-ci-robot 69bac68f8c
Update Pytest image changes (#26796) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-09-07 09:47:16 +08:00
nico 5e4ae757f5
update test cases (#26872) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2023-09-07 09:45:15 +08:00
sre-ci-robot 56a6559fe7
Update knowhere commit (#26888) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-09-07 09:19:16 +08:00
yiwangdr 337edc321b
tikv integration (#26246) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-09-07 07:25:14 +08:00
cai.zhang 8b5b137207
Should set index task retry not failed when task canceled (#26878) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-09-06 19:01:14 +08:00
congqixia 0844e19505
Fix mergify rules for unit test file changed under pkg folder (#26882) 
See #26865
 for only unit test file changed
removed due to exclude rule only apply to _test.go under internal/

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-06 18:33:14 +08:00
yah01 4a2377e669
Fix retrieving may return more result than limit (#26670) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-06 17:47:15 +08:00
SimFG 28681276e2
Improve the retry of the rpc client (#26795) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-06 17:43:14 +08:00
Jiquan Long 61c7b0990d
Workaround fix ChannelManager holding mutex too long (#26870) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-06 14:29:40 +08:00
zhagnlu 9b3e744b24
Fix embedded milvus log bug (#26864) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-09-06 14:27:14 +08:00
zhuwenxing bd4b842153
[test]Update chaos apply and chaos scope (#26873) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-09-06 14:03:14 +08:00
MrPresent-Han 528948559f
fix false load failure for long unserviable period(#26813) (#26818) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-09-06 12:57:15 +08:00
congqixia 7e2fa55440
Fix misuse of waitgroup in metrics mutex ut (#26865) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-06 12:39:14 +08:00
zhuwenxing 2f32623d16
[test]Fix occasional FileExistsError triggered when concurrently executing pytest (#26867) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-09-06 12:07:14 +08:00
wei liu 0e2085b77f
fix dc standby to active (#26810) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-09-06 10:41:49 +08:00
wayblink d7b6f3a9e8
Enhance FlushAll/GetFlushAllState API (#26802) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-09-06 10:35:48 +08:00
sre-ci-robot 78a2638fd4
Update knowhere commit (#26861) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-09-06 09:21:49 +08:00
sre-ci-robot f14dfde338 Update all contributors 
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
 2023-09-06 00:01:27 +00:00
bjzhjing 548c82eca5
Refactor storage.MergeInsertData() to optimize the merging process (#26839) 
Benchmark Milvus with https://github.com/qdrant/vector-db-benchmark and
specify the datasets as 'deep-image-96-angular'. Meanwhile, do perf
profiling during 'upload + index' stage of vector-db-benchmark and see
the following hot spots.

39.59%--github.com/milvus-io/milvus/internal/storage.MergeInsertData
        |
        |--21.43%--github.com/milvus-io/milvus/internal/storage.MergeFieldData
        |          |
        |          |--17.22%--runtime.memmove
        |                     |
        |                     |--1.53%--asm_exc_page_fault
        |                     ......
        |
        |--18.16%--runtime.memmove
                   |
                   |--1.66%--asm_exc_page_fault
                   ......

The hot code path is in storage.MergeInsertData() which updates
buffer.buffer by creating a new 'InsertData' instance and merging both
the old buffer.buffer and addedBuffer into it. When it calls golang
runtime.memmove to move buffer.buffer which is with big size (>1M), the
hot spots appear.

To avoid the above overhead, update storage.MergeInsertData() by
appending addedBuffer to buffer.buffer, instead of moving buffer.buffer
and addedBuffer to a new 'InsertData'. This change removes the hot spots
'runtime.memmove' from perf profiling output. Additionally, the 'upload
+ index' time, which is one performance metric of vector-db-benchmark,
is reduced around 60% with this change.

Signed-off-by: Cathy Zhang <cathy.zhang@intel.com>
 2023-09-05 21:41:48 +08:00
congqixia 9ed5841730
Use specified mockery & golangci-lint version (#26845) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-05 17:39:48 +08:00
Enwei Jiao 73f4347ac6
Fix code checker error (#26844) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 15:51:48 +08:00
zhuwenxing 1a80380a6a
[skip e2e]Add wait time when verifying the chaos is deleted (#26847) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2023-09-05 14:33:49 +08:00
congqixia fe7f7ea237
Add cpp-build and setenv for querynode mockery regen command (#26843) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-05 14:13:49 +08:00
congqixia 901e98a868
Update CONTRIBUTING.md for mockery generation (#26842) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-05 14:11:54 +08:00
Enwei Jiao fb0705df1b
Decouple basetable and componentparam (#26725) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 10:31:48 +08:00
nico f0c911afc2
add case about diskann and binary index (#26804) 
Signed-off-by: nico <cheng.yuan@zilliz.com>
 2023-09-05 10:23:48 +08:00