Commit Graph

398 Commits (4fc7915c70499c06e954a031dd1e9ad2cf2de853)

Author SHA1 Message Date
congqixia 244d2c04f6
feat: Add `milvusclient` package and migrate GoSDK (#32907)
Related to #31293

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-10 18:01:30 +08:00
aoiasd 54a51b1236
enhance: Support dynamic config for opentelemetry trace (#32169)
relate: https://github.com/milvus-io/milvus/issues/31940

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-05-09 17:43:30 +08:00
Aldrin b6c06ff2c3
fix: Removed tests on non existent directories and solved name resolution errors (#32266)
issue: #32254

Signed-off-by: Ald392 <imagesai32@gmail.com>
2024-04-30 11:33:26 +08:00
shaoting-huang f5e1fc757c
fix: [skip e2e] dev container yaml indentation error (#32601)
issue: #32594

Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>
2024-04-26 10:15:27 +08:00
pingliu 72b054e0c3
doc: change milvus docker image version to v2.4.0 (#32489)
change milvus docker image version to v2.4.0

Signed-off-by: ping.liu <ping.liu@zilliz.com>
2024-04-23 09:47:23 +08:00
Ted Xu 744a54a534
enhance: enforce milvus.yaml assertion in UT (#32357)
See #32168

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-04-19 16:47:20 +08:00
yiwangdr 1cd15d9322
test: support segment release in integration test (#31190)
issue: #29507

Notice that api_testonly.go files should be guarded by compiler tag
`test`, so that production build rules don't compile them and these APIs
don't get misused.

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-04-09 11:39:17 +08:00
congqixia f399416b92
enhance: Use gotestsum to run go unit test (#31622)
See also #31490

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-27 15:29:10 +08:00
congqixia 4da8b6607d
enhance: Add scripts to use `gotestsum` to execute integration test (#31490)
See also #31489

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-03-22 10:29:06 +08:00
pingliu d5ba221b25
doc: change milvus docker image version to v2.4.0-rc.1 (#31461)
Signed-off-by: ping.liu <ping.liu@zilliz.com>
2024-03-20 22:15:05 +08:00
pingliu 26539a1b6d
doc: change milvus docker image version to 2.3.12 (#31336)
Signed-off-by: ping.liu <ping.liu@zilliz.com>
2024-03-18 14:19:05 +08:00
Jiquan Long 375190e76e
fix: cpp format check not work (#30767)
fix: #30765

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-02-22 19:40:53 +08:00
zhagnlu 976b6fc0e4
enhance: change opendal as compile configurable (#30384)
#30373

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-02-20 19:16:52 +08:00
pingliu f4559cbe54
doc: change image version to 2.3.9 (#30695)
Signed-off-by: ping.liu <ping.liu@zilliz.com>
2024-02-20 18:52:52 +08:00
yiwangdr 32cff25f97
enhance: decrease coordinator init time (#29822)
This PR mainly improve two items:
1. Target observer should refresh loading status during init time. An
uninitialized loading status blocks search/query. Currently, the target
observer refreshes every 10 seconds, i.e. we'd need to wait for 10s for
no reason. That's also the reason why we constantly see false log
"collection unloaded" upon mixcoord restarts.
2. Delete session when service is stopped. So that the new service
doesn't need to wait for the previous session to expire (~10s).

Item 1 is the major improvement of this PR, which should speed up init
time by 10s.
Item 2 is not a big concern in most cases as coordinators usually shut
down after stop(). In those cases, coordinator restart triggers serverID
change which further triggers an existing logic that deletes expired
session. This PR only fixes rare cases where serverID doesn't change.

integration test:
`go test -tags dynamic -v -coverprofile=profile.out -covermode=atomic
tests/integration/coordrecovery/coord_recovery_test.go -timeout=20m`
Performance after the change:
Average init time of coordinators: 10s
Hardware: M2 Pro
Test setup: 1000 collections with 1000 rows (dim=128) per collection.


issue: #29409

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2024-02-05 14:00:12 +08:00
pingliu 5cdf0f2490
doc: fix standalone stop cannot start issue (#30481)
Signed-off-by: ping.liu <ping.liu@zilliz.com>
2024-02-02 21:05:05 +08:00
pingliu e4a033858e
doc: Add Milvus standalone all in one script (#30459)
Signed-off-by: ping.liu <ping.liu@zilliz.com>
2024-02-02 18:05:04 +08:00
yah01 ec688f5bf6
fix: failed to download OpenDAL (#30380)
OpenDAL's url has been changed
fix #30379

Signed-off-by: yah01 <yang.cen@zilliz.com>
2024-02-02 14:35:04 +08:00
sammy.huang 265453f400
enhance: [skip e2e]revert back to original way to archive, upload and download (#30248)
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
2024-02-01 10:15:04 +08:00
congqixia 9f8eb0e527
enhance: make integration test case timeout configurable (#30073)
currently integration test may timeout if any case run time is above 3
minutes. This duration was hard coded.

This PR change this duration into a customized parameter and could be
passed via test running commands.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-18 12:22:54 +08:00
yah01 031243fee7
feat: support mmap for marisa trie (#29613)
this supports mmap for marisa trie index
related https://github.com/milvus-io/milvus/issues/21866

Signed-off-by: yah01 <yang.cen@zilliz.com>
2024-01-11 10:22:50 +08:00
Jiquan Long 4b3de64733
enhance: add rust to install_dep.sh (#29586)
fix: #29585

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-01-07 20:04:12 +08:00
yihao.dai 3561586edf
feat: Add import reader for binlog (#28910)
This PR defines the new import reader interfaces and implement a binlog
reader for import.

issue: https://github.com/milvus-io/milvus/issues/28521

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-01-05 11:48:47 +08:00
PowderLi 5f00bad4b8
fix: link with install path's libblob-chunk-manager (#29496)
issue: #29494

1. link with install path's libblob-chunk-manager
2. performance of `ShouldBindWith` is better than `ShouldBindBodyWith`
3. the middleware shouldn't read the unrefreshed parameter repeatly

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-31 20:02:48 +08:00
wei liu e41fd6fbde
enhance: Move proxy client manager to util package (#28955)
issue:  #28898

This PR move the `ProxyClientManager` to util package, in case of
reusing it's implementation in querycoord

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-20 19:22:42 +08:00
zhagnlu 56d7225673
fix: Add LD_LIBRARY_PATH when local start milvus (#29287)
#28165

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-12-19 10:08:41 +08:00
zhagnlu a602171d06
enhance: Refactor runtime and expr framework (#28166)
#28165

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-12-18 12:04:42 +08:00
Jiquan Long ed79505d31
enhance: use specific rust version to compile opendal (#29106)
/kind improvement

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-13 16:34:38 +08:00
XuanYang-cn d2207cac27
fix: Fix rustup update (#29093)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-12-11 11:36:37 +08:00
Bingyi Sun 36f69ea031
feat: integrate storagev2 in building index of segcore (#28768)
issue: https://github.com/milvus-io/milvus/issues/28655

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-12-05 16:48:54 +08:00
congqixia e79c3eaa90
enhance: [skip e2e] Update install_deps.sh cmake version (#28946)
cmake version needs to be greater or equal than 3.26.4, sync
`install_deps.sh` scripts version to 3.26.5

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-04 18:26:34 +08:00
PowderLi cac802ef7f
enhance: use already installed vcpkg (#28703)
issue #28686
1. Update Builder gpu image changes, see changes #28505
2. update azure-identity-cpp from beta to release

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-30 15:58:32 +08:00
Xiaofan f13ef1665a
fix : macos clean compilation fail, brew install some missed library (#28717)
fix macOS compilation fail, related #28715
1) install_deps.sh missed to install some dependencies 
2) llvm can not be found without a soft link

Signed-off-by: xiaofan luan <xiaofanluan@xiaofandeMacBook-Pro.local>
Co-authored-by: xiaofan luan <xiaofanluan@xiaofandeMacBook-Pro.local>
2023-11-27 10:06:26 +08:00
yah01 da03feb79a
enhance: upgrade OpenDAL to v0.42.0 and optimize the building (#28556)
- Upgrade OpenDAL to v0.42.0
- Clone OpenDAL with depth 1
- Add build log for OpenDAL

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-20 14:58:20 +08:00
yah01 6f59510723
Upgrade go to 1.20 (#24390)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-10 07:46:18 +08:00
Enwei Jiao cbf25f8585
Use OpenDAL 0.42 (#28258)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-11-09 11:26:21 +08:00
PowderLi 9f9726f8b7
print azure sdk log (#28240)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-08 17:50:18 +08:00
PowderLi 7bb0fa9c70
reduce useless ObjectExists (#28156)
replace ListBlobs() with GetProperties()
unified style std::string& / char*
config azure requestTimeoutMs

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-07 16:32:20 +08:00
wei liu 0c69f48ba4
refine stop order (#28016)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-01 11:40:15 +08:00
Enwei Jiao 8ae9c947ae
Use OpenDAL to access object store (#25642)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-11-01 09:00:14 +08:00
XuanYang-cn 6ce98cf542
Fix sript stop unable to find milvus process (#27957)
/kind improvement

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-10-26 20:44:14 +08:00
Enwei Jiao 4faba61647
Upgrade conan to 1.61.0 (#27870)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-10-24 17:50:10 +08:00
jaime ac2d1bb5c2
Support receive signals from parent process (#27756)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-18 20:20:11 +08:00
chen a66d1cf4ff
remove unused code (#27134)
Signed-off-by: xiedeyantu <czjourney@163.com>
2023-10-16 23:06:08 +08:00
jaime ec1fe3549e
Add a stop hook to clean session (#27564)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-16 10:24:10 +08:00
BUPTAnderson ff19cb2719
Fix cannot find -latomic (#27598)
Signed-off-by: hg.anderson <yunlongkong@126.com>
2023-10-15 20:08:04 +08:00
zhagnlu 2efc4ec703
Remove deprecated cmake parameter (#27695)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-10-14 02:57:35 +08:00
Sheldon fc1d59f64b
only build cpu in default devcontainer (#27570)
now the default devcontainer will use gpubuilder, which is so big,
almost >8G

Signed-off-by: Sheldon <chuanfeng.liu@zilliz.com>
2023-10-13 14:23:36 +08:00
xige-16 bf46ffd6c4
Fix build with diskann failed (#27672)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-10-12 19:25:35 +08:00
Enwei Jiao 0f2f4a0a75
Remove useless parameters for Makefile (#27622)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-10-11 20:45:35 +08:00