mirror of https://github.com/milvus-io/milvus.git
Remove Ubuntu18.04 support (#24666)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>pull/24675/head
Enwei Jiao
GitHub
parent
fb66924c90
commit
c9d9940918
|
|
@ -24,7 +24,7 @@ jobs:
|
|||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
os: [ubuntu18.04, ubuntu20.04, centos7]
|
||||
os: [ubuntu20.04, centos7]
|
||||
env:
|
||||
OS_NAME: ${{ matrix.os }}
|
||||
steps:
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@
|
|||
|
||||
This document will help to set up your Milvus development environment and to run tests. Please [file an issue](https://github.com/milvus-io/milvus/issues/new/choose) if there's a problem.
|
||||
|
||||
Table of contents
|
||||
=================
|
||||
# Table of contents
|
||||
|
||||
- [Development](#development)
|
||||
- [Table of contents](#table-of-contents)
|
||||
- [Building Milvus with Docker](#building-milvus-with-docker)
|
||||
|
|
@ -28,7 +28,6 @@ Table of contents
|
|||
- [GitHub Flow](#github-flow)
|
||||
- [FAQs](#faqs)
|
||||
|
||||
|
||||
## Building Milvus with Docker
|
||||
|
||||
Our official Milvus versions are releases as Docker images. To build Milvus Docker on your own, please follow [these instructions](https://github.com/milvus-io/milvus/blob/master/build/README.md).
|
||||
|
|
@ -52,13 +51,15 @@ All Linux distributions are available for Milvus development. However a majority
|
|||
|
||||
Here's a list of verified OS types where Milvus can successfully build and run:
|
||||
|
||||
* Debian/Ubuntu
|
||||
* CentOS
|
||||
* MacOS (x86_64)
|
||||
* MacOS (Apple Silicon)
|
||||
- Debian/Ubuntu
|
||||
- CentOS
|
||||
- MacOS (x86_64)
|
||||
- MacOS (Apple Silicon)
|
||||
|
||||
#### Prerequisites
|
||||
Linux systems (Recommend Ubuntu 18.04 or later):
|
||||
|
||||
Linux systems (Recommend Ubuntu 20.04 or later):
|
||||
|
||||
```bash
|
||||
go: >= 1.18
|
||||
cmake: >= 3.18
|
||||
|
|
@ -66,6 +67,7 @@ gcc: 7.5
|
|||
```
|
||||
|
||||
MacOS systems with x86_64 (Big Sur 11.5 or later recommended):
|
||||
|
||||
```bash
|
||||
go: >= 1.18
|
||||
cmake: >= 3.18
|
||||
|
|
@ -73,22 +75,24 @@ llvm: >= 15
|
|||
```
|
||||
|
||||
MacOS systems with Apple Silicon (Monterey 12.0.1 or later recommended):
|
||||
|
||||
```bash
|
||||
go: >= 1.18 (Arch=ARM64)
|
||||
cmake: >= 3.18
|
||||
llvm: >= 15
|
||||
```
|
||||
|
||||
|
||||
#### Installing Dependencies
|
||||
|
||||
In the Milvus repository root, simply run:
|
||||
|
||||
```bash
|
||||
$ ./scripts/install_deps.sh
|
||||
```
|
||||
|
||||
#### Caveats
|
||||
* [Google Test](https://github.com/google/googletest.git) is automatically cloned from GitHub, which in some case could conflict with your local google test library.
|
||||
|
||||
- [Google Test](https://github.com/google/googletest.git) is automatically cloned from GitHub, which in some case could conflict with your local google test library.
|
||||
|
||||
Once you have finished, confirm that `gcc` and `make` are installed:
|
||||
|
||||
|
|
@ -106,15 +110,17 @@ Confirm that cmake is available:
|
|||
```shell
|
||||
$ cmake --version
|
||||
```
|
||||
|
||||
Note: 3.25 or higher cmake version is required to build Milvus.
|
||||
|
||||
Milvus uses Conan to manage third-party dependencies for c++.
|
||||
|
||||
Install Conan
|
||||
Install Conan
|
||||
|
||||
```shell
|
||||
pip install conan==1.58.0
|
||||
```
|
||||
|
||||
Note: Conan version 2.x is not currently supported, please use version 1.58.
|
||||
|
||||
#### Go
|
||||
|
|
@ -126,14 +132,15 @@ Confirm that your `GOPATH` and `GOBIN` environment variables are correctly set a
|
|||
```shell
|
||||
$ go version
|
||||
```
|
||||
|
||||
Note: go >= 1.18 is required to build Milvus.
|
||||
|
||||
#### Docker & Docker Compose
|
||||
|
||||
Milvus depends on etcd, Pulsar and MinIO. Using Docker Compose to manage these is an easy way in local development. To install Docker and Docker Compose in your development environment, follow the instructions from the Docker website below:
|
||||
|
||||
- Docker: https://docs.docker.com/get-docker/
|
||||
- Docker Compose: https://docs.docker.com/compose/install/
|
||||
- Docker: https://docs.docker.com/get-docker/
|
||||
- Docker Compose: https://docs.docker.com/compose/install/
|
||||
|
||||
### Building Milvus
|
||||
|
||||
|
|
@ -146,6 +153,7 @@ $ make
|
|||
If this command succeed, you will now have an executable at `bin/milvus` off of your Milvus project directory.
|
||||
|
||||
If you want to update proto file before `make`, we can use the following command:
|
||||
|
||||
```shell
|
||||
$ make generated-proto-go
|
||||
```
|
||||
|
|
@ -171,35 +179,43 @@ It is required that all pull request candidates should pass all Milvus unit test
|
|||
Beforce running unit tests, you need to first bring up the Milvus deployment environment.
|
||||
You may set up a local docker environment with our docker compose yaml file to start unit testing.
|
||||
For Apple Silicon users (Apple M1):
|
||||
|
||||
```shell
|
||||
$ cd deployments/docker/dev
|
||||
$ docker-compose -f docker-compose-apple-silicon.yml up -d
|
||||
$ cd ../../../
|
||||
$ make unittest
|
||||
```
|
||||
|
||||
For others:
|
||||
|
||||
```shell
|
||||
$ cd deployments/docker/dev
|
||||
$ docker-compose up -d
|
||||
$ cd ../../../
|
||||
$ make unittest
|
||||
```
|
||||
|
||||
To run only cpp test:
|
||||
|
||||
```shell
|
||||
$ make test-cpp
|
||||
```
|
||||
|
||||
To run only go test:
|
||||
|
||||
```shell
|
||||
$ make test-go
|
||||
```
|
||||
|
||||
To run a single test case (TestSearchTask in /internal/proxy directory, for example):
|
||||
|
||||
```shell
|
||||
$ source scripts/setenv.sh && go test -v ./internal/proxy/ -test.run TestSearchTask
|
||||
```
|
||||
|
||||
If using Mac with M1 chip
|
||||
|
||||
```
|
||||
$ source scripts/setenv.sh && go test -tags=dynamic -v ./internal/proxy/ -test.run TestSearchTask
|
||||
```
|
||||
|
|
@ -209,18 +225,23 @@ $ source scripts/setenv.sh && go test -tags=dynamic -v ./internal/proxy/ -test.r
|
|||
Before submitting your pull request, make sure your code change is covered by unit test. Use the following commands to check code coverage rate:
|
||||
|
||||
Run unit test and generate code coverage report:
|
||||
|
||||
```shell
|
||||
$ make codecov
|
||||
```
|
||||
|
||||
This command will generate html report for Golang and C++ respectively.
|
||||
For Golang report, open the `go_coverage.html` under milvus project path.
|
||||
For C++ report, open the `cpp_coverage/index.html` under milvus project path.
|
||||
|
||||
You also can generate Golang coverage report by:
|
||||
|
||||
```shell
|
||||
$ make codecov-go
|
||||
```
|
||||
|
||||
Or C++ coverage report by:
|
||||
|
||||
```shell
|
||||
$ make codecov-cpp
|
||||
```
|
||||
|
|
@ -252,7 +273,9 @@ $ pytest --tags=L0 -n auto
|
|||
```
|
||||
|
||||
### Test on local branch
|
||||
|
||||
#### With Linux and MacOS
|
||||
|
||||
After preparing deployment environment, we can start the cluster on your host machine
|
||||
|
||||
```shell
|
||||
|
|
@ -260,7 +283,9 @@ $ ./scripts/start_cluster.sh
|
|||
```
|
||||
|
||||
#### With docker
|
||||
|
||||
start the cluster on your host machine
|
||||
|
||||
```shell
|
||||
$ ./build/builder.sh make install // build milvus
|
||||
$ ./build/build_image.sh // build milvus lastest docker image
|
||||
|
|
@ -270,33 +295,38 @@ milvusdb/milvus latest 63c62ff7c1b7 52
|
|||
$ install with docker compose
|
||||
```
|
||||
|
||||
|
||||
## GitHub Flow
|
||||
|
||||
To check out code to work on, please refer to the [GitHub Flow](https://guides.github.com/introduction/flow/).
|
||||
|
||||
## FAQs
|
||||
|
||||
Q: The go building phase fails on Apple Silicon (Mac M1) machines.
|
||||
|
||||
A: Please double-check that you have [right Go version](https://go.dev/dl/) installed, i.e. with OS=macOS and Arch=ARM64.
|
||||
|
||||
---
|
||||
Q: "make" fails with "*ld: library not found for -lSystem*" on MacOS.
|
||||
|
||||
Q: "make" fails with "_ld: library not found for -lSystem_" on MacOS.
|
||||
|
||||
A: There are a couple of things you could try:
|
||||
|
||||
1. Use **Software Update** (from **About this Mac** -> **Overview**) to install updates.
|
||||
2. Try the following commands:
|
||||
|
||||
```bash
|
||||
sudo rm -rf /Library/Developer/CommandLineTools
|
||||
sudo xcode-select --install
|
||||
```
|
||||
|
||||
---
|
||||
Q: Rocksdb fails to compile with "*ld: warning: object file was built for newer macOS version (11.6) than being linked (11.0).*" on MacOS.
|
||||
|
||||
Q: Rocksdb fails to compile with "_ld: warning: object file was built for newer macOS version (11.6) than being linked (11.0)._" on MacOS.
|
||||
|
||||
A: Use **Software Update** (from **About this Mac** -> **Overview**) to install updates.
|
||||
|
||||
---
|
||||
|
||||
Q: Some Go unit tests failed.
|
||||
|
||||
A: We are aware that some tests can be flaky occasionally. If there's something you believe is abnormal (i.e. tests that fail every single time). You are more than welcome to [file an issue](https://github.com/milvus-io/milvus/issues/new/choose)!
|
||||
|
|
|
|||
|
|
@ -3,10 +3,12 @@
|
|||
Building Milvus is easy if you take advantage of the containerized build environment. This document will guide you through this build process.
|
||||
|
||||
1. Docker, using one of the following configurations:
|
||||
* **macOS** Install Docker for Mac. See installation instructions [here](https://docs.docker.com/docker-for-mac/).
|
||||
**Note**: You will want to set the Docker VM to have at least 2 vCPU and 8GB of initial memory or building will likely fail.
|
||||
* **Linux with local Docker** Install Docker according to the [instructions](https://docs.docker.com/installation/#installation) for your OS.
|
||||
* **Windows with Docker Desktop WSL2 backend** Install Docker according to the [instructions](https://docs.docker.com/desktop/windows/wsl/). Be sure to store your sources in the local Linux file system, not the Windows remote mount at `/mnt/c`.
|
||||
|
||||
- **macOS** Install Docker for Mac. See installation instructions [here](https://docs.docker.com/docker-for-mac/).
|
||||
**Note**: You will want to set the Docker VM to have at least 2 vCPU and 8GB of initial memory or building will likely fail.
|
||||
- **Linux with local Docker** Install Docker according to the [instructions](https://docs.docker.com/installation/#installation) for your OS.
|
||||
- **Windows with Docker Desktop WSL2 backend** Install Docker according to the [instructions](https://docs.docker.com/desktop/windows/wsl/). Be sure to store your sources in the local Linux file system, not the Windows remote mount at `/mnt/c`.
|
||||
|
||||
2. **Optional** [Google Cloud SDK](https://developers.google.com/cloud/sdk/)
|
||||
|
||||
You must install and configure Google Cloud SDK if you want to upload your release to Google Cloud Storage and may safely omit this otherwise.
|
||||
|
|
@ -15,7 +17,6 @@ You must install and configure Google Cloud SDK if you want to upload your relea
|
|||
|
||||
While it is possible to build Milvus using a local golang installation, we have a build process that runs in a Docker container. This simplifies initial set up and provides a very consistent build and test environment.
|
||||
|
||||
|
||||
## Before You Begin
|
||||
|
||||
Before building Milvus, you must check the eligibility of Docker, Docker Compose, and hardware in line with Milvus' requirements.
|
||||
|
|
@ -36,7 +37,6 @@ See <a href="https://docs.docker.com/compose/install/">Install Docker Compose</a
|
|||
|
||||
</details>
|
||||
|
||||
|
||||
<details><summary>Check whether your CPU supports SIMD extension instruction set</summary>
|
||||
|
||||
Milvus' computing operations depend on CPU’s support for SIMD (Single Instruction, Multiple Data) extension instruction set. Whether your CPU supports SIMD extension instruction set is crucial to index building and vector similarity search within Milvus. Ensure that your CPU supports at least one of the following SIMD instruction sets:
|
||||
|
|
@ -53,20 +53,20 @@ lscpu | grep -e sse4_2 -e avx -e avx2 -e avx512
|
|||
```
|
||||
|
||||
Check Wikipedia [CPU with AVX](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX) for more details.
|
||||
</details>
|
||||
|
||||
</details>
|
||||
|
||||
## Key scripts
|
||||
|
||||
The following scripts are found in the [`build/`](.) directory. Note that all scripts must be run from the Milvus root directory.
|
||||
|
||||
* [`build/builder.sh`](builder.sh): Run a command in a build docker container. Common invocations are:
|
||||
* `build/builder.sh make`: Build just linux binary in the container. Pass options and packages as necessary.
|
||||
* `build/builder.sh make verifiers`: Run all pre-submission verification check.
|
||||
* `build/builder.sh make unittest`: Run all unit tests.
|
||||
* `build/builder.sh make clean`: Clean up all the generated files.
|
||||
- [`build/builder.sh`](builder.sh): Run a command in a build docker container. Common invocations are:
|
||||
- `build/builder.sh make`: Build just linux binary in the container. Pass options and packages as necessary.
|
||||
- `build/builder.sh make verifiers`: Run all pre-submission verification check.
|
||||
- `build/builder.sh make unittest`: Run all unit tests.
|
||||
- `build/builder.sh make clean`: Clean up all the generated files.
|
||||
|
||||
You can specify different OS for builder by setting `OS_NAME` which defaults to `ubuntu20.04`. Valid OS are `ubuntu20.04`,`ubuntu18.04`, `centos7`.
|
||||
You can specify different OS for builder by setting `OS_NAME` which defaults to `ubuntu20.04`. Valid OS are `ubuntu20.04`, `centos7`.
|
||||
|
||||
To specify `centos7` builder, use these commands:
|
||||
|
||||
|
|
@ -76,6 +76,7 @@ build/builder.sh make
|
|||
```
|
||||
|
||||
## Dev Containers
|
||||
|
||||
You can also get into the dev containers for development.
|
||||
|
||||
Enter root path of Milvus project on your host machine, execute the following commands:
|
||||
|
|
@ -122,7 +123,7 @@ make milvus
|
|||
make unittest
|
||||
```
|
||||
|
||||
Stop Dev Container
|
||||
Stop Dev Container
|
||||
|
||||
```shell
|
||||
./scripts/devcontainer.sh down
|
||||
|
|
@ -138,7 +139,9 @@ docker-compose up -d
|
|||
cd ../../../
|
||||
build/builder.sh /bin/bash -c "export ROCKSMQ_PATH='/tmp/milvus/rdb_data' && ./scripts/start_standalone.sh && cat"
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```shell
|
||||
build/builder.sh /bin/bash -c "./scripts/start_cluster.sh && cat"
|
||||
```
|
||||
|
|
@ -151,10 +154,9 @@ cd tests/docker
|
|||
docker-compose run --rm pytest /bin/bash -c "pytest --host ${MILVUS_SERVICE_IP}"
|
||||
```
|
||||
|
||||
|
||||
## Basic Flow
|
||||
|
||||
The scripts under [`build/`](.) are used to build and test. They will ensure that the `builder` Docker image is built (based on [`build/docker/builder`] ) and then execute the appropriate command in that container. These scripts will both ensure that the right data is cached from run to run for incremental builds and will copy the results back out of the container. You can specify a different registry/name for `builder` by setting `IMAGE_REPO` which defaults to `milvusdb`.
|
||||
The scripts under [`build/`](.) are used to build and test. They will ensure that the `builder` Docker image is built (based on [`build/docker/builder`] ) and then execute the appropriate command in that container. These scripts will both ensure that the right data is cached from run to run for incremental builds and will copy the results back out of the container. You can specify a different registry/name for `builder` by setting `IMAGE_REPO` which defaults to `milvusdb`.
|
||||
|
||||
The `builder.sh` is executed by first creating a “docker volume“ directory in `.docker/`. The `.docker/` directory is used to cache the third-party package and compiler cache data. It speeds up recompilation by caching previous compilations and detecting when the same compilation is being done again.
|
||||
|
||||
|
|
@ -162,11 +164,11 @@ The `builder.sh` is executed by first creating a “docker volume“ directory i
|
|||
|
||||
### Integrate vscode with docker
|
||||
|
||||
* Install [Visual Studio Code](https://code.visualstudio.com/)
|
||||
- Install [Visual Studio Code](https://code.visualstudio.com/)
|
||||
|
||||
* Install [Remote Development extension pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack)
|
||||
- Install [Remote Development extension pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack)
|
||||
|
||||
* Integrate with VS Code
|
||||
- Integrate with VS Code
|
||||
|
||||
The working principle is as follows: mount the local file system to the workspace inside the container, or copy it to the container. The extension of vs code is installed inside the container and runs in it, so that the vs Code of the host can fully access the tools, platforms and file systems inside the container. This means that you just need to connect to different containers to switch the entire development environment seamlessly.
|
||||
|
||||
|
|
@ -174,18 +176,15 @@ The working principle is as follows: mount the local file system to the workspac
|
|||
|
||||
Taking the Milvus project as an example, there is a file named **.devcontainer.json** in the root directory of the project. This file describes how vs code accesses (or creates) a development container environment, and defines the container environment, working directory, extension tool set, etc.
|
||||
|
||||
* The steps to configure the development environment are as follows:
|
||||
- The steps to configure the development environment are as follows:
|
||||
|
||||
Start VS Code，in the command panel ( F1 ) input **“Remote-Containers: Open Folder in Container”** , then select the project folder which contains devcontainer.json file.
|
||||
|
||||
or click right-bottom corner button > < , choose **“Remote-Containers: Open Folder in Container”**，then select the project folder which contains devcontainer.json file.
|
||||
|
||||
or click right-bottom corner button > < , choose **“Remote-Containers: Open Folder in Container”**，then select the project folder which contains devcontainer.json file.
|
||||
|
||||
|
||||
|
||||
|
||||
VS Code begin load and construct Devcontainer, the progress bar display the construction state.
|
||||
|
||||
VS Code begin load and construct Devcontainer, the progress bar display the construction state.
|
||||
|
||||
|
||||
|
||||
|
|
@ -195,7 +194,7 @@ You can also use terminal of VS Code to enter the Dev container to do something.
|
|||
|
||||
|
||||
|
||||
Modify vscode go setups if necessary, the setting path is **code -> preference -> settings**
|
||||
Modify vscode go setups if necessary, the setting path is **code -> preference -> settings**
|
||||
|
||||
```shell
|
||||
"go.testFlags": ["-v"] //if you want say detailed output when running unit test
|
||||
|
|
@ -206,5 +205,5 @@ Modify vscode go setups if necessary, the setting path is **code -> preference -
|
|||
|
||||
|
||||
### Integrate goland with docker
|
||||
TBD
|
||||
|
||||
TBD
|
||||
|
|
|
|||
|
|
@ -1,64 +0,0 @@
|
|||
# Copyright (C) 2019-2020 Zilliz. All rights reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software distributed under the License
|
||||
# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
|
||||
# or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
|
||||
FROM milvusdb/openblas:ubuntu18.04-20210428
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends wget curl ca-certificates gnupg2 && \
|
||||
wget -qO- "https://cmake.org/files/v3.24/cmake-3.24.0-linux-x86_64.tar.gz" | tar --strip-components=1 -xz -C /usr/local && \
|
||||
apt-get update && apt-get install -y --no-install-recommends \
|
||||
g++ gcc gfortran git make ccache libssl-dev zlib1g-dev unzip \
|
||||
clang-format-10 clang-tidy-10 lcov libtool m4 autoconf automake python3 python3-setuptools python3-pip \
|
||||
pkg-config uuid-dev libaio-dev libgoogle-perftools-dev && \
|
||||
apt-get remove --purge -y && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN pip3 install conan==1.58.0
|
||||
|
||||
# Install Go
|
||||
ENV GOPATH /go
|
||||
ENV GOROOT /usr/local/go
|
||||
ENV GO111MODULE on
|
||||
ENV PATH $GOPATH/bin:$GOROOT/bin:$PATH
|
||||
RUN mkdir -p /usr/local/go && wget -qO- "https://golang.org/dl/go1.20.4.linux-amd64.tar.gz" | tar --strip-components=1 -xz -C /usr/local/go && \
|
||||
mkdir -p "$GOPATH/src" "$GOPATH/bin" && \
|
||||
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b ${GOPATH}/bin v1.46.2 && \
|
||||
# export GO111MODULE=on && go get github.com/quasilyte/go-ruleguard/cmd/ruleguard@v0.2.1 && \
|
||||
go install github.com/ramya-rao-a/go-outline@latest && \
|
||||
go install golang.org/x/tools/gopls@latest && \
|
||||
go install github.com/uudashr/gopkgs/v2/cmd/gopkgs@latest && \
|
||||
go install github.com/go-delve/delve/cmd/dlv@latest && \
|
||||
go install honnef.co/go/tools/cmd/staticcheck@2022.1 && \
|
||||
go clean --modcache && \
|
||||
chmod -R 777 "$GOPATH" && chmod -R a+w $(go env GOTOOLDIR)
|
||||
|
||||
RUN ln -s /go/bin/dlv /go/bin/dlv-dap
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
gdb gdbserver && \
|
||||
apt-get remove --purge -y && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN echo 'root:root' | chpasswd
|
||||
|
||||
# refer: https://code.visualstudio.com/docs/remote/containers-advanced#_avoiding-extension-reinstalls-on-container-rebuild
|
||||
RUN mkdir -p /home/milvus/.vscode-server/extensions \
|
||||
/home/milvus/.vscode-server-insiders/extensions \
|
||||
&& chmod -R 777 /home/milvus
|
||||
|
||||
COPY --chown=0:0 build/docker/builder/entrypoint.sh /
|
||||
|
||||
RUN wget -qO- "https://github.com/jeffoverflow/autouseradd/releases/download/1.2.0/autouseradd-1.2.0-amd64.tar.gz" | tar xz -C / --strip-components 1
|
||||
|
||||
RUN wget -O /tini https://github.com/krallin/tini/releases/download/v0.19.0/tini && \
|
||||
chmod +x /tini
|
||||
|
||||
ENTRYPOINT [ "/tini", "--", "autouseradd", "--user", "milvus", "--", "/entrypoint.sh" ]
|
||||
CMD ["tail", "-f", "/dev/null"]
|
||||
|
|
@ -9,7 +9,7 @@
|
|||
# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
|
||||
# or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
|
||||
FROM milvusdb/openblas:ubuntu18.04-20210428 AS build
|
||||
FROM milvusdb/openblas:ubuntu20.04-20220914-179ea77 AS build
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends wget curl ca-certificates gnupg2 && \
|
||||
wget -qO- "https://cmake.org/files/v3.18/cmake-3.18.6-Linux-x86_64.tar.gz" | tar --strip-components=1 -xz -C /usr/local && \
|
||||
|
|
|
|||
|
|
@ -1,42 +0,0 @@
|
|||
# Copyright (C) 2019-2022 Zilliz. All rights reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
|
||||
# with the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software distributed under the License
|
||||
# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
|
||||
# or implied. See the License for the specific language governing permissions and limitations under the License.
|
||||
|
||||
FROM milvusdb/openblas:ubuntu18.04-20210428 AS openblas
|
||||
|
||||
#FROM alpine
|
||||
FROM ubuntu:bionic-20200921
|
||||
|
||||
RUN apt-get update && \
|
||||
apt-get install -y --no-install-recommends libtbb-dev libzstd-dev gfortran netcat iputils-ping ca-certificates uuid-dev libaio-dev libboost-program-options-dev && \
|
||||
apt-get remove --purge -y && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY --from=openblas /usr/lib/libopenblas-r0.3.9.so /usr/lib/
|
||||
|
||||
RUN ln -s /usr/lib/libopenblas-r0.3.9.so /usr/lib/libopenblas.so.0 && \
|
||||
ln -s /usr/lib/libopenblas.so.0 /usr/lib/libopenblas.so
|
||||
|
||||
COPY ./bin/ /milvus/bin/
|
||||
|
||||
COPY ./configs/ /milvus/configs/
|
||||
|
||||
COPY ./lib/ /milvus/lib/
|
||||
|
||||
ENV PATH=/milvus/bin:$PATH
|
||||
ENV LD_LIBRARY_PATH=/milvus/lib:$LD_LIBRARY_PATH:/usr/lib
|
||||
ENV LD_PRELOAD=/milvus/lib/libjemalloc.so
|
||||
|
||||
# Add Tini
|
||||
ADD https://github.com/krallin/tini/releases/download/v0.19.0/tini /tini
|
||||
RUN chmod +x /tini
|
||||
ENTRYPOINT ["/tini", "--"]
|
||||
|
||||
WORKDIR /milvus/
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
FROM ubuntu:bionic-20200921
|
||||
|
||||
# pipefail is enabled for proper error detection in the `wget | apt-key add`
|
||||
# step
|
||||
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
|
||||
|
||||
ENV DEBIAN_FRONTEND noninteractive
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends wget ca-certificates gnupg2 && \
|
||||
apt-get update && apt-get install -y --no-install-recommends g++ gcc gfortran git make && \
|
||||
apt-get remove --purge -y && \
|
||||
rm -rf /var/lib/apt/lists/* && \
|
||||
wget https://github.com/xianyi/OpenBLAS/archive/v0.3.9.tar.gz && \
|
||||
tar zxvf v0.3.9.tar.gz && cd OpenBLAS-0.3.9 && \
|
||||
make TARGET=CORE2 DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 USE_THREAD=0 USE_OPENMP=0 FC=gfortran CC=gcc COMMON_OPT="-O3 -g -fPIC" FCOMMON_OPT="-O3 -g -fPIC -frecursive" NMAX="NUM_THREADS=128" LIBPREFIX="libopenblas" INTERFACE64=0 NO_STATIC=1 && \
|
||||
make -j4 PREFIX=/usr NO_STATIC=1 install && \
|
||||
cd .. && rm -rf OpenBLAS-0.3.9 && rm v0.3.9.tar.gz
|
||||
|
||||
ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib"
|
||||
|
|
@ -3,7 +3,7 @@
|
|||
## Environment
|
||||
|
||||
```
|
||||
OS: Ubuntu 18.04
|
||||
OS: Ubuntu 20.04
|
||||
go：1.18
|
||||
cmake: >=3.18
|
||||
gcc： 7.5
|
||||
|
|
@ -12,6 +12,7 @@ gcc： 7.5
|
|||
## Install dependencies
|
||||
|
||||
Install compile dependencies
|
||||
|
||||
```shell
|
||||
$ sudo apt install -y g++ gcc make libssl-dev zlib1g-dev libboost-regex-dev \
|
||||
libboost-program-options-dev libboost-system-dev libboost-filesystem-dev \
|
||||
|
|
@ -52,6 +53,7 @@ $ make milvus
|
|||
## Install docker-compose
|
||||
|
||||
refer: https://docs.docker.com/compose/install/
|
||||
|
||||
```shell
|
||||
$ sudo curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
|
||||
$ sudo chmod +x /usr/local/bin/docker-compose
|
||||
|
|
@ -62,12 +64,14 @@ $ docker-compose --version
|
|||
## Start service
|
||||
|
||||
Start third-party service:
|
||||
|
||||
```shell
|
||||
$ cd [milvus project path]/deployments/docker/cluster
|
||||
$ docker-compose up -d
|
||||
```
|
||||
|
||||
Start milvus cluster:
|
||||
|
||||
```shell
|
||||
$ cd [milvus project path]
|
||||
$ ./scripts/start_cluster.sh
|
||||
|
|
@ -76,16 +80,19 @@ $ ./scripts/start_cluster.sh
|
|||
## Run unittest
|
||||
|
||||
Run all unittest including go and cpp cases:
|
||||
|
||||
```shell
|
||||
$ make unittest
|
||||
```
|
||||
|
||||
You also can run go unittest only:
|
||||
|
||||
```shell
|
||||
$ make test-go
|
||||
```
|
||||
|
||||
Run cpp unittest only:
|
||||
|
||||
```shell
|
||||
$ make test-cpp
|
||||
```
|
||||
|
|
@ -93,16 +100,19 @@ $ make test-cpp
|
|||
## Run code coverage
|
||||
|
||||
Run code coverage including go and cpp:
|
||||
|
||||
```shell
|
||||
$ make codecov
|
||||
```
|
||||
|
||||
You also can run go code coverage only:
|
||||
|
||||
```shell
|
||||
$ make codecov-go
|
||||
```
|
||||
|
||||
Run cpp code coverage only:
|
||||
|
||||
```shell
|
||||
$ make codecov-cpp
|
||||
```
|
||||
|
|
|
|||
|
|
@ -18,16 +18,16 @@
|
|||
|
||||
function install_linux_deps() {
|
||||
if [[ -x "$(command -v apt)" ]]; then
|
||||
# for Ubuntu 18.04 & 20.04
|
||||
# for Ubuntu 20.04
|
||||
sudo apt install -y wget curl ca-certificates gnupg2 \
|
||||
g++ gcc gfortran git make ccache libssl-dev zlib1g-dev unzip \
|
||||
clang-format-10 clang-tidy-10 lcov libtool m4 autoconf automake python3 python3-pip \
|
||||
pkg-config uuid-dev libaio-dev libgoogle-perftools-dev
|
||||
pkg-config uuid-dev libaio-dev libgoogle-perftools-dev
|
||||
|
||||
sudo pip3 install conan==1.58.0
|
||||
elif [[ -x "$(command -v yum)" ]]; then
|
||||
# for CentOS devtoolset-11
|
||||
sudo yum install -y epel-release centos-release-scl-rh
|
||||
sudo yum install -y epel-release centos-release-scl-rh
|
||||
sudo yum install -y wget curl which \
|
||||
git make automake python3-devel \
|
||||
devtoolset-11-gcc devtoolset-11-gcc-c++ devtoolset-11-gcc-gfortran \
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@
|
|||
|
||||
function install_linux_deps() {
|
||||
if [[ -x "$(command -v apt)" ]]; then
|
||||
# for Ubuntu 18.04
|
||||
# for Ubuntu 20.04
|
||||
sudo apt install -y g++ gcc make ccache libssl-dev zlib1g-dev libboost-regex-dev \
|
||||
libboost-program-options-dev libboost-system-dev libboost-filesystem-dev \
|
||||
libboost-serialization-dev python3-dev libboost-python-dev libcurl4-openssl-dev gfortran libtbb-dev libzstd-dev
|
||||
|
|
|
|||
|
|
@ -25,7 +25,7 @@ pipeline {
|
|||
axes {
|
||||
axis {
|
||||
name 'OS_NAME'
|
||||
values 'ubuntu18.04', 'centos7'
|
||||
values 'ubuntu20.04', 'centos7'
|
||||
}
|
||||
|
||||
axis {
|
||||
|
|
|
|||
Loading…
Reference in New Issue