milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[skip ci] update install (#5914) 

Signed-off-by: LocoRichard <lichen.wang@zilliz.com>
pull/5926/head
LocoRichard 2021-06-21 16:06:08 +08:00 committed by GitHub
parent 36fd713a68
commit cecaa40f92
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 47 additions and 288 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

335
INSTALL.md
View File

@ -1,317 +1,76 @@
# Install Milvus from Source Code
# Build and Start Milvus from Source Code
<!-- TOC -->
This article describes how to build and start Milvus Standalone and Cluster from source code.
- [Build from source](#build-from-source)
- [Prerequisites](#prerequisites)
- [Build Milvus](#build-milvus)
- [Start Milvus](#start-milvus)
- [Requirements](#requirements)
## Prerequisites
- [Compilation](#compilation)
Install the following before building Milvus from source code.
- [Launch Milvus server](#launch-milvus-server)
- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) for version control.
- [Golang](https://golang.org/doc/install) version 1.15 or higher and associated toolkits.
- [CMake](https://cmake.org/install/) version 3.14 or higher for compilation.
- [OpenBLAS](https://github.com/xianyi/OpenBLAS/wiki/Installation-Guide) (Basic Linear Algebra Subprograms) library version 0.3.9 or higher for matrix operations.
- [Compile Milvus on Docker](#compile-milvus-on-docker)
- [Step 1 Pull Milvus Docker images](#step-1-pull-milvus-docker-images)
- [Step 2 Start the Docker container](#step-2-start-the-docker-container)
## Build Milvus
- [Step 3 Download Milvus source code](#step-3-download-milvus-source-code)
1. Clone Milvus' GitHub repository:
- [Step 4 Compile Milvus in the container](#step-4-compile-milvus-in-the-container)
- [Troubleshooting](#troubleshooting)
- [Error message: `protocol https not supported or disabled in libcurl`](#error-message-protocol-https-not-supported-or-disabled-in-libcurl)
- [Error message: `internal compiler error`](#error-message-internal-compiler-error)
- [Error message: `error while loading shared libraries: libmysqlpp.so.3`](#error-message-error-while-loading-shared-libraries-libmysqlppso3)
- [CMake version is not supported](#cmake-version-is-not-supported)
<!-- /TOC -->
## Build from source
### Requirements
- Operating system
- Ubuntu 18.04 or higher
- CentOS 7
> Note: If your Linux operating system does not meet the requirements, we recommend that you pull a Docker image of [Ubuntu 18.04](https://docs.docker.com/install/linux/docker-ce/ubuntu/) or [CentOS 7](https://docs.docker.com/install/linux/docker-ce/centos/) as your compilation environment.
- GCC 7.0 or higher to support C++ 17
- CMake 3.14 or higher
- Git
For GPU-enabled version, you will also need:
- CUDA 10.x (10.0, 10.1, 10.2)
- NVIDIA driver 418 or higher
### Compilation
#### Step 1 Download Milvus source code and specify version
Download Milvus source code, change directory and specify version (for example, 0.10.3):
```shell
$ git clone https://github.com/milvus-io/milvus
$ cd ./milvus/core
$ git checkout 0.10.3
```
$ cd /home/$USER
$ git clone https://github.com/milvus-io/milvus.git
```
#### Step 2 Install dependencies
2. Install third-party dependencies:
##### Install in Ubuntu
```shell
$ ./ubuntu_build_deps.sh
```
$ cd milvus
$ ./scripts/install_deps.sh
```
##### Install in CentOS
3. Compile executed binary for Milvus:
```shell
$ ./centos7_build_deps.sh
```
$ make milvus
```
#### Step 3 Build Milvus source code
## Start Milvus
If you want to use CPU-only:
1. Start infrastructure services:
run `build.sh`:
```shell
$ ./build.sh -t Release
```
$ cd /home/$USER/milvus/deployments/docker/dev
$ sudo docker-compose up -d
```
If you want to use GPU-enabled:
2. Start Milvus:
1. Add cuda library path to `LD_LIBRARY_PATH`:
- For Milvus Standalone
```shell
$ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
```
$ cd /home/$USER/milvus
./bin/milvus run standalone > /tmp/standalone.log 2>&1 &
```
2. Add cuda binary path to `PATH`:
- For Milvus Cluster:
```shell
$ export PATH=/usr/local/cuda/bin:$PATH
```
$ cd /home/$USER/milvus
#start RootCoord
./bin/milvus run rootcoord > /tmp/rootcoord.log 2>&1 &
#start coord
./bin/milvus run datacoord > /tmp/datacoord.log 2>&1 &
./bin/milvus run indexcoord > /tmp/indexcoord.log 2>&1 &
./bin/milvus run querycoord > /tmp/querycoord.log 2>&1 &
#start node
./bin/milvus run proxy > /tmp/proxy.log 2>&1 &
./bin/milvus run datanode > /tmp/data_node.log 2>&1 &
./bin/milvus run indexnode > /tmp/index_node.log 2>&1 &
./bin/milvus run querynode > /tmp/query_node.log 2>&1 &
```
3. Add a `-g` parameter to run `build.sh`:
```shell
$ ./build.sh -g -t Release
```
When the build completes, everything that you need to run Milvus is under `[Milvus root path]/core/milvus`.
### Launch Milvus server
```shell
$ cd [Milvus root path]/core/milvus
```
Add `lib/` directory to `LD_LIBRARY_PATH`
```shell
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:[Milvus root path]/core/milvus/lib
```
Then start Milvus server:
```shell
$ cd scripts
$ ./start_server.sh
```
To stop Milvus server, run:
```shell
$ ./stop_server.sh
```
## Compile Milvus on Docker
With the following Docker images, you should be able to compile Milvus on any Linux platform that runs Docker. To build a GPU supported Milvus, you need to install [NVIDIA Docker](https://github.com/NVIDIA/nvidia-docker/) first.
### Step 1 Pull Milvus Docker images
#### Ubuntu 18.04
Pull CPU-only image:
```shell
$ docker pull milvusdb/milvus-cpu-build-env:v0.7.0-ubuntu18.04
```
Pull GPU-enabled image:
```shell
$ docker pull milvusdb/milvus-gpu-build-env:v0.7.0-ubuntu18.04
```
#### CentOS 7
Pull CPU-only image:
```shell
$ docker pull milvusdb/milvus-cpu-build-env:v0.7.0-centos7
```
Pull GPU-enabled image:
```shell
$ docker pull milvusdb/milvus-gpu-build-env:v0.7.0-centos7
```
### Step 2 Start the Docker container
Start a CPU-only container:
```shell
$ docker run -it -p 19530:19530 -d <milvus_cpu_docker_image>
```
Start a GPU container:
- For nvidia docker 2:
```shell
$ docker run --runtime=nvidia -it -p 19530:19530 -d <milvus_gpu_docker_image>
```
- For nvidia container toolkit:
```shell
docker run --gpus all -it -p 19530:19530 -d <milvus_gpu_docker_image>
```
To enter the container:
```shell
$ docker exec -it [container_id] bash
```
### Step 3 Download Milvus source code
Download Milvus source code:
```shell
$ git clone https://github.com/milvus-io/milvus
```
To enter its core directory:
```shell
$ cd ./milvus/core
```
Specify version (for example, 0.10.3):
```shell
$ git checkout 0.10.3
```
### Step 4 Compile Milvus in the container
If you are using a CPU-only image:
1. run `build.sh`:
```shell
$ ./build.sh -t Release
```
2. Start Milvus server：
```shell
$ ./start_server.sh
```
If you are using a GPU-enabled image:
1. Add cuda library path to `LD_LIBRARY_PATH`:
```shell
$ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
```
2. Add cuda binary path to `PATH`:
```shell
$ export PATH=/usr/local/cuda/bin:$PATH
```
3. Add a `-g` parameter to run `build.sh`:
```shell
$ ./build.sh -g -t Release
```
4. Start Milvus server：
```shell
$ ./start_server.sh
```
## Troubleshooting
### Error message: `protocol https not supported or disabled in libcurl`
Follow the steps below to solve this problem:
1. Make sure you have `libcurl4-openssl-dev` installed in your system.
2. Try reinstalling the latest CMake from source with `--system-curl` option:
```shell
$ ./bootstrap --system-curl
$ make
$ sudo make install
```
If the `--system-curl` command doesn't work, you can also reinstall CMake in **Ubuntu Software** on your local computer.
### Error message: `internal compiler error`
Try increasing the memory allocated to Docker. If this doesn't work, you can reduce the number of threads in CMake build in `[Milvus root path]/core/build.sh`.
```shell
make -j 8 install || exit 1 # The default number of threads is 8.
```
Note: You might also need to configure CMake build for faiss in `[Milvus root path]/core/src/index/thirdparty/faiss`.
### Error message: `error while loading shared libraries: libmysqlpp.so.3`
Follow the steps below to solve this problem:
1. Check whether `libmysqlpp.so.3` is correctly installed.
2. If `libmysqlpp.so.3` is installed, check whether it is added to `LD_LIBRARY_PATH`.
### CMake version is not supported
Follow the steps below to install a supported version of CMake:
1. Remove the unsupported version of CMake.
2. Get CMake 3.14 or higher. Here we get CMake 3.14.
```shell
$ wget https://cmake.org/files/v3.14/cmake-3.14.7-Linux-x86_64.tar.gz
```
3. Extract the file and install CMake.
```shell
$ tar zxvf cmake-3.14.7-Linux-x86_64.tar.gz
$ mv cmake-3.14.7-Linux-x86_64 /opt/cmake-3.14.7
$ ln -sf /opt/cmake-3.14.7/bin/* /usr/bin/
```