From 9a17db48bc691dca98160ba206c54540c2ccbc31 Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Thu, 28 Nov 2019 17:15:21 +0800 Subject: [PATCH 01/10] Add docker compilation images --- install.md | 88 +++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 81 insertions(+), 7 deletions(-) diff --git a/install.md b/install.md index f53586af36..854326f664 100644 --- a/install.md +++ b/install.md @@ -1,6 +1,11 @@ # Install Milvus from Source Code -## Software requirements +- [Build from source](https://github.com/jielinxu/milvus/edit/0.6.0/install.md#build-from-source) +- [Compile Milvus on Docker](https://github.com/jielinxu/milvus/edit/0.6.0/install.md#compile-milvus-on-docker) + +## Build from source + +### Requirements - Ubuntu 18.04 or higher @@ -8,21 +13,21 @@ - CMake 3.12 or higher -##### For GPU version, you will also need: +##### For GPU-enabled version, you will also need: - CUDA 10.0 or higher - NVIDIA driver 418 or higher -## Compilation +### Compilation -### Step 1 Install dependencies +#### Step 1 Install dependencies ```shell $ cd [Milvus root path]/core $ ./ubuntu_build_deps.sh ``` -### Step 2 Build +#### Step 2 Build ```shell $ cd [Milvus root path]/core @@ -31,7 +36,7 @@ or $ ./build.sh -t Release ``` -By default, it will build CPU version. To build GPU version, add `-g` option +By default, it will build CPU-only version. To build GPU version, add `-g` option ```shell $ ./build.sh -g ``` @@ -43,7 +48,7 @@ $./build.sh -h When the build is completed, all the stuff that you need in order to run Milvus will be installed under `[Milvus root path]/core/milvus`. -## Launch Milvus server +### Launch Milvus server ```shell $ cd [Milvus root path]/core/milvus @@ -68,6 +73,75 @@ To stop Milvus server, run: $ ./stop_server.sh ``` +## Compile Milvus on Docker + +With this dockerfile, you should be able to compile CPU-only or GPU-enabled Milvus on any Linux platform that run Docker. To build a GPU-enabled Milvus, you neeed to install [NVIDIA Docker](https://github.com/NVIDIA/nvidia-docker/) first. + +### Step 1 Pull Milvus Docker images + +Pull CPU-only image: + +```shell +$ Docker pull milvusdb/milvus-cpu-build-env:v0.6.0-ubuntu18.04 +``` + +Pull GPU-enabled image: + +```shell +$ Docker pull milvusdb/milvus-gpu-build-env:v0.6.0-ubuntu18.04 +``` +### Step 2 Start the Docker container + +Start a CPU-only container: + +```shell +$ Docker run -it -p 19530:19530 -d milvusdb/milvus-cpu-build-env:v0.6.0-ubuntu18.04 +``` + +Start a GPU container: + +```shell +$ Docker run — runtime=nvidia -it -p 19530:19530 -d milvusdb/milvus-gpu-build-env:v0.6.0-ubuntu18.04 +``` +To enter the container: + +```shell +$ Docker exec -it [container_id] bash +``` +### Step 3 Download Milvus source code and compile it in the container + +Download Milvus source code: + +```shell +$ cd /home +$ wget https://github.com/milvus-io/milvus/archive/0.6.0.zip +``` + +Unzip the source package: + +```shell +$ Apt-get update +$ Apt-get install unzip +$ unzip ./0.6.0.zip +``` + +The source code is extracted into a folder called `milvus-0.6.0`. + +If you are using a CPU-only image, compile it like this: +```shell +$ ./build.sh -t Release +``` + +For GPU-enabled image, you need to add a `-g` parameter: +```shell +$ ./build.sh -g -t Release +``` + +Then start Milvus server: +```shell +$ ./start_server.sh +``` + ## Troubleshooting 1. If you encounter the following error when compiling: `protocol https not supported or disabled in libcurl`. From 534d0558291497fc7f09bf5d5ca5725ce4fcf7fc Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Thu, 28 Nov 2019 17:18:01 +0800 Subject: [PATCH 02/10] Update install.md --- install.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/install.md b/install.md index 854326f664..4bdb4a6127 100644 --- a/install.md +++ b/install.md @@ -1,7 +1,7 @@ # Install Milvus from Source Code -- [Build from source](https://github.com/jielinxu/milvus/edit/0.6.0/install.md#build-from-source) -- [Compile Milvus on Docker](https://github.com/jielinxu/milvus/edit/0.6.0/install.md#compile-milvus-on-docker) +- [Build from source](#build-from-source) +- [Compile Milvus on Docker](#compile-milvus-on-docker) ## Build from source From add279dadc1d628c0eea24218b6e40802c54434a Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Thu, 28 Nov 2019 17:33:52 +0800 Subject: [PATCH 03/10] Update install.md --- install.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/install.md b/install.md index 4bdb4a6127..b7293c788c 100644 --- a/install.md +++ b/install.md @@ -75,9 +75,9 @@ $ ./stop_server.sh ## Compile Milvus on Docker -With this dockerfile, you should be able to compile CPU-only or GPU-enabled Milvus on any Linux platform that run Docker. To build a GPU-enabled Milvus, you neeed to install [NVIDIA Docker](https://github.com/NVIDIA/nvidia-docker/) first. +With this dockerfile, you should be able to compile Milvus on any Linux platform that run Docker. To build a GPU supported Milvus, you neeed to install [NVIDIA Docker](https://github.com/NVIDIA/nvidia-docker/) first. -### Step 1 Pull Milvus Docker images +#### Step 1 Pull Milvus Docker images Pull CPU-only image: @@ -90,7 +90,7 @@ Pull GPU-enabled image: ```shell $ Docker pull milvusdb/milvus-gpu-build-env:v0.6.0-ubuntu18.04 ``` -### Step 2 Start the Docker container +#### Step 2 Start the Docker container Start a CPU-only container: @@ -108,7 +108,7 @@ To enter the container: ```shell $ Docker exec -it [container_id] bash ``` -### Step 3 Download Milvus source code and compile it in the container +#### Step 3 Download Milvus source code Download Milvus source code: @@ -125,14 +125,19 @@ $ Apt-get install unzip $ unzip ./0.6.0.zip ``` -The source code is extracted into a folder called `milvus-0.6.0`. +The source code is extracted into a folder called `milvus-0.6.0`. To enter its core directory: + +```shell +$ cd ./milvus-0.6.0/core +``` +#### Step 4 Compile Milvus in the container If you are using a CPU-only image, compile it like this: ```shell $ ./build.sh -t Release ``` -For GPU-enabled image, you need to add a `-g` parameter: +If you are using a GPU-enabled image, you need to add a `-g` parameter: ```shell $ ./build.sh -g -t Release ``` From c438fa77be4aeab69ef76051799a8710e8b9ae58 Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Thu, 28 Nov 2019 17:34:45 +0800 Subject: [PATCH 04/10] Update install.md --- install.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/install.md b/install.md index b7293c788c..0e4701f6cf 100644 --- a/install.md +++ b/install.md @@ -77,7 +77,7 @@ $ ./stop_server.sh With this dockerfile, you should be able to compile Milvus on any Linux platform that run Docker. To build a GPU supported Milvus, you neeed to install [NVIDIA Docker](https://github.com/NVIDIA/nvidia-docker/) first. -#### Step 1 Pull Milvus Docker images +### Step 1 Pull Milvus Docker images Pull CPU-only image: @@ -90,7 +90,7 @@ Pull GPU-enabled image: ```shell $ Docker pull milvusdb/milvus-gpu-build-env:v0.6.0-ubuntu18.04 ``` -#### Step 2 Start the Docker container +### Step 2 Start the Docker container Start a CPU-only container: @@ -108,7 +108,7 @@ To enter the container: ```shell $ Docker exec -it [container_id] bash ``` -#### Step 3 Download Milvus source code +### Step 3 Download Milvus source code Download Milvus source code: @@ -130,7 +130,7 @@ The source code is extracted into a folder called `milvus-0.6.0`. To enter its c ```shell $ cd ./milvus-0.6.0/core ``` -#### Step 4 Compile Milvus in the container +### Step 4 Compile Milvus in the container If you are using a CPU-only image, compile it like this: ```shell From 627576d575f24c29e08f72c751fb580374c99ea5 Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Thu, 28 Nov 2019 17:36:21 +0800 Subject: [PATCH 05/10] Update install.md --- install.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/install.md b/install.md index 0e4701f6cf..48ad3d7577 100644 --- a/install.md +++ b/install.md @@ -152,9 +152,10 @@ $ ./start_server.sh `protocol https not supported or disabled in libcurl`. First, make sure you have `libcurl4-openssl-dev` installed in your system. Then try reinstall CMake from source with `--system-curl` option: -```shell -$ ./bootstrap --system-curl -$ make -$ sudo make install -``` + + ```shell + $ ./bootstrap --system-curl + $ make + $ sudo make install + ``` From c7ca9a770b380e28642bfcdce538cde385b31d78 Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Thu, 28 Nov 2019 18:20:57 +0800 Subject: [PATCH 06/10] Update install.md --- install.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/install.md b/install.md index 48ad3d7577..c226d67c7d 100644 --- a/install.md +++ b/install.md @@ -82,31 +82,31 @@ With this dockerfile, you should be able to compile Milvus on any Linux platform Pull CPU-only image: ```shell -$ Docker pull milvusdb/milvus-cpu-build-env:v0.6.0-ubuntu18.04 +$ docker pull milvusdb/milvus-cpu-build-env:v0.6.0-ubuntu18.04 ``` Pull GPU-enabled image: ```shell -$ Docker pull milvusdb/milvus-gpu-build-env:v0.6.0-ubuntu18.04 +$ docker pull milvusdb/milvus-gpu-build-env:v0.6.0-ubuntu18.04 ``` ### Step 2 Start the Docker container Start a CPU-only container: ```shell -$ Docker run -it -p 19530:19530 -d milvusdb/milvus-cpu-build-env:v0.6.0-ubuntu18.04 +$ docker run -it -p 19530:19530 -d milvusdb/milvus-cpu-build-env:v0.6.0-ubuntu18.04 ``` Start a GPU container: ```shell -$ Docker run — runtime=nvidia -it -p 19530:19530 -d milvusdb/milvus-gpu-build-env:v0.6.0-ubuntu18.04 +$ docker run — runtime=nvidia -it -p 19530:19530 -d milvusdb/milvus-gpu-build-env:v0.6.0-ubuntu18.04 ``` To enter the container: ```shell -$ Docker exec -it [container_id] bash +$ docker exec -it [container_id] bash ``` ### Step 3 Download Milvus source code From 1e4442e33ce4f37234f2e8002bb4404aaf01d5e0 Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Thu, 28 Nov 2019 18:28:57 +0800 Subject: [PATCH 07/10] Update install.md --- install.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/install.md b/install.md index c226d67c7d..fd11121f39 100644 --- a/install.md +++ b/install.md @@ -120,8 +120,8 @@ $ wget https://github.com/milvus-io/milvus/archive/0.6.0.zip Unzip the source package: ```shell -$ Apt-get update -$ Apt-get install unzip +$ apt-get update +$ apt-get install unzip $ unzip ./0.6.0.zip ``` From 9c9af89f7b2f7c0aad2ae1bc4a4fd259a0e9dd07 Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Mon, 2 Dec 2019 11:46:29 +0800 Subject: [PATCH 08/10] [skip ci] Add note about troubleshooting --- install.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/install.md b/install.md index fd11121f39..d9914f66b8 100644 --- a/install.md +++ b/install.md @@ -3,6 +3,8 @@ - [Build from source](#build-from-source) - [Compile Milvus on Docker](#compile-milvus-on-docker) +If you encounter any problems/issues compiling Milvus from source, please refer to [Troubleshooting](#troubleshooting). + ## Build from source ### Requirements @@ -151,11 +153,11 @@ $ ./start_server.sh 1. If you encounter the following error when compiling: `protocol https not supported or disabled in libcurl`. First, make sure you have `libcurl4-openssl-dev` installed in your system. -Then try reinstall CMake from source with `--system-curl` option: +Then try reinstalling the latest CMake from source with `--system-curl` option: ```shell $ ./bootstrap --system-curl $ make $ sudo make install ``` - +If the `--system-curl` doesn't work, you can also reinstall CMake in **Ubuntu Software** on your local computer. From a78fdb2fd6d035f3b110f1db32ead2302cce6e7f Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Mon, 2 Dec 2019 11:47:13 +0800 Subject: [PATCH 09/10] [skip ci] minor change --- install.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/install.md b/install.md index d9914f66b8..35884ffd52 100644 --- a/install.md +++ b/install.md @@ -160,4 +160,4 @@ Then try reinstalling the latest CMake from source with `--system-curl` option: $ make $ sudo make install ``` -If the `--system-curl` doesn't work, you can also reinstall CMake in **Ubuntu Software** on your local computer. +If the `--system-curl` command doesn't work, you can also reinstall CMake in **Ubuntu Software** on your local computer. From 39d6b633376f0d8d72972ffbe562eec02130b944 Mon Sep 17 00:00:00 2001 From: jielinxu <52057195+jielinxu@users.noreply.github.com> Date: Mon, 2 Dec 2019 15:24:15 +0800 Subject: [PATCH 10/10] [skip ci] Replace zip with tar --- install.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/install.md b/install.md index 35884ffd52..ae9c262f8b 100644 --- a/install.md +++ b/install.md @@ -116,15 +116,13 @@ Download Milvus source code: ```shell $ cd /home -$ wget https://github.com/milvus-io/milvus/archive/0.6.0.zip +$ wget https://github.com/milvus-io/milvus/archive/0.6.0.tar.gz ``` -Unzip the source package: +Extract the source package: ```shell -$ apt-get update -$ apt-get install unzip -$ unzip ./0.6.0.zip +$ tar xvf ./v0.6.0.tar.gz ``` The source code is extracted into a folder called `milvus-0.6.0`. To enter its core directory: