Installation Guide
==================

This page gives instructions of how to build and install the MXNet package from
scratch on various systems. It consists of two steps:

1. First build the shared library from the C++ codes (`libmxnet.so` for linux,
 `libmxnet.dylib` for osx and `libmxnet.dll` for windows).
2. Then install the language packages (e.g. Python package).

Please refer to [FAQ](#frequently-asked-questions) if you have any problems during installation. If the instructions do not work for you, please feel free
to ask questions at [mxnet/issues](https://github.com/dmlc/mxnet/issues), or
even better to send pull request if you can fix the problem.

## Contents
- [Build the Shared Library](#build-mxnet-library)
  - [Prerequisites](#prerequisites)
  - [Building on Ubuntu/Debian](#building-on-ubuntu-debian)
  - [Building on OSX](#building-on-osx)
  - [Building on Windows](#building-on-windows)
  - [Installing pre-built packages on Windows](#installing-pre-built-packages-on-windows)
  - [Customized Building](#customized-building)
- [Python Package Installation](#python-package-installation)
- [R Package Installation](#r-package-installation)
- [Docker Images](#docker-images)
- [Trouble Shooting](#trouble-shooting)

## Build the Shared Library

Our goal is to build the shared library:
- On Linux/OSX the target library is ```libmxnet.so```
- On Windows the target libary is ```libmxnet.dll```

The minimal building requirement is

- A recent c++ compiler supporting C++ 11 such as `g++ >= 4.8` or `clang`
- A BLAS library, such as `libblas`, `atlas`, `openblas` or `intel mkl`

Optional libraries

- `CUDA Toolkit >= v7.0` to run on nvidia GPUs
  - Requires GPU with support for `Compute Capability >= 2.0`
- CUDNN to accelerate the GPU computation 
- opencv for image augmentation

We can edit `make/config.mk` to change the compile options, and then build by
`make`. If everything goes well, we can go the
[language package installation](#install-language-packages) step.

On the remaining of this section, we provide instructions to install the
dependencies and build mxnet from scratch for various systems.

### Building on Ubuntu/Debian

On Ubuntu >= 13.10, one can install the dependencies by

```bash
sudo apt-get update
sudo apt-get install -y build-essential git libatlas-base-dev libopencv-dev
```

Then build mxnet
```bash
git clone --recursive https://github.com/dmlc/mxnet
cd mxnet; make -j4
```

### Building on OSX

On OSX, we can install the dependencies by

```bash
brew update
brew tap homebrew/science
brew info opencv
brew install opencv
```

Then build mxnet

```bash
git clone --recursive https://github.com/dmlc/mxnet
cd mxnet; cp make/osx.mk ./config.mk; make -j4
```

Troubleshooting:

Some users experience the link error `ld: library not found for -lgomp`, indicating that the GNU implementation of OpenMP is not in the library path of operating system.

To resolve this issue, run the following commands:

```
sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.locate.plist # this creates the locate database if it does not exist

locate libgomp.dylib #copy the path which is generated by this command, say path1

ln -s path1 /usr/local/lib/libgomp.dylib

```

then run `make -j4` again.


### Building on Windows

First, enable Visual Studio 2013 to support more C++11 features.

 - Download and install [Visual C++ Compiler Nov 2013 CTP](http://www.microsoft.com/en-us/download/details.aspx?id=41151).
 - Copy all files in `C:\Program Files (x86)\Microsoft Visual C++ Compiler Nov 2013 CTP` (or the folder where you extracted the zip archive) to `C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC` and overwrite all existed files. Don't forget to backup the original files before copying.

Second, fetch the third-party libraries, including [OpenCV](http://sourceforge.net/projects/opencvlibrary/files/opencv-win/3.0.0/opencv-3.0.0.exe/download), [CuDNN](https://developer.nvidia.com/cudnn) and [OpenBlas](http://sourceforge.net/projects/openblas/files/v0.2.14/)(ignore this if you have MKL).

 - NOTICE: You need to register as a NVIDIA community user to get the download link of CuDNN.

Finally, use CMake to create a Visual Studio solution in `./build/`. During configuration, you may need to set the path of each third-party library, until no error is reported. Open the solution and compile, you will get a `mxnet.dll` in `./build/Release` or `./build/Debug`.

### Installing pre-built packages on Windows

Mxnet also provides pre-built packages on Windows. The pre-built package includes pre-build MxNet library, the dependent thrid-party libraries, a sample C++ solution in Visual Studio and the Python install script.

You can download the packages from the [Releases tab](https://github.com/dmlc/mxnet/releases) of MxNet. There are two variants provided: one with GPU support (using CUDA and CUDNN v3) and one without GPU support. You can choose one that fits your hardward configuration.

After download, unpack the package into a folder, say D:\MxNet, then install the package by double clicking the setupenv.cmd inside the folder. It will setup environmental variables needed by MxNet. After that, you should be able to usee the provided VS solution to build C++ programs, or to [install Python package](#python-package-installation).

### Customized Building

The configuration of mxnet can be modified by ```config.mk```
- modify the compiling options such as compilers, CUDA, CUDNN, Intel MKL,
various distributed filesystem such as HDFS/Amazon S3/...
- First copy [make/config.mk](../make/config.mk) to the project root, on which
  any local modification will be ignored by git, then modify the according flags.

#### Building with Intel MKL Support
First, `source /path/to/intel/bin/compilervars.sh` to automatically set environment variables. Then, edit [make/config.mk](../make/config.mk), let `USE_BLAS = mkl`. `USE_INTEL_PATH = NONE` is usually not necessary to be modified.

 - NOTICE: For `intel-mkl` version later than `parallel_studio_xe_2016.2.062`, you should use `source /path/to/intel/pkg_bin/compilervars.sh -arch intel64` instead where `-arch` must be specified according to your own architecture.

## Python Package Installation

The python package is located at [python/mxnet](../python/mxnet). It requires
`python>=2.7` and `numpy`. To install the latter, if `pip` is available, then

```bash
sudo pip install numpy
```

otherwise use your package manager, e.g.

```bash
sudo apt-get install python-numpy # for debian
sudo yum install python-numpy # for redhat
```

To have a quick test of the python package, we can train
a MLP on the mnist dataset:

```bash
python example/image-classification/train_mnist.py
```

or train a convolution neural network using GPU 0 if we set `USE_CUDA=1` during
compiling:

```bash
python example/image-classification/train_mnist.py --network lenet --gpus 0
```

There are several ways to install the package:

1. Install system-wide, which requires root permission

   ```bash
   cd python; sudo python setup.py install
   ```

   You will however need Python `distutils` module for this to
   work. It is often part of the core python package or it can be installed using your
   package manager, e.g. in Debian use

   ```bash
   sudo apt-get install python-setuptools
   ```

   *NOTE: If you recompiled mxnet, then you need to reinstall mxnet again to
    make the new library take effect*

2. Only set the environment variable `PYTHONPATH` to tell python where to find
   the library. For example, assume we cloned `mxnet` on the home directory
   `~`. then we can added the following line in `~/.bashrc`
   It is ***recommended for developers*** who may change the codes. The changes will be immediately reflected once you pulled the code and rebuild the project (no need to call ```setup``` again)

    ```bash
    export PYTHONPATH=~/mxnet/python
    ```

3. Install only for the current user.

    ```bash
    cd python; python setup.py develop --user
    ```

4. Copy the package into the working directory which contains the mxnet
   application programs. In this approach we don't need to change the system,
   and therefore is recommended for distributed training.

   Assume we are on the working directory, and `mxnet` is cloned on the home
   directory `~`.

   ```bash
   cp -r ~/mxnet/python/mxnet .
   cp ~/mxnet/lib/libmxnet.so mxnet/
   ```

## R Package Installation

For Windows/Mac users, we provide pre-built binary package using CPU.
You can install weekly updated package directly in R console:

```r
install.packages("drat", repos="https://cran.rstudio.com")
drat:::addRepo("dmlc")
install.packages("mxnet")
```

To install the R package. First finish the [Build the Shared Library](#build-the-shared-library) step.
Then use the following command to install dependencies and build the package at root folder

```bash
Rscript -e "install.packages('devtools', repo = 'https://cran.rstudio.com')"
cd R-package
Rscript -e "library(devtools); library(methods); options(repos=c(CRAN='https://cran.rstudio.com')); install_deps(dependencies = TRUE)"
cd ..
make rpkg
```

Now you should have the R package as a tar.gz file and you can install it as a normal package by (the version number might be different)

```bash
R CMD INSTALL mxnet_0.5.tar.gz
```


To install the package using GPU on Windows without building the package from scratch. Note that you need a couple of programs installed already:
- You'll need the [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit). This depends on Visual Studio, and a free compatible version would be [Visual Studio Community 2013](https://www.visualstudio.com/en-us/news/vs2013-community-vs.aspx). For instructions and compatibility checks, read http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-microsoft-windows/ .

- You will also need to register as a developer at nvidia and download CUDNN V3, https://developer.nvidia.com/cudnn .


1. Download the mxnet package as a ZIP from the Github repository https://github.com/dmlc/mxnet and unpack it. You will be editing the `/mxnet/R-package` folder.

2. Download the most recent GPU-enabled package from the [Releases tab](https://github.com/dmlc/mxnet/releases). Unzip this file so you have a folder `/nocudnn`. Note that this file and the folder you'll save it in will be used for future reference and not directly for installing the package. Only some files will be copied from it into the `R-package` folder.

(Note: you now have 2 folders we're working with, possibly in different locations, that we'll reference with `R-package/` and `nocudnn/`.)

3. Download CUDNN V3 from https://developer.nvidia.com/cudnn. Unpack the .zip file and you'll see 3 folders, `/bin`, `/include`, `/lib`. Copy and replace these 3 folders into `nocudnn/3rdparty/cudnn/`, or unpack the .zip file there directly.

4. Create the folder `R-package/inst/libs/x64`. We only support 64-bit operating system now, so you need the x64 folder;

5. Put dll files in `R-package/inst/libs/x64`.

The first dll file you need is `nocudnn/lib/libmxnet.dll`. The other dll files you need are the ones in all 4 subfolders of `nocudnn/3rdparty/`, for the `cudnn` and `openblas` you'll need to look in the `/bin` folders. There should be 11 dll files now in `R-package/inst/libs/x64`.

6. Copy the folder `nocudnn/include/` to `R-package/inst/`. So now you should have a folder `R-package/inst/include/` with 3 subfolders.

7. Run `R CMD INSTALL --no-multiarch R-package`. Make sure that R is added to your PATH in Environment Variables. Running the command `Where R` in Command Prompt should return the location.

Note on Library Build:

We isolate the library build with Rcpp end to maximize the portability
  - MSVC is needed on windows to build the mxnet library, because of CUDA compatiblity issue of toolchains.

## Docker Images

Builds of MXNet are available as [Docker](https://www.docker.com/whatisdocker) images:
[MXNet Docker (CPU)](https://hub.docker.com/r/kaixhin/mxnet/) or [MXNet Docker (CUDA)](https://hub.docker.com/r/kaixhin/cuda-mxnet/).
These are updated on a weekly basis with the latest builds of MXNet. Examples of running bash in a Docker container
are as follows:

```bash
# CPU only docker
sudo docker run -it kaixhin/mxnet

OR
# GPU enabled docker
sudo docker run -it --device /dev/nvidiactl --device /dev/nvidia-uvm --device /dev/nvidia0 kaixhin/cuda-mxnet:latest
```

For a guide to Docker, see the [official docs](https://docs.docker.com/userguide/). For more details on how to use the
MXNet Docker images, including requirements for CUDA support, consult the [source project](https://github.com/Kaixhin/dockerfiles).

## Trouble Shooting

### Compile failed after git pull

   Please first update the submodules, clean all and recompile:

   ```bash
   git submodule update && make clean_all && make -j4
   ```

### Compile failed after config.mk is modified

   This often happens if `USE_CUDA` or `USE_DIST_KVSTORE` has been changed. You
   need to clean all first:

    ```bash
    make clean_all && make -j4
    ```

### Still get the error message after re-installation

   e.g. `compile with USE_DIST_KVSTORE=1 to use
   dist` after recomplied with `USE_DIST_KVSTORE=1`**

   It is often because mxnet is failed to load the new built library. If you
   installed mxnet system-widely, e.g. `python setup.py install`, then you need
   to reinstall the package again.
