Installation

There are two ways to install tsne-cuda. By using the pip modules/binaries on a supported system configuration, or by installing from source. We recommend using the pip installation, however some systems may not support the built binaries, or require additional optimizations - both of which can be solved by building directly from the sources.

Binaries

Requirements

If you are installing from a binary, you need to be sure that you have the following requirements:

CUDA v9.0: The binaries are built to support CUDA v9.0 (Obtainable here)
Compatible GPU: Our code uses optimized features in the binaries which rely on the GPU having compute architecture 5.0 or greater. To find your GPU compatibility, use the list here.
OpenBLAS: The binary code requires the OpenBLAS framework. On ubuntu/debian based systems, this can be obtained using sudo apt install libopenblas-dev.
FAISS: The FAISS library for GPU can be obtained here. We currently cannot provide binaries - but we are working on Conda support for this library.
Python (For python packages): Our pip installation supports Python 2.7,3.5 and 3.6.

Installing

If you are using a supported configuration, you can install tsne-cuda using the binaries provided on the releases page, or if you have the above listed configuration, you can use the Python module using pip by running:

pip install tsnecuda

We use a minor hack of the pip package manager to support multiple versions of the code. Running pip install tsnecuda==0.1.0 will install the MKL version of the code, while the default version is 0.1.1, which supports openblas.

Binary Supported Configuations

None Yet.

Building from Source

Requirements

A number of requirements are necessary for building our code from source.

CUDA: You will need a working version of the CUDA toolkit which can be obtained from here. Our code has been tested compiling with CUDA versions 8.0, 9.0 and 9.1. Other versions may not be supported.
CMAKE: Version >= 3.5.1 which can be obtained by running sudo apt install cmake on ubuntu/debian systems.
MKL/OpenBLAS: If you're using MKL, install it using the intel provided installation scripts. If you're using OpenBLAS install it using sudo apt install libopenblas-dev on ubuntu/debian systems.
GCC/llvm-clang: This is likely already installed on your system. If not, on ubuntu you can run sudo apt install build-essential to get a version.
OpenMP: On ubuntu this is likely already installed with your version of GCC. For other distributions, be sure your compiler has OpenMP support.
Python (for Python bindings): Python is not required, however to build the python bindings you must install Python. This library was tested with Python 3, however it is possible that Python 2 will work as well (though it is untested).
Doxygen: To build the documentation, a working version of doxygen is required (which can be obtained using sudo apt install doxygen on debian/ubuntu systems).
ZMQ: Necessary for building the interactive visualization. On ubuntu you can obtain ZMQ by using sudo apt install libzmq-dev.

Compiling from source

First, clone the repository, and change into the cloned directory using:

git clone https://github.com/rmrao/tsne-cuda.git && cd tsne-cuda

Next, initialize the submodules from the root directory using:

git submodule init
git submodule update

Next, change in the build directory:

cd build/

From the build directory, we can configure our project. There are a number of options that may be necessary:

-DBUILD_PYTHON: (DEFAULT OFF) Build the python package. This is necessary for the python bindings. To turn this on, use -DBUILD_PYTHON=TRUE
-DBUILD_TEST: (DEFAULT OFF) Build the test suite. To turn this on, use -DBUILD_TEST=TRUE
-WITH_MKL: (DEFAULT ON) Build with MKL support. If your MKL is installed in the default location "/opt/intel/mkl" then this is the only argument you need to change. If MKL is installed somewhere else, you must also pass the root MKL directory with -DMKL_DIR= . If you turn this off, you must have OpenBLAS installed
-DWITH_ZMQ: (DEFAULT OFF) There is a bug when using GCC version >= 6.0 with nvcc which means that ZMQ cannot be properly compiled. Thus, to avoid this bug which is present in Ubuntu versions 17.10 and 18.04 by default, you must use -DWITH_ZMQ=FALSE.
-DCMAKE_CXX_COMPILER,-DCMAKE_C_COMPILER (DEFAULT system default) It is possible on newer systems that you will get a compatability error "NVCC does not support GCC versions greater than 6.4.0". To fix this error, you can install an older compiler, and use these cmake options to build the library.

To configure, use the following CMAKE command:

cmake .. CMAKE_ARGS

where the CMAKE_ARGS are taken from the above. Thus, if you wanted to build the python bindings without ZMQ, then you would use the following command:

cmake .. -DBUILD_PYTHON=TRUE -DWITH_ZMQ=FALSE

Finally, to build the library use:

make

For speedy compilation (using multiple threads), you can use

make -j<num cores>

Using multiple threads may throw errors in the compilation due to nonexistent files. To fix this, just run a single threaded make after compilation completes.

Installing the python bindings

Once you have compiled the python bindings using -DBUILD_PYTHON you can install the Python bindings by changing into the build/python directory, and runing:

python setup.py install

Validating the Install

When compiling from sources, the most effective way to validate your installation is by building the tests (including -DBUILD_TEST=TRUE in your CMAKE_ARGS), and then running ctest and verifying that all of the tests pass.