Installing Neural Toolkits

Tensorflow

Simple CPU installation instructions

(If you want the GPU build, skip to the next section)
These instructions use virtualenv. However, you can simply use the --user flag with pip if you do not want to use virtualenv.
virtualenv tf_cpu
source tf_cpu/bin/activate
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.12.1-cp27-none-linux_x86_64.whl
pip install --upgrade $TF_BINARY_URL

Usage

$ python
>>> import tensorflow as tf
>>> hello = tf.constant("Hello, Tensorflow!")
>>> sess = tf.Session()
>>> print (sess.run(hello))
Hello, Tensorflow!

Requirements (to install from scratch)

1. >= GCC 4.9 (/home/gkumar/.local)
2. Python 2.7 or Python 3.*
3. >= Numpy 1.1
4. CUDA 8.0
5. CuDNN5 (/home/gkumar/.local/cudnn)

CUDNN

First, make sure CUDNN (5.*) is installed
IFS=:;for p in ${LD_LIBRARY_PATH}; do if [ -e ${p}/libcudnn.so.5 ]; then echo "Found CUDNN5 at ${p}"; break; fi; done
My output looks something like :
Found CUDNN5 at /home/ws15gkumar/.local/cudnn/lib64\\

CUDA

Now, add these lines to your bashrc file so that CUDA can be located by tensorflow
export CUDA_HOME=/opt/NVIDIA/cuda-8.0
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib64:$CUDA_HOME/extras/CUPTI/lib64:$LD_LIBRARY_PATH
export CPLUS_INCLUDE_PATH=$CUDA_HOME/include:$CPLUS_INCLUDE_PATH
export C_INCLUDE_PATH=$CUDA_HOME/include:$C_INCLUDE_PATH
export CPATH=$CUDA_HOME/include:$CPATH

Python Modules

To make sure that you have the correct versions of the python dependencies, run:
(This is a sanity check step, you may skip it. Tensorflow will install the correct version via dependencies)
python -c "import pip; import numpy; print pip.__version__; print numpy.__version__"
My output looks something like :
7.1.2
1.10.1

Installation Instructions

First, upgrade pip
pip install --user --upgrade pip
For the CPU build, set
->export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow_cpu-0.12.1-cp27-none-linux_x86_64.whl
and for the GPU build
->export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-0.12.1-cp27-none-linux_x86_64.whl
Now, install tensorflow
CUDA_VISIBLE_DEVICES=`/home/gkumar/scripts/free-gpu` pip install --user --upgrade $TF_BINARY_URL

An important note about GPU allocation

1. Always request a GPU via the SGE qsub command on the CLSP cluster! To start an interactive GPU session do : qlogin -l gpu=1
2. Tensorflow will always try to occupy the first GPU on the machine without regard for whether it is already occupied. (Order of devices is listed by `ls /dev/nvidia*`). Among other problems, this will cause your code to crash if the device is busy. See next item.
3. Use this script : /home/gkumar/scripts/free-gpu . It will return the identifier of the first free GPU available on the machine. In conjunction with the bash variable CUDA_VISIBLE_DEVICES you can limit the scope of GPUs available to Tensorflow.

1. Read special instructions for the CLSP cluster below

Usage

Start a python process
CUDA_VISIBLE_DEVICES=`/home/gkumar/scripts/free-gpu` python
Run (you should see something like this
-------- $ CUDA_VISIBLE_DEVICES=`/home/gkumar/scripts/free-gpu` python
Python 2.7.9 (default, Jun 29 2016, 13:08:31)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally
>>> hello = tf.constant("Hello, Tensorflow!")
>>> sess = tf.Session()
I tensorflow/core/common_runtime/gpu/gpu_device.cc:885] Found device 0 with properties:
name: Tesla K20m
major: 3 minor: 5 memoryClockRate (GHz) 0.7055
pciBusID 0000:03:00.0
Total memory: 4.63GiB
Free memory: 4.57GiB
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0: Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Tesla K20m, pci bus id: 0000:03:00.0)
>>> print (sess.run(hello))
Hello, Tensorflow!
>>> quit()
-----------

"IMPORTANT" : Running this on the CLSP machines

1. Always run without GPU support unless you know what you are doing.
2. Tensorflow will use all of the threads on the machine it is running on. It is important to follow steps
3. Change the examples : Find the "Session" creation in the example code. Make the changes outlined in http://stackoverflow.com/questions/33617638/tensorflow-mac-os-x-cant-determine-number-of-cpu-cores. Basically, you set a number of threads and give that as config to the Session creation.
4. Now, ask the grid for the number of threads you need. So if you need 20 threads, run "qlogin -pe smp 20"
5. For the sequence to sequence model, a smaller dataset is here : /export/a13/kduh/tensorflow_tutorial/rnn/translate/trial_enfr . Copy this to your work directory.
6. export install=/path/to/your/tensorflow/github/repo
7. cd $install/tensorflow/models/rnn/translate
8. python translate.py --data_dir trial_enfr/ --train_dir train_dir --en_vocab_size=5000 --fr_vocab_size=5000 --steps_per_checkpoint=1

Data

1. The processed WMT en-fr data is here : /export/a13/kduh/tensorflow_tutorial/rnn/translate/wmt_enfr/ (Vocab size = 10000)
2. A smaller version of the en-fr data is here : /export/a13/kduh/tensorflow_tutorial/rnn/translate/trial_enfr (Vocab size = 5000)

Dynet

Installation Instructions

1. hg clone https://noisychannel@bitbucket.org/eigen/eigen
2. cd eigen
3. hg pull && hg update 3.2
4. cd ..
5. git clone https://github.com/clab/cnn.git
6. cd cnn
7. mkdir build
8. cd build
9. cmake .. -DEIGEN3_INCLUDE_DIR=${DATADIR}/code/eigen
10. make -j${MAX_THREADS}

Examples

Check cnn/examples for examples of common neural MT models.