I had to rebuild the tensorflow pip package from source to make it work in CentOS 6. Here's a memo I made of it. (Note I did this a month ago) 1. download bazel-4.5-dist.zip and follow these steps to install, newer versions of bazel don't work as of 2017-09-04 ~$ cd ~$ wget https://github.com/bazelbuild/bazel/releases/download/0.4.5/bazel-0.4.5-dist.zip ~$ cd /usr/src ~$ mkdir bazel-0.4.5-dist.zip ~$ cd bazel-0.4.5-dist ~# mv ~/bazel-0.4.5-dist.zip ./ ~# unzip bazel-0.4.5-dist.zip ~# ./compile.sh 2. Modify ~/.bashrc to activate devtoolset-2 instead of devtoolset-6. Tensorflow will not build with newer gcc, only up to gcc 4 in ~/.bashrc source /opt/rh/devtoolset-2/enable #source /opt/rh/devtoolset-6/enable 3. Clone tensorflow into /usr/src ~$ cd /usr/src ~# git clone https://github.com/tensorflow/tensorflow 4. Configure tensorflow ~$ cd tensorflow ~# ./configure Select "No" for all support options except CUDA. Everything else should be default 5. go to /usr/src/tensorflow/third_party/gpus/crosstool modify CROSSTOOL_clang.tpl and CROSSTOOL_nvcc.tpl add the following line to the section labeled "toolchain" linker_flag : "-B/opt/rh/devtoolset-2/root/usr/bin" 6. Build tensorflow ~$ cd /usr/src/tensorflow ~# bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package 7. Create pip package ~# bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg 8. Install custom pip package ~# sudo pip install /tmp/tensorflow_pkg/tensorflow-1.3.0-cp34-cp34m-linux_x86_64.whl