I'm searching for a way to use the GPU from inside a docker container.
The container will execute arbitrary code so i don't want to use the privileged mode.
Any tips?
From previous research i understood that run -v and/or LXC cgroup was the way to go but i'm not sure how to pull that off exactly
Writing an updated answer since most of the already present answers are obsolete as of now.
Versions earlier than Docker 19.03 used to require nvidia-docker2 and the --runtime=nvidia flag.
Since Docker 19.03, you need to install nvidia-container-toolkit package and then use the --gpus all flag.
So, here are the basics,
Package Installation
Install the nvidia-container-toolkit package as per official documentation at Github.
For Redhat based OSes, execute the following set of commands:
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) $ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.repo | sudo tee /etc/yum.repos.d/nvidia-docker.repo $ sudo yum install -y nvidia-container-toolkit $ sudo systemctl restart docker For Debian based OSes, execute the following set of commands:
# Add the package repositories $ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) $ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - $ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list $ sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit $ sudo systemctl restart docker Running the docker with GPU support
docker run --name my_all_gpu_container --gpus all -t nvidia/cuda Please note, the flag --gpus all is used to assign all available gpus to the docker container.
To assign specific gpu to the docker container (in case of multiple GPUs available in your machine)
docker run --name my_first_gpu_container --gpus device=0 nvidia/cuda Or
docker run --name my_first_gpu_container --gpus '"device=0"' nvidia/cuda 
Regan's answer is great, but it's a bit out of date, since the correct way to do this is avoid the lxc execution context as Docker has dropped LXC as the default execution context as of docker 0.9.
Instead it's better to tell docker about the nvidia devices via the --device flag, and just use the native execution context rather than lxc.
These instructions were tested on the following environment:
See CUDA 6.5 on AWS GPU Instance Running Ubuntu 14.04 to get your host machine setup.
$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9 $ sudo sh -c "echo deb https://get.docker.com/ubuntu docker main > /etc/apt/sources.list.d/docker.list" $ sudo apt-get update && sudo apt-get install lxc-docker ls -la /dev | grep nvidia crw-rw-rw- 1 root root 195, 0 Oct 25 19:37 nvidia0 crw-rw-rw- 1 root root 195, 255 Oct 25 19:37 nvidiactl crw-rw-rw- 1 root root 251, 0 Oct 25 19:37 nvidia-uvm I've created a docker image that has the cuda drivers pre-installed. The dockerfile is available on dockerhub if you want to know how this image was built.
You'll want to customize this command to match your nvidia devices. Here's what worked for me:
$ sudo docker run -ti --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidiactl:/dev/nvidiactl --device /dev/nvidia-uvm:/dev/nvidia-uvm tleyden5iwx/ubuntu-cuda /bin/bash This should be run from inside the docker container you just launched.
Install CUDA samples:
$ cd /opt/nvidia_installers $ ./cuda-samples-linux-6.5.14-18745345.run -noprompt -cudaprefix=/usr/local/cuda-6.5/ Build deviceQuery sample:
$ cd /usr/local/cuda/samples/1_Utilities/deviceQuery $ make $ ./deviceQuery If everything worked, you should see the following output:
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 6.5, CUDA Runtime Version = 6.5, NumDevs = 1, Device0 = GRID K520 Result = PASS 
ls -la /dev | grep nvidia but CUDA can't find any CUDA-capable device: ./deviceQuery ./deviceQuery Starting... CUDA Device Query (Runtime API) version (CUDART static linking) cudaGetDeviceCount returned 38 -> no CUDA-capable device is detected Result = FAIL Is it because of the mismatch of the CUDA libs on the host and in the container?
Ok i finally managed to do it without using the --privileged mode.
I'm running on ubuntu server 14.04 and i'm using the latest cuda (6.0.37 for linux 13.04 64 bits).
Install nvidia driver and cuda on your host. (it can be a little tricky so i will suggest you follow this guide https://askubuntu.com/questions/451672/installing-and-testing-cuda-in-ubuntu-14-04)
ATTENTION : It's really important that you keep the files you used for the host cuda installation
We need to run docker daemon using lxc driver to be able to modify the configuration and give the container access to the device.
One time utilization :
sudo service docker stop sudo docker -d -e lxc Permanent configuration Modify your docker configuration file located in /etc/default/docker Change the line DOCKER_OPTS by adding '-e lxc' Here is my line after modification
DOCKER_OPTS="--dns 8.8.8.8 --dns 8.8.4.4 -e lxc" Then restart the daemon using
sudo service docker restart How to check if the daemon effectively use lxc driver ?
docker info The Execution Driver line should look like that :
Execution Driver: lxc-1.0.5 Here is a basic Dockerfile to build a CUDA compatible image.
FROM ubuntu:14.04 MAINTAINER Regan <http://stackoverflow.com/questions/25185405/using-gpu-from-a-docker-container> RUN apt-get update && apt-get install -y build-essential RUN apt-get --purge remove -y nvidia* ADD ./Downloads/nvidia_installers /tmp/nvidia > Get the install files you used to install CUDA and the NVIDIA drivers on your host RUN /tmp/nvidia/NVIDIA-Linux-x86_64-331.62.run -s -N --no-kernel-module > Install the driver. RUN rm -rf /tmp/selfgz7 > For some reason the driver installer left temp files when used during a docker build (i don't have any explanation why) and the CUDA installer will fail if there still there so we delete them. RUN /tmp/nvidia/cuda-linux64-rel-6.0.37-18176142.run -noprompt > CUDA driver installer. RUN /tmp/nvidia/cuda-samples-linux-6.0.37-18176142.run -noprompt -cudaprefix=/usr/local/cuda-6.0 > CUDA samples comment if you don't want them. RUN export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64 > Add CUDA library into your PATH RUN touch /etc/ld.so.conf.d/cuda.conf > Update the ld.so.conf.d directory RUN rm -rf /temp/* > Delete installer files. First you need to identify your the major number associated with your device. Easiest way is to do the following command :
ls -la /dev | grep nvidia If the result is blank, use launching one of the samples on the host should do the trick. The result should look like that 
As you can see there is a set of 2 numbers between the group and the date. These 2 numbers are called major and minor numbers (wrote in that order) and design a device. We will just use the major numbers for convenience.
Why do we activated lxc driver? To use the lxc conf option that allow us to permit our container to access those devices. The option is : (i recommend using * for the minor number cause it reduce the length of the run command)
--lxc-conf='lxc.cgroup.devices.allow = c [major number]:[minor number or *] rwm'
So if i want to launch a container (Supposing your image name is cuda).
docker run -ti --lxc-conf='lxc.cgroup.devices.allow = c 195:* rwm' --lxc-conf='lxc.cgroup.devices.allow = c 243:* rwm' cuda 
--device option to allow container to access host's device. However I tried to use --device=/dev/nvidia0 to allow docker container to run cuda and failed./dev/nvidiao, /dev/nvidia1, /dev/nvidiactl and /dev/nvidia-uvm with --device. Though don't know why./dev/nvidia* @Regan. For @ChillarAnand I have made a cuda-docker
We just released an experimental GitHub repository which should ease the process of using NVIDIA GPUs inside Docker containers.
Recent enhancements by NVIDIA have produced a much more robust way to do this.
Essentially they have found a way to avoid the need to install the CUDA/GPU driver inside the containers and have it match the host kernel module.
Instead, drivers are on the host and the containers don't need them. It requires a modified docker-cli right now.
This is great, because now containers are much more portable.
A quick test on Ubuntu:
# Install nvidia-docker and nvidia-docker-plugin wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nvidia-docker_1.0.1-1_amd64.deb sudo dpkg -i /tmp/nvidia-docker*.deb && rm /tmp/nvidia-docker*.deb # Test nvidia-smi nvidia-docker run --rm nvidia/cuda nvidia-smi For more details see: GPU-Enabled Docker Container and: https://github.com/NVIDIA/nvidia-docker
Install docker https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04
Build the following image that includes the nvidia drivers and the cuda toolkit
Dockerfile
FROM ubuntu:16.04 MAINTAINER Jonathan Kosgei <[email protected]> # A docker container with the Nvidia kernel module and CUDA drivers installed ENV CUDA_RUN https://developer.nvidia.com/compute/cuda/8.0/prod/local_installers/cuda_8.0.44_linux-run RUN apt-get update && apt-get install -q -y \ wget \ module-init-tools \ build-essential RUN cd /opt && \ wget $CUDA_RUN && \ chmod +x cuda_8.0.44_linux-run && \ mkdir nvidia_installers && \ ./cuda_8.0.44_linux-run -extract=`pwd`/nvidia_installers && \ cd nvidia_installers && \ ./NVIDIA-Linux-x86_64-367.48.run -s -N --no-kernel-module RUN cd /opt/nvidia_installers && \ ./cuda-linux64-rel-8.0.44-21122537.run -noprompt # Ensure the CUDA libs and binaries are in the correct environment variables ENV LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-8.0/lib64 ENV PATH=$PATH:/usr/local/cuda-8.0/bin RUN cd /opt/nvidia_installers &&\ ./cuda-samples-linux-8.0.44-21122537.run -noprompt -cudaprefix=/usr/local/cuda-8.0 &&\ cd /usr/local/cuda/samples/1_Utilities/deviceQuery &&\ make WORKDIR /usr/local/cuda/samples/1_Utilities/deviceQuery sudo docker run -ti --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidiactl:/dev/nvidiactl --device /dev/nvidia-uvm:/dev/nvidia-uvm <built-image> ./deviceQuery
You should see output similar to:
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 8.0, NumDevs = 1, Device0 = GRID K520 Result = PASS
My goal was to make a CUDA enabled docker image without using nvidia/cuda as base image. Because I have some custom jupyter image, and I want to base from that.
The host machine had nvidia driver, CUDA toolkit, and nvidia-container-toolkit already installed. Please refer to the official docs, and to Rohit's answer.
Test that nvidia driver and CUDA toolkit is installed correctly with: nvidia-smi on the host machine, which should display correct "Driver Version" and "CUDA Version" and shows GPUs info.
Test that nvidia-container-toolkit is installed correctly with: docker run --rm --gpus all nvidia/cuda:latest nvidia-smi
I found what I assume to be the official Dockerfile for nvidia/cuda here I "flattened" it, appended the contents to my Dockerfile and tested it to be working nicely:
FROM sidazhou/scipy-notebook:latest # FROM ubuntu:18.04 ########################################################################### # See https://gitlab.com/nvidia/container-images/cuda/-/blob/master/dist/10.1/ubuntu18.04-x86_64/base/Dockerfile # See https://sarus.readthedocs.io/en/stable/user/custom-cuda-images.html ########################################################################### USER root ########################################################################### # base RUN apt-get update && apt-get install -y --no-install-recommends \ gnupg2 curl ca-certificates && \ curl -fsSL https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub | apt-key add - && \ echo "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/cuda.list && \ echo "deb https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/nvidia-ml.list && \ apt-get purge --autoremove -y curl \ && rm -rf /var/lib/apt/lists/* ENV CUDA_VERSION 10.1.243 ENV CUDA_PKG_VERSION 10-1=$CUDA_VERSION-1 # For libraries in the cuda-compat-* package: https://docs.nvidia.com/cuda/eula/index.html#attachment-a RUN apt-get update && apt-get install -y --no-install-recommends \ cuda-cudart-$CUDA_PKG_VERSION \ cuda-compat-10-1 \ && ln -s cuda-10.1 /usr/local/cuda && \ rm -rf /var/lib/apt/lists/* # Required for nvidia-docker v1 RUN echo "/usr/local/nvidia/lib" >> /etc/ld.so.conf.d/nvidia.conf && \ echo "/usr/local/nvidia/lib64" >> /etc/ld.so.conf.d/nvidia.conf ENV PATH /usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH} ENV LD_LIBRARY_PATH /usr/local/nvidia/lib:/usr/local/nvidia/lib64 ########################################################################### #runtime next ENV NCCL_VERSION 2.7.8 RUN apt-get update && apt-get install -y --no-install-recommends \ cuda-libraries-$CUDA_PKG_VERSION \ cuda-npp-$CUDA_PKG_VERSION \ cuda-nvtx-$CUDA_PKG_VERSION \ libcublas10=10.2.1.243-1 \ libnccl2=$NCCL_VERSION-1+cuda10.1 \ && apt-mark hold libnccl2 \ && rm -rf /var/lib/apt/lists/* # apt from auto upgrading the cublas package. See https://gitlab.com/nvidia/container-images/cuda/-/issues/88 RUN apt-mark hold libcublas10 ########################################################################### #cudnn7 (not cudnn8) next ENV CUDNN_VERSION 7.6.5.32 RUN apt-get update && apt-get install -y --no-install-recommends \ libcudnn7=$CUDNN_VERSION-1+cuda10.1 \ && apt-mark hold libcudnn7 && \ rm -rf /var/lib/apt/lists/* ENV NVIDIA_VISIBLE_DEVICES all ENV NVIDIA_DRIVER_CAPABILITIES all ENV NVIDIA_REQUIRE_CUDA "cuda>=10.1" ########################################################################### #docker build -t sidazhou/scipy-notebook-gpu:latest . #docker run -itd -gpus all\ # -p 8888:8888 \ # -p 6006:6006 \ # --user root \ # -e NB_UID=$(id -u) \ # -e NB_GID=$(id -g) \ # -e GRANT_SUDO=yes \ # -v ~/workspace:/home/jovyan/work \ # --name sidazhou-jupyter-gpu \ # sidazhou/scipy-notebook-gpu:latest #docker exec sidazhou-jupyter-gpu python -c "import tensorflow as tf; print(tf.config.experimental.list_physical_devices('GPU'))" 
USER after installing CUDA. This can lead to issues with the image. Doing USER $NB_UID should reset it. Furthermore, I think this can be simplified by using conda, which is available inside the scipy-notebook image anyway.sidazhou/scipy-notebook?
jupyter/scipy-notebook with some additional packages installed. I thought it is not relevant to this discussion tho.jupyter/scipy-notebook then that’s fine, yes. I haven’t gotten this working for myself but I haven’t worked on this project in over a month and I don’t remember whether I tried basing it off scipy-notebook.To use GPU from docker container, instead of using native Docker, use Nvidia-docker. To install Nvidia docker use following commands
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu16.04/amd64/nvidia- docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list sudo apt-get update sudo apt-get install -y nvidia-docker sudo pkill -SIGHUP dockerd # Restart Docker Engine sudo nvidia-docker run --rm nvidia/cuda nvidia-smi # finally run nvidia-smi in the same container I would not recommend installing CUDA/cuDNN on the host if you can use docker. Since at least CUDA 8 it has been possible to "stand on the shoulders of giants" and use nvidia/cuda base images maintained by NVIDIA in their Docker Hub repo. Go for the newest and biggest one (with cuDNN if doing deep learning) if unsure which version to choose.
A starter CUDA container:
mkdir ~/cuda11 cd ~/cuda11 echo "FROM nvidia/cuda:11.0-cudnn8-devel-ubuntu18.04" > Dockerfile echo "CMD [\"/bin/bash\"]" >> Dockerfile docker build --tag mirekphd/cuda11 . docker run --rm -it --gpus 1 mirekphd/cuda11 nvidia-smi Sample output:
(if nvidia-smi is not found in the container, do not try install it there - it was already installed on thehost with NVIDIA GPU driver and should be made available from the host to the container system if docker has access to the GPU(s)):
+-----------------------------------------------------------------------------+ | NVIDIA-SMI 450.57 Driver Version: 450.57 CUDA Version: 11.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 GeForce GTX 108... Off | 00000000:01:00.0 On | N/A | | 0% 50C P8 17W / 280W | 409MiB / 11177MiB | 7% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ Prerequisites
Appropriate NVIDIA driver with the latest CUDA version support to be installed first on the host (download it from NVIDIA Driver Downloads and then mv driver-file.run driver-file.sh && chmod +x driver-file.sh && ./driver-file.sh). These are have been forward-compatible since CUDA 10.1.
GPU access enabled in docker by installing sudo apt get update && sudo apt get install nvidia-container-toolkit (and then restarting docker daemon using sudo systemctl restart docker).
Use x11docker by mviereck:
https://github.com/mviereck/x11docker#hardware-acceleration says
Hardware acceleration
Hardware acceleration for OpenGL is possible with option -g, --gpu.
This will work out of the box in most cases with open source drivers on host. Otherwise have a look at wiki: feature dependencies. Closed source NVIDIA drivers need some setup and support less x11docker X server options.
This script is really convenient as it handles all the configuration and setup. Running a docker image on X with gpu is as simple as
x11docker --gpu imagename 
x11docker seems to be a GUI, with the option to enable GPU acceleration.OS: Ubuntu 22.04
Kernel: 5.15.0-127-generic
Docker version: 27.4.1
GPU: NVIDIA RTX 4000 SFF Ada Generation
The goal was to make docker container to use GPU's computing power to work with AI model, before it was using only CPU
To make your docker see your GPU you need to install NVIDIA Container Toolkit:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \ && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list than run this:
sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list update and install it:
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit after successful installation you need to restart Docker:
sudo systemctl restart docker also you need to add flag "--gpus all" in docker run command, for example:
docker run --name lingua-platform-backend --gpus all your-image:latest than you can monitor usage of GPU with command nvtop or nvidia-smi
For me, the issue was resolved by specifying the capabilities when running the container:
docker run --rm --gpus 'all,"capabilities=compute,utility"' IMAGE_NAME reference: docker-specialized
For those interested in running Containers with full GPU access check the following git repository https://github.com/brumocas/Docker/tree/main/GPU
