3

I am trying to create a Docker image for my flask application, shown below:

# syntax=docker/dockerfile:1 FROM python:3.9.5-slim-buster as build RUN python3 -m venv /app/venv COPY . /app RUN /app/venv/bin/pip install -r /app/requirements.txt FROM gcr.io/distroless/python3 COPY --from=build /app /app # ENV PATH = "/app/venv/bin:${PATH}" EXPOSE 5000 ENTRYPOINT [ "/app/venv/bin/python3" , "main.py"]

Basically, I have two build stages: the first one creates a virtual environment with venv, and the second uses a distroless image and copies the virtual environment (along with the rest of my files) from the previous build stage to the new one.

The Docker images builds without issue, but once I try to run the image with docker run, I get the following error:

docker: Error response from daemon: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "/app/venv/bin/python3": stat /app/venv/bin/python3: no such file or directory: unknown.

This error confuses me, since I know the python executable is located at /app/venv/bin, and I double checked this by exporting the container using docker export <container name> > container.tar and exploring the tar file's contents. From what I can tell, I should not be receiving this error.

What am I doing wrong?

Edit: As requested by @RQDQ, below are bare minimum versions of my requirements.txt and main.py:

requirements.txt:

click==8.1.3 Flask==2.1.2 itsdangerous==2.1.2 Jinja2==3.1.2 MarkupSafe==2.0.1 Werkzeug==2.1.2

main.py:

from flask import Flask app = Flask(__name__, static_folder='build') @app.route("/") def hello_world(): return "<p>Hello, World!</p>" if (__name__ == "__main__"): app.run(use_reloader=False, host='0.0.0.0', port=5000, threaded=True)
Improve this question
5
  • Do you have a minimal reproduction available so I can try to run it here? Commented Jun 1, 2022 at 22:02
  • What would I need to provide for you to reproduce it? Do you mean my main.py and requirements.txt? Commented Jun 1, 2022 at 22:10
  • Yes - I'm only asking because I don't have a lot of python experience but I'm willing to run the build here to see if I can reproduce your error. Commented Jun 1, 2022 at 22:13
  • In my experience, a virtual environment needs to exactly match the Python it was initially built with; you may not be able to copy a virtual environment from one image to another with a different Python installation. If the python binary inside your virtual environment is a dynamically linked binary then the "distroless" image might be missing key parts like shared libraries and the dynamic loader. Commented Jun 1, 2022 at 23:47
  • 1
    I've seen some examples doing this same process though, like this one from GoogleContainerTools. Commented Jun 2, 2022 at 1:03

1 Answer 1

Reset to default
1

virtualenv is not standalone environment that can be copied between OSes (or containers):

$ python -m venv venv $ ls -l venv/bin/ total 36 -rw-r--r-- 1 user user 1990 Jun 2 08:35 activate -rw-r--r-- 1 user user 916 Jun 2 08:35 activate.csh -rw-r--r-- 1 user user 2058 Jun 2 08:35 activate.fish -rw-r--r-- 1 user user 9033 Jun 2 08:35 Activate.ps1 -rwxr-xr-x 1 user user 239 Jun 2 08:35 pip -rwxr-xr-x 1 user user 239 Jun 2 08:35 pip3 -rwxr-xr-x 1 user user 239 Jun 2 08:35 pip3.10 lrwxrwxrwx 1 user user 46 Jun 2 08:35 python -> /home/user/.pyenv/versions/3.10.2/bin/python lrwxrwxrwx 1 user user 6 Jun 2 08:35 python3 -> python lrwxrwxrwx 1 user user 6 Jun 2 08:35 python3.10 -> python

As you can see python executables are just links to original python executable. It is something like snapshot for your original python that may be reverted or applied. But snapshot is useless if you don't have original base. So you have to create venv at same environment that will be used on.

However in case of containers you don't need venv at all. Container is already an isolated environment and you don't need one more isolation level with venv. (At least my question about why we need to use venv inside container is still don't have an answer)

In short: remove all venv related lines:

# syntax=docker/dockerfile:1 FROM gcr.io/distroless/python3 COPY . /app WORKDIR /app RUN pip install -r requirements.txt EXPOSE 5000 ENTRYPOINT [ "python" , "main.py"]

However if you need to some extra libraries/compile tools (like gcc) to build python libraries when installing it by pip then venv may be used to be able move only resulted library binaries without store compile tools inside container.

In this case you have to use same (or compatible) python base at build image and resulted image (venv "snapshot" should be applied to compatible base).

Let's see this example:

FROM debian:11-slim AS build ... FROM gcr.io/distroless/python3-debian11 ...

Both images at least Debian based.

Or another example:

FROM python:3.9-slim as compiler ... FROM python:3.9-slim as runner ...

And again base of builder and runner is the same

Looks like python:3.9.5-slim-buster and gcr.io/distroless/python3 are both Debian based and should be compatible, but probably it is not fully compatible.

You change endpoint to ENTRYPOINT [ "sleep" , "600"]. That will allow to keep container running for 10 minutes. After that attach to running container: docker exec -it container_name bash and check is python executable exists: ls -l /app/venv/bin/

or just simply use it without venv as I said before

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

See this answer regarding why venv might be used in a docker image. In short, it's not for extra environment isolation, but rather to get all the necessary build files to run the application and put them into a fresh image to save space.
@gorilla_glue Most popular answer inside link is You don't need to use virtualenv inside a Docker Container. Yes it has addition that we may save space, but I can't see reason why dependencies inside venv folder will be smaller than same dependencies inside default /usr/lib/python/... folder. The only one reasonable case I can see that if you doing many complicated builds at builder and want to move only resulted binaries to runner. Also runner should be super slim manually created image without pip (because even smallest python-alpine has pip)
I'll accept this answer, primarily because of the mention of the distroless image probably being incompatible somehow between the build stages. Making my images identical in both stages fixed the problem. That being said, as I mentioned before, the use of venv was to save space and move files between build stages. Removing this (as you recommend) increases my image size from 1.2 GB to 1.59 GB, which is a noticeable difference. Sure, there are other ways to move those files with a user install or something else, but this seemed like the easiest.
I'll also add that using the gcr.io distroless image for Python may not have been a good idea, since the repo lists it as experimental, and does not have many options for other versions of Python (I think the base is 3.8, so if you want another version, you are out of luck).

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.