How can I upgrade pip inside a venv inside a Dockerfile?

Question

While running

$ sudo docker build -t myproj:tag .

I am hit with the message

You are using pip version 10.0.1, however version 18.0 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

and given recent occasional subtleties manifesting themselves with the error:

"/usr/bin/pip" "from pip import main" "ImportError: cannot import .."

I'd rather yield and indeed upgrade.

And so I add the pip upgrade command in the DockerFile, after the venv is built, since the pip that matters is the one inside the venv (am I getting this right?). So my Dockerfile now has this:

... RUN python -m venv venv RUN pip install --upgrade pip ...

But doing so does not avoid the "You are using pip 10.x" message. What am I missing?

Update

Though a promising suggestion, neither

RUN source venv/bin/activate RUN pip install --upgrade pip

nor

RUN source venv/bin/activate RUN python -m pip install --upgrade pip

eliminate the "You are using pip version 10.0.1, ..." message.

can you share more of your Dockerfile? Like more RUN or CMD lines? — Nils Werner
– Nils Werner, Commented Sep 7, 2018 at 11:33

David Maze · Accepted Answer · 2018-09-10 09:56:49Z

The single easiest answer to this is to just not bother with a virtual environment in a Docker image. A virtual environment gives you an isolated filesystem space with a private set of Python packages that don't conflict with the system install, but so does a Docker image. You can just use the system pip in a Docker image and it will be fine.

FROM python:3.7 RUN pip install --upgrade pip WORKDIR /usr/src/app COPY . . RUN pip install . CMD ["myscript"]

If you really want a virtual environment, you either need to specifically run the wrapper scripts from the virtual environment's path

RUN python -m venv venv RUN venv/bin/pip install --upgrade pip

or run the virtual environment "activate" script on every RUN command; the environment variables it sets won't carry over from one step to another. (Each RUN command in effect does its own docker run; docker commit sequence under the hood and will launch a new shell in a new container; the Dockerfile reference describes this a little bit.)

RUN python -m venv venv RUN . venv/bin/activate \ && pip install --upgrade pip COPY . . RUN . venv/bin/activate \ && pip install . CMD ["venv/bin/myscript"]

Trying to activate the virtual environment in its own RUN instruction does nothing beyond generate a no-op layer.

# This step does nothing RUN . venv/bin/activate # And therefore this upgrades the system pip RUN pip install --upgrade pip

Very nice and very convincing. Still (aside from the fact that you sound authoritative) could you provide a reference, or some reasoning, for the general principle that each command in a Dockerfile spawns its own shell, and that, in particular, entering a virtual environment will be transient? If all this meshes together, the biggest lesson is that one cannot dry-run a set of commands on an actual command line and then copy them to the Dockerfile.
I added a short section describing what RUN really does. I can't find the exact wording I'm looking for in the Docker documentation (it keeps using "layer") but there's at least a mention that RUN command does so in a separate shell.
Since I originally wrote this answer, there's become a reasonably commonplace pattern of using a Docker multi-stage build to create a virtual environment in a first stage, and COPY it into a final stage. This is useful if you're trying to install an image that doesn't have a precompiled "wheel" package, and you need a full (and very large) C toolchain to build it. If you're using exclusively Python wheels (you're not using an Alpine-based image) and don't need the C toolchain, then you still don't need the virtual environment usually.

not_a_bot_no_really_82353 · Accepted Answer · 2018-09-07 11:32:15Z

Before you can use your virtual environment venvyou need to activate it with

On Windows:

venv\Scripts\activate.bat

On Unix or MacOS, run:

source venv/bin/activate

Please note that venv is the name of your environment. You created this environment with RUN python -m venv venv. I strongly recommend to use a other name.

Then you can upgrade with python -m pip install --upgrade pip

Calaf · Accepted Answer · 2018-09-10 10:03:53Z

After you create a virtual environment in a Docker container through

RUN python -m venv venv

then run either

RUN venv/bin/pip install --upgrade pip

or

RUN venv/bin/python -m pip install --upgrade pip

but neither

RUN pip install --upgrade pip

nor

RUN python -m pip install --upgrade pip

Collectives™ on Stack Overflow

How can I upgrade pip inside a venv inside a Dockerfile?

3 Answers 3

3 Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Related