Upgrade Python in a virtual environment

If you happen to be using the venv module that comes with Python 3.3 (or later), it supports an --upgrade option.

Per the documentation:

Upgrade the environment directory to use this version of Python, assuming Python has been upgraded in-place

python3 -m venv --upgrade ENV_DIR 

Step 1: Freeze requirement and take a backup of the existing env

pip freeze > requirements.txt deactivate mv env env_old 

Step 2: Install Python 3.7 and activate the virtual environment

sudo apt-get install python3.7-venv python3.7 -m venv env source env/bin/activate python --version 

Step 3: Install requirements

sudo apt-get install python3.7-dev pip3 install -r requirements.txt 

The following method might not work in newer versions of virtualenv. Before you try to make modifications to the old virtualenv, you should save the dependencies in a requirement file (pip freeze > requirements.txt) and make a backup of it somewhere else. If anything goes wrong, you can still create a new virtualenv and install the old dependencies in it (pip install -r requirements.txt).

The following method is more convenient and robust.

Side effect: it also fixes the Symbol not found: _SSLv2_method exception when you do import ssl in a virtual environment after upgrading Python to v2.7.8.

Notice: Currently, this is for Python 2.7.x only.

If you're using Homebrew Python on OS X, first deactivate all virtualenv, then upgrade Python:

brew update && brew upgrade python 

Run the following commands (<EXISTING_ENV_PATH> is path of your virtual environment):

cd <EXISTING_ENV_PATH> rm .Python rm bin/pip{,2,2.7} rm bin/python{,2,2.7} rm -r include/python2.7 rm lib/python2.7/* rm -r lib/python2.7/distutils rm lib/python2.7/site-packages/easy_install.* rm -r lib/python2.7/site-packages/pip rm -r lib/python2.7/site-packages/pip-*.dist-info rm -r lib/python2.7/site-packages/setuptools rm -r lib/python2.7/site-packages/setuptools-*.dist-info 

Finally, recreate your virtual environment:

virtualenv <EXISTING_ENV_PATH> 

By doing so, old Python core files and standard libraries (plus setuptools and pip) are removed, while the custom libraries installed in site-packages are preserved and working, as soon as they are in pure Python. Binary libraries may or may not need to be reinstalled to function properly.

This worked for me on 5 virtual environments with Django installed.

BTW, if ./manage.py compilemessages is not working afterwards, try this:

brew install gettext && brew link gettext --force 

Let's consider that the environment that one wants to update has the name venv.

1. Back up venv requirements (optional)

First of all, back up the requirements of the virtual environment:

pip freeze > requirements.txt deactivate # Move the folder to a new one mv venv venv_old 

2. Install Python

Assuming that one doesn't have sudo access, pyenv is a reliable and fast way to install Python. For that, one should run

curl https://pyenv.run | bash 

and then

exec $SHELL 

as suggested here.

If, when one tries to update pyenv

pyenv update 

and one gets the error

bash: pyenv: command not found

that is because pyenv path wasn't exported to .bashrc. It can be solved by executing the following commands:

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bashrc 

Then restart the shell

exec "$SHELL" 

Now one should install the Python version that one wants. Let's say version 3.8.3

pyenv install 3.8.3 

One can confirm if it was properly installed by running

pyenv versions 

The output should be the location and the versions (in this case 3.8.3)

3. Create the new virtual environment

Finally, with the new Python version installed, create a new virtual environment (let's call it venv)

python3.8 -m venv venv 

Activate it

source venv/bin/activate 

and install the requirements

pip install -r requirements.txt 

Now one should be up and running with a new environment.

How to upgrade the Python version for an existing virtualenvwrapper project and keep the same name

This is for anyone using Doug Hellmann's excellent virtualenvwrapper, specifically since the existing answers may not work.

Some context:

• I work on some projects that are Python 2 and some that are Python 3; while I'd love to use python3 -m venv, it doesn't support Python 2 environments
• When I start a new project, I use mkproject which creates the virtual environment, creates an empty project directory, and cds into it
• I want to continue using virtualenvwrapper's workon command to activate any project irrespective of Python version

Directions:

Let's say your existing project is named foo and is currently running Python 2 (mkproject -p python2 foo), though the commands are the same whether upgrading from 2.x to 3.x, 3.6.0 to 3.6.1, etc. I'm also assuming you're currently inside the activated virtual environment.

1. Deactivate and remove the old virtual environment:

deactivate rmvirtualenv foo 

Note that if you've added any custom commands to the hooks (e.g., bin/postactivate), you'd need to save those before removing the environment.

2. Stash the real project in a temporary directory:

cd .. mv foo foo-tmp 

3. Create the new virtual environment (and project directory) and activate:

mkproject -p python3 foo 

4. Replace the empty generated project directory with the real project, and change back into the project directory:

cd .. mv -f foo-tmp foo cdproject 

5. reinstall dependencies, confirm new Python version, etc.:

pip install -r requirements.txt python --version 

If this is a common use case, I'll consider opening a PR to add something like upgradevirtualenv / upgradeproject to virtualenvwrapper.

I just want to clarify, because some of the answers refer to venv and others refer to virtualenv.

Use of the -p or --python flag is supported on virtualenv, but not on venv. If you have more than one Python version and you want to specify which one to create the venv with, do it on the command line, like this:

cd ~/Projects python3.6 -m venv {path to pre-existing directory you want venv in} 

You can of course upgrade with venv as others have pointed out, but that assumes you have already upgraded the Python that was used to create that venv in the first place. You can't upgrade to a Python version you don't already have on your system somewhere, so make sure to get the version you want, first, then make all the venvs you want from it.

I wasn't able to create a new virtualenv on top of the old one. But there are tools in pip which make it much faster to re-install requirements into a brand new venv. Pip can build each of the items in your requirements.txt file into a wheel package, and store that in a local cache. When you create a new venv and run pip install in it, pip will automatically use the prebuilt wheel packages if it finds them. Wheel packages install much faster than running setup.py for each module.

My ~/.pip/pip.conf file looks like this:

[global] download-cache = /Users/me/.pip/download-cache find-links = /Users/me/.pip/wheels/ [wheel] wheel-dir = /Users/me/.pip/wheels 

I install wheel (pip install wheel), then run pip wheel -r requirements.txt. This stores the built wheel packages in the wheel-dir in my pip.conf file.

From then on, any time I pip install any of these requirements, it installs them from the wheel packages, which is pretty quick.

This approach always works for me:

# First of all, delete all broken links. Replace # my_project_name` to your virtual env name find ~/.virtualenvs/my_project_name/ -type l -delete # Then create new links to the current Python version virtualenv ~/.virtualenvs/my_project_name/ # It's it. Just repeat for each virtualenv # located in ~/.virtualenvs 

It was taken from:

• Python virtualenv
• Fix virtualenv after Python upgrade via Homebrew

If you're using pipenv, I don't know if it's possible to upgrade an environment in place, but at least for minor version upgrades it seems to be smart enough not to rebuild packages from scratch when it creates a new environment. E.g., from 3.6.4 to 3.6.5:

pipenv --python 3.6.5 install 

Output:

Virtualenv already exists! Removing existing virtualenv… Creating a v$ pipenv --python 3.6.5 install Virtualenv already exists! Removing existing virtualenv… Creating a virtualenv for this project… Using /usr/local/bin/python3.6m (3.6.5) to create virtualenv… ⠋Running virtualenv with interpreter /usr/local/bin/python3.6m Using base prefix '/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6' New python executable in /Users/dmoles/.local/share/virtualenvs/autoscale-aBUhewiD/bin/python3.6 Also creating executable in /Users/dmoles/.local/share/virtualenvs/autoscale-aBUhewiD/bin/python Installing setuptools, pip, wheel...done. Virtualenv location: /Users/dmoles/.local/share/virtualenvs/autoscale-aBUhewiD Installing dependencies from Pipfile.lock (84dd0e)… 🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 47/47 — 00:00:24 To activate this project's virtualenv, run the following: pipenv shell 

And:

pipenv shell 

Output:

Spawning environment shell (/bin/bash). Use 'exit' to leave. . /Users/dmoles/.local/share/virtualenvs/autoscale-aBUhewiD/bin/activate 

And:

. /Users/dmoles/.local/share/virtualenvs/autoscale-aBUhewiD/bin/activate python 

Output:

Python 3.6.5 (default, Mar 30 2018, 06:41:53) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. 

And:

import numpy as np 

I moved my home directory from one Mac to another (Mac OS X v10.8 (Mountain Lion) to Mac OS X v10.10 (Yosemite)) and didn't realize about the broken virtualenv until I lost hold of the old laptop.

I had the virtualenv point to Python 2.7 installed by Homebrew and since Yosemite came with Python 2.7, I wanted to update my virtualenv to the system Python interpreter.

When I ran virtualenv on top of the existing directory, I was getting this error:

OSError: [Errno 17] File exists: '/Users/hdara/bin/python2.7/lib/python2.7/config'

By trial and error, I worked around this issue by removing a few links and fixing up a few more manually. This is what I finally did (similar to what @Rockalite did, but simpler):

cd <virtualenv-root> rm lib/python2.7/config rm lib/python2.7/lib-dynload rm include/python2.7 rm .Python cd lib/python2.7 gfind . -type l -xtype l | while read f; do ln -s -f /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/${f#./} $f; done 

After this, I was able to just run virtualenv on top of the existing directory.

On OS X or macOS using Homebrew to install and upgrade Python3 I had to delete symbolic links before python -m venv --upgrade ENV_DIR would work.

I saved the following in file upgrade_python3.sh, so I would remember how months from now when I need to do it again:

brew upgrade python3 find ~/.virtualenvs/ -type l -delete find ~/.virtualenvs/ -type d -mindepth 1 -maxdepth 1 -exec python3 -m venv --upgrade "{}" \; 

While this seemed to work well at first, when I ran py.test, it gave an error. In the end, I just recreated the environment from a requirements file.

For everyone with the problem

Error: Command '['/Users/me/Sites/site/venv3/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

You have to install python3.6-venv

sudo apt-get install python3.6-venv