Understanding Virtual Environment for Python

Question

I've been reading up on virtual environment, and it seems like an extremely useful tool, but now I'm questioning how I've set up my entire python environment thus far. Right now, all of the modules and packages that I have installed are residing in this directory:

/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages

But the virtualenv docs seem to suggest that such universal system installs are a bad thing. If that's the case, then what should I do with my current modules and how should I install future modules? For instance, I recently installed flask from my user directory with this command:

pip install flask

It now resides in site-packages. Should I have done something different? I'm having trouble with the documentation, which seems to suggest that I need to go into a project directory, set up a virtual environment, and install all of the modules that I need using virtualenv. Is this the case? Is there any way to make things less cumbersome? It seems like installing potentially dozens of packages for every single project directory would be a little much.

Or is it the case that I only need to create virtual environments for projects that use older versions of modules than the ones I have installed in the system directory? If that's the case, however, then what's up with the virtualenv mantra that seems to discourage all system installs?

The widespread encouragement to use virtualenv is probably a consideration for multiuser machines in which system installs can result in all sorts of unknown consequences for other users. It's also handy/prudent to use it per-project yourself when experimenting with package versions and such. Finally, you can export environments to make it easier for others to use your scripts which is nice. — m.brindley
– m.brindley, Commented Feb 2, 2013 at 22:43
In addition you could take a look at virtualenvwrapper which is a very powerful tool. It permits you to organizes all of your virtual environments in one place, switch between and configure. — ScotchAndSoda
– ScotchAndSoda, Commented Feb 2, 2013 at 22:59

Rob Carpenter · Accepted Answer · 2013-02-02 22:28:06Z

If you've already installed virtualenv like this:

pip install virtualenv

You'll then want to setup a particular virtualenv folder:

virtualenv [your project folder name]

This will create that project folder with a few important subdirectories.

You'll activate your virtualenv first before installing anything new, the newly installed modules will be available to you only when 'sourced' into your virtualenv. From your project folder type:

source bin/activate

You then will see your virtualenv name in parenthesis on each terminal line. This indicates you are 'sourced' in. NOW install stuff with pip or easy_install.

pip install flask

virtualenv basically sets your path to look in [venv folder]/bin for executables instead of /usr/local/bin or whatever. So you can copy files straight into your virtual env bin folder. (MongoDB files for instance just come in a zip/tar file, you can just untar them into your venv bin folder and you will have access to that particular version of MongoDB when 'sourced' in.) Try for yourself, run this command from your virtual and then default environment to see how it changes.

echo $PATH && echo $PYTHONPATH

To exit out of your virtualenv:

deactivate

Typing this will get you back to your default environment.

If you haven't read this yet, it's a pretty good resource.

https://python-guide.readthedocs.org/en/latest/dev/virtualenvs/

Makes sense, but should I be doing this for all of my projects, or only those that have dependencies that conflict with packages I have installed on the system? In other words, should I always avoid system installs, leaving site-packages essentially empty, and only import modules/packages locally using virtualenv? If so, is there a rationale behind this other than the danger of updating modules that are not backwards compatible? Is there a speed consideration?
As far as I know there are no repercussions for using virtualenv. Like I mentioned it just sets up your environment variables for you and creates a new lib and bin folder. Its a great way to keep your projects organized. Its also a great way to share your entire environment. If you git init your virtualenv folder then anyone who clones it doesn't have to download all the extra dependencies. They're right there in the bin and lib folders. You may need to write a bash file that allows them to source into it, but thats trivial. Install in your default env if other apps require dependencies.

thebjorn · Accepted Answer · 2013-02-02 22:53:37Z

Before you put/support anything in production there is minimal benefit from a virtualenv. It's just an extra step to activate the virtualenv, and of course you have to install your standard environment in every virtualenv.. not really that much extra effort...

When you've put something in production, however, it's potentially a great win when things go bump in the night :-)

Collectives™ on Stack Overflow

Understanding Virtual Environment for Python

2 Answers 2

2 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Related