Skip to main content
Updated my answer to refer to some new sources and tone down my opinions and be more helpful.
Source Link

This isAs a poorly-worded and very generalPython newcomer this question frustrated me endlessly and confused me for months. It Which virtual environment and package manager(s) should I invest in learning when I know that I will be made more specificusing it for years to include different aspects Python-only virtual environments built on top of pipcome?

The best article answering this vexing question is https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/ by Jake Vanderplas. There are 100sAlthough a few years old, it provides practical answers and the history of differences amongPython package and virtual environment managers from the trenches as these 6+ different pipstate-virtualof-environment variantsthe-art was developing. The OP didn't include poetry

It was particularly frustrating for me in the data science and "big data cloud computing" communities, because conda is widely used as a relative newcomervirtual environment manager and full function package manager for Python and JavaScript, SQL, Java, HTML5, and Jupyter Notebooks.

So why use pip at all, when conda does everything that pip and venv variants do?

The answer is, "because you MUST use pip if a conda package is simply not available." Many times a required package is only available in pip format and there is no easy solution but to an already complex set ofuse pip packages. You can learn to use conda build but if you are not the package maintainer, then you must convince the package owner to generate a conda package for virtual environmentseach new release (or do it yourself.)

These pip-based VE packages differ along many important and practical dimensions:

My strong recommendation is to avoid ALL pip virtual environments whenever possible. Use conda instead. Conda provides a unified approach. It is maintained by teams of professional open source developers and has a reputable company providing funding and a commercially supported version. The teams that maintain pip, venv, virtualenv, pipenv, and many other pip variants have limited resources by comparison. The pip virtual environment plurality is frustrating for beginners. UseThe pip-based virtual environmentsenvironment tools complexity, fragmentation, fringe and their (too) many variantsunsupported packages, and wildly inconsistent support drove me to use conda. For data science work, my recommendation is that to use a pip-based virtual environment manager as a last resort when conda packages do not exist.

InThe differences between the spiritvenv variants still scare me because my time is limited to learn new packages. pipenv, venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, poetry, and others have dozens of being helpfuldifferences and not simply ranting about the excessive complexity (especiallycomplexities that take days to understand. I hate going down a path and find support for newcomers), fragmentationa package goes belly-up when a maintainer resigns (and at times contentionor gets too busy to maintain it), and inconsistent support of fringe packages in. I just need to get my job done.

In the pip-virtual-environment ethosspirit of being helpful, here are a few links to help you dive in over your head, but not get lost in Dante's Inferno (re: pip).

Choosing "core" Python packages to invest in for your career (long-term), versus getting a job done short term) is as much an art andimportant. However, it is a business analysis question. Are you trying to be a Python coder tosimply get a task done, or learning to become a professional software engineer who builds scalable performant systems that require the least amount of maintenance effort over time?

IMHO IMHO, conda will take you to the latter place more easily than dealing thewith pip-plurality problems. conda is still missing 1-step pip-package migration tools that make this a moot question. If we could easilysimply convert pip packages into conda packages then pypi.org and conda-forge could be merged. Pip is a necessary evil because conda packages are not (yet) universal, and many. Many Python programmers are either too lazy to create conda packages, or they don'tonly program in other languages enough toPython and don't need conda's design center of being language-agnostic / multi-lingual support.

Keep it simple! I need one package that does 90% of what I need and guidance and workarounds for the 10% remaining edge cases.

Check out the articles linked herein to learn more about pip-based virtual environments (and more angles).

I hope this is helpful to your open-ended questionthe original poster and gives pip and conda aficionados some things to think about.

This is a poorly-worded and very general question. It should be made more specific to include different aspects Python-only virtual environments built on top of pip. There are 100s of differences among these 6+ different pip-virtual-environment variants. The OP didn't include poetry, a relative newcomer to an already complex set of pip packages for virtual environments.

These pip-based VE packages differ along many dimensions:

My strong recommendation is to avoid ALL pip virtual environments whenever possible. Use conda instead. Conda provides a unified approach. It is maintained by teams of professional open source developers and has a reputable company providing funding and a commercially supported version. The teams that maintain pip, venv, virtualenv, pipenv, and many other pip variants have limited resources by comparison. The pip virtual environment plurality is frustrating for beginners. Use pip virtual environments and their (too) many variants as a last resort when conda packages do not exist.

In the spirit of being helpful and not simply ranting about the excessive complexity (especially for newcomers), fragmentation (and at times contention), and inconsistent support of fringe packages in the pip-virtual-environment ethos, here are a few links to help you dive in over your head, but not get lost in Dante's Inferno (re: pip).

Choosing "core" Python packages to invest in for your career (long-term), versus getting a job done short term) is as much an art and it is a business analysis question. Are you trying to be a Python coder to get a task done, or learning to become a professional software engineer who builds scalable performant systems that require the least amount of maintenance effort over time?

IMHO, conda will take you to the latter place more easily than dealing the pip-plurality problems. conda is still missing 1-step pip-package migration tools that make this a moot question. If we could easily convert pip packages into conda packages then pypi.org and conda-forge could be merged. Pip is a necessary evil because conda packages are not (yet) universal, and many Python programmers are either too lazy to create conda packages, or they don't program in other languages enough to need conda's design center of being language-agnostic / multi-lingual.

Check out the articles linked herein to learn more about pip-based virtual environments (and more angles) to your open-ended question.

As a Python newcomer this question frustrated me endlessly and confused me for months. Which virtual environment and package manager(s) should I invest in learning when I know that I will be using it for years to come?

The best article answering this vexing question is https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/ by Jake Vanderplas. Although a few years old, it provides practical answers and the history of Python package and virtual environment managers from the trenches as these state-of-the-art was developing.

It was particularly frustrating for me in the data science and "big data cloud computing" communities, because conda is widely used as a virtual environment manager and full function package manager for Python and JavaScript, SQL, Java, HTML5, and Jupyter Notebooks.

So why use pip at all, when conda does everything that pip and venv variants do?

The answer is, "because you MUST use pip if a conda package is simply not available." Many times a required package is only available in pip format and there is no easy solution but to use pip. You can learn to use conda build but if you are not the package maintainer, then you must convince the package owner to generate a conda package for each new release (or do it yourself.)

These pip-based packages differ along many important and practical dimensions:

My recommendation is to avoid ALL pip virtual environments whenever possible. Use conda instead. Conda provides a unified approach. It is maintained by teams of professional open source developers and has a reputable company providing funding and a commercially supported version. The teams that maintain pip, venv, virtualenv, pipenv, and many other pip variants have limited resources by comparison. The pip virtual environment plurality is frustrating for beginners. The pip-based virtual environment tools complexity, fragmentation, fringe and unsupported packages, and wildly inconsistent support drove me to use conda. For data science work, my recommendation is that to use a pip-based virtual environment manager as a last resort when conda packages do not exist.

The differences between the venv variants still scare me because my time is limited to learn new packages. pipenv, venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, poetry, and others have dozens of differences and complexities that take days to understand. I hate going down a path and find support for a package goes belly-up when a maintainer resigns (or gets too busy to maintain it). I just need to get my job done.

In the spirit of being helpful, here are a few links to help you dive in over your head, but not get lost in Dante's Inferno (re: pip).

Choosing "core" Python packages to invest in for your career (long-term), versus getting a job done short term) is important. However, it is a business analysis question. Are you trying to simply get a task done, or a professional software engineer who builds scalable performant systems that require the least amount of maintenance effort over time? IMHO, conda will take you to the latter place more easily than dealing with pip-plurality problems. conda is still missing 1-step pip-package migration tools that make this a moot question. If we could simply convert pip packages into conda packages then pypi.org and conda-forge could be merged. Pip is necessary because conda packages are not (yet) universal. Many Python programmers are either too lazy to create conda packages, or they only program in Python and don't need conda's language-agnostic / multi-lingual support.

Keep it simple! I need one package that does 90% of what I need and guidance and workarounds for the 10% remaining edge cases.

Check out the articles linked herein to learn more about pip-based virtual environments.

I hope this is helpful to the original poster and gives pip and conda aficionados some things to think about.

My answer did not directly address the OP's question in any direct way.
Source Link

AvoidThis is a poorly-worded and very general question. It should be made more specific to include different aspects Python-only virtual environments built on top of pip. There are 100s of differences among these 6+ different pip-virtual-environment variants. The OP didn't include poetry, a relative newcomer to an already complex set of pip packages for virtual environments.

These pip-based VE packages differ along many dimensions:

  • stability
  • maturity
  • complexity
  • active support (versus dying or dead)
  • levels of adoption near the Python ecosystem "core" versus "on the fringes" (i.e., integrated into Python.org distro)
  • easy to figure out and use (for beginners)

I will answer your question for two packages from dimension of package maturity and stability.

venv and virtualenv are the most mature, stability, and community support. From the online documentation you can see that virtualenv is in version 20.x as of today. virtualenv

virtualenv is a tool to create isolated Python environments. Since Python 3.3, a subset of it has been integrated into the standard library under the venv module. The venv module does not offer all features of this library, to name just a few more prominent:

is slower (by not having the app-data seed method), is not as extendable, cannot create virtual environments for arbitrarily installed python versions (and automatically discover these), is not upgrade-able via pip, does not have as rich programmatic API (describe virtual environments without creating them). 

virtualenvwrapper is set of scripts to help people use virtualenv (it is a "wrapper" that not well-maintained, its last update was in 2019. virtualenvwrapper

My strong recommendation is to avoid ALL pip virtual environments whenever possible. Use conda instead. Conda provides a unified approach. It is maintained by teams of professional open source developers and has a reputable company providing funding and a commercially supported version. The teams that maintain pip, venv, virtualenv, pipenv, and many other pip variants have limited resources by comparison. The pip virtual environment plurality is frustrating for beginners. Use pip virtual environments and their (too) many variants as a last resort when conda packages do not exist.

In the spirit of being helpful and not simply ranting about the excessive complexity (especially for newcomers), fragmentation (and at times contention), and inconsistent support of fringe packages in the pip-virtual-environment ethos, here are a few links to help you dive in over your head, but not get lost in Dante's Inferno (re: pip).

A Guide to Python’s Virtual Environments

Choosing "core" Python packages to invest in for your career (long-term), versus getting a job done short term) is as much an art and it is a business analysis question. Are you trying to be a Python coder to get a task done, or learning to become a professional software engineer who builds scalable performant systems that require the least amount of maintenance effort over time?

IMHO, conda will take you to the latter place more easily than dealing the pip-plurality problems. conda is still missing 1-step pip-package migration tools that make this a moot question. If we could easily convert pip packages into conda packages then pypi.org and conda-forge could be merged. Pip is a necessary evil because conda packages are not (yet) universal, and many Python programmers are either too lazy to create conda packages, or they don't program in other languages enough to need conda's design center of being language-agnostic / multi-lingual.

conda has been a god-send for me, because it supports cloud software engineering and data science's need for multilingual support of JavaScript, SQL, and Jupyter Notebook extensions, and conda plays well within Docker and other cloud-native environments. I encourage you to learn and master conda, which will enable you to side-step many complex questions that pip-based tools may never answer.

Check out the articles linked herein to learn more about pip-based virtual environments (and more angles) to your open-ended question.

Avoid ALL pip virtual environments whenever possible. Use conda instead. Conda provides a unified approach. It is maintained by teams of professional open source developers and has a reputable company providing funding and a commercially supported version. The teams that maintain pip, venv, virtualenv, pipenv, and many other pip variants have limited resources by comparison. The pip virtual environment plurality is frustrating for beginners. Use pip virtual environments and their (too) many variants as a last resort when conda packages do not exist.

This is a poorly-worded and very general question. It should be made more specific to include different aspects Python-only virtual environments built on top of pip. There are 100s of differences among these 6+ different pip-virtual-environment variants. The OP didn't include poetry, a relative newcomer to an already complex set of pip packages for virtual environments.

These pip-based VE packages differ along many dimensions:

  • stability
  • maturity
  • complexity
  • active support (versus dying or dead)
  • levels of adoption near the Python ecosystem "core" versus "on the fringes" (i.e., integrated into Python.org distro)
  • easy to figure out and use (for beginners)

I will answer your question for two packages from dimension of package maturity and stability.

venv and virtualenv are the most mature, stability, and community support. From the online documentation you can see that virtualenv is in version 20.x as of today. virtualenv

virtualenv is a tool to create isolated Python environments. Since Python 3.3, a subset of it has been integrated into the standard library under the venv module. The venv module does not offer all features of this library, to name just a few more prominent:

is slower (by not having the app-data seed method), is not as extendable, cannot create virtual environments for arbitrarily installed python versions (and automatically discover these), is not upgrade-able via pip, does not have as rich programmatic API (describe virtual environments without creating them). 

virtualenvwrapper is set of scripts to help people use virtualenv (it is a "wrapper" that not well-maintained, its last update was in 2019. virtualenvwrapper

My strong recommendation is to avoid ALL pip virtual environments whenever possible. Use conda instead. Conda provides a unified approach. It is maintained by teams of professional open source developers and has a reputable company providing funding and a commercially supported version. The teams that maintain pip, venv, virtualenv, pipenv, and many other pip variants have limited resources by comparison. The pip virtual environment plurality is frustrating for beginners. Use pip virtual environments and their (too) many variants as a last resort when conda packages do not exist.

In the spirit of being helpful and not simply ranting about the excessive complexity (especially for newcomers), fragmentation (and at times contention), and inconsistent support of fringe packages in the pip-virtual-environment ethos, here are a few links to help you dive in over your head, but not get lost in Dante's Inferno (re: pip).

A Guide to Python’s Virtual Environments

Choosing "core" Python packages to invest in for your career (long-term), versus getting a job done short term) is as much an art and it is a business analysis question. Are you trying to be a Python coder to get a task done, or learning to become a professional software engineer who builds scalable performant systems that require the least amount of maintenance effort over time?

IMHO, conda will take you to the latter place more easily than dealing the pip-plurality problems. conda is still missing 1-step pip-package migration tools that make this a moot question. If we could easily convert pip packages into conda packages then pypi.org and conda-forge could be merged. Pip is a necessary evil because conda packages are not (yet) universal, and many Python programmers are either too lazy to create conda packages, or they don't program in other languages enough to need conda's design center of being language-agnostic / multi-lingual.

conda has been a god-send for me, because it supports cloud software engineering and data science's need for multilingual support of JavaScript, SQL, and Jupyter Notebook extensions, and conda plays well within Docker and other cloud-native environments. I encourage you to learn and master conda, which will enable you to side-step many complex questions that pip-based tools may never answer.

Check out the articles linked herein to learn more about pip-based virtual environments (and more angles) to your open-ended question.

Source Link

Avoid ALL pip virtual environments whenever possible. Use conda instead. Conda provides a unified approach. It is maintained by teams of professional open source developers and has a reputable company providing funding and a commercially supported version. The teams that maintain pip, venv, virtualenv, pipenv, and many other pip variants have limited resources by comparison. The pip virtual environment plurality is frustrating for beginners. Use pip virtual environments and their (too) many variants as a last resort when conda packages do not exist.