34

I'm writing a bit of code that will report and reconcile differences between two pip-managed python installations.

How can I programmatically get the information provided by pip list without making a subprogram invocation of pip?

Improve this question
1

6 Answers 6

Reset to default
46

It's possible to get a list of packages programmatically:

Options:

A. _internal.main

from pip import _internal _internal.main(['list'])

This will print out three columns with Package. Version, and Location

Note that usage of pip's internal api is not recommended.

B. pkg_resources

import pkg_resources print([p.project_name for p in pkg_resources.working_set]) # note that this is same as calling pip._vendor.pkg_resources.working_set

C. iter_modules

Takes a long time to execute (~300ms on computer w/ I5 CPU, SSD, & 8 gigs ram). The benefit is that it will have a far more extensive list of modules and it will output importable names.

Ex: python-dateutil is imported as dateutil, but iter_modules will give you the importable name: dateutil

from pkgutil import iter_modules print([p.name for p in iter_modules()])

D. Call pip in command line via subprocess

The solution to this is trivial and I'll leave this as an exercise to the reader

aka I'm too lazy to do this, good luck! :D

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

I've verified A and B and am accepting this answer. Most excellent!
I guess the approach with pkg_resources is best if you also want to know the package version, as the returned object has a .version member containing it.
The pkg_resources alternative is deprecated as an API: setuptools.pypa.io/en/latest/pkg_resources.html. New code should utilize importlib, instead.
20

Update for Python 3.6 and Pip 19.0.1

> from pip._internal.utils.misc import get_installed_distributions > p = get_installed_distributions() > pprint.pprint(p) [wheel 0.32.3 (/usr/local/lib/python3.7/site-packages), wcwidth 0.1.7 (/usr/local/lib/python3.7/site-packages), virtualenv 16.0.0 (/usr/local/lib/python3.7/site-packages), virtualenv-clone 0.3.0 (/usr/local/lib/python3.7/site-packages), urllib3 1.24.1 (/usr/local/lib/python3.7/site-packages), typing 3.6.6 (/usr/local/lib/python3.7/site-packages), terminaltables 3.1.0 (/usr/local/lib/python3.7/site-packages), ...

Original Answer

Pip is just python module, so just import it and call list:

import pip pip.main(['list']) # you can get details on package using show: pip.main(['show', 'wheel'])

Ok so there is better way:

pip.utils.get_installed_distributions()

returns you list of packages installed.

packages = pip.utils.get_installed_distributions() p = packages[0] p.project_name p.version p.egg_name p.location

You can see what pip list is doing from the source code here

Also get_installed_distributions accept whole bunch of parameters to return only local packages (from current virtualenv) etc. Please see help here.

There is also underlying low level command from _vendor module:

list(pip._vendor.pkg_resources.working_set)

However get_installed_distributions provide simplier api.

Improve this answer

14 Comments

How can I get the resulting output? x=pip.main(['list']) sets x to 0 indicating successful execution, with the results going to stdout.
There is number of ways given here: stackoverflow.com/questions/5136611/… however I do not like any of them, because they are all cumbersome, let me see how to call it slightly different way,
@vittore I could not find documentation for pip module. I keep finding this. But that doesn't have anything for the module api
Seems like pip changed its structure. As of pip 19.0.1 (on Python 3.6, although I don't think that matters), this works: from pip._internal.utils.misc import get_installed_distributions; p = get_installed_distributions()
Thank you, indeed it seems to be so (they even use it at the relevant tests) but it was a change that puzzled me. In the mean time, I used pkg_resources from setuptools to gather all the licenses.
|
6

python -m pip list (robust method)

import subprocess import sys def pip_list(): args = [sys.executable, "-m", "pip", "list"] p = subprocess.run(args, check=True, capture_output=True) return p.stdout.decode() print(pip_list())

As @Aaron mentions:

The officially recommended way to install packages from a script is by calling pip's command-line interface via a subprocess. Most other answers presented here are not supported by pip. Furthermore since pip v10, all code has been moved to pip._internal precisely in order to make it clear to users that programmatic use of pip is not allowed.

Use sys.executable to ensure that you will call the same pip associated with the current runtime.

Improve this answer

Comments

3

Use os module or system module 

import os import subprocess as su os.system("pip list") su.call(["pip","list"])
Improve this answer

3 Comments

The most useful answer. But the output of either of these calls is an int, why?
This would be useful for returning conda list as well, but same int issue.
@VaidøtasI. @MinnegapolisCoder9 use pip_list = os.popen('pip list').read() to assign output of the call to a variable. (Same holds for conda list) On unix the return value of os.system is the exit status of the call.
0

EDIT: I finally write a package to handle this called pru. GitHub repo at here. Usage as simple as,

pru

or in explicit way,

pru "requirements.txt" upgrade_requirements --cmd "pip install --upgrade"

OLD ANSWER

After testing some solution here that is either very slow, deprecated, or return error in my Python 3.10, I'm using this solution:

Note

This function can retrieve all installed package with version installed.

import pkg_resources def get_installed_packages(): installed_packages = [] for package in pkg_resources.working_set: installed_packages.append(package.key) return installed_packages def get_package_version(package_name): try: return pkg_resources.get_distribution(package_name).version except pkg_resources.DistributionNotFound: return None # Get a list of all installed packages installed_packages = get_installed_packages() # Iterate over the installed packages and get their versions package_versions = {} for package_name in installed_packages: version = get_package_version(package_name) package_versions[package_name] = version # Print the package versions for package_name, version in package_versions.items(): print(f"{package_name} - {version}")

Example output:

xlsxwriter - 3.0.9 argcomplete - 2.0.0 comm - 0.1.2 debugpy - 1.6.6

Combined with regex to list package name, we can check all installed version from requirements that is actually installed in the machine.

import re def extract_package_names(file_path): with open(file_path, 'r') as file: requirements = file.readlines() package_names = [] for requirement in requirements: match = re.search(r'^([\w.-]+)', requirement) if match: package_names.append(match.group(1)) return package_names # Example usage file_path = 'requirements.txt' package_names = extract_package_names(file_path) print(package_names)

Example output:

['numpy', 'pandas', 'xlsxwriter']

The two combined:

file_path = 'requirements.txt' package_names = extract_package_names(file_path) packages = {} for package_name in package_names: packages[package_name] = get_package_version(package_name) packages

Note

DISCLAIMER: This code is initially written with the help of ChatGPT 3.5

Improve this answer

Comments

-1

For completeness, here's vittore's pip.main() idea fleshed out with the capture of stdout. Of course using get_installed_distributions() is the preferred solution.

import contextlib @contextlib.contextmanager def capture(): import sys from cStringIO import StringIO oldout,olderr = sys.stdout, sys.stderr try: out=[StringIO(), StringIO()] sys.stdout,sys.stderr = out yield out finally: sys.stdout,sys.stderr = oldout, olderr out[0] = out[0].getvalue() out[1] = out[1].getvalue() with capture() as out: import pip pip.main(['list']) print out ['awscli (1.7.45)\nboto (2.38.0) ...
Improve this answer

1 Comment

pip doesn't support main anymore - see stackoverflow.com/questions/49839610/…

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.