12

How do I get the current topmost package, i.e., the name defined in setup.py?

Here is my tree:

. |-- README.md |-- the_project_name_for_this_pkg | |-- __init__.py | |-- __main__.py | |-- _config | | `-- foo.conf | |-- _data | | `-- logging.yml | `-- tests | |-- __init__.py | `-- test_foo.py <--- # executing from here |-- requirements.txt `-- setup.py 4 directories, 9 files

The only solution I've gotten to work so far is this:

import os import sys os.path.basename(sys.path[1])

But this is obviously a bad solution. Other solutions like having a __name__ in my uppermost __init__.py file and using ast.parse to read in the relevant section of setup.py also seems cumbersome.

Other solutions I've tried—by calling them within a unittest.TestCase inheriting class in my tests python [sub]package—include checking sys.modules[__name__], inspect.getmodule & inspect.stack, as well as the answers to these questions:

BTW: In case you were wondering why I want the package name… it's so I can run things like:

import pkg_resources version = pkg_resources.require('the_project_name_for_this_pkg')[0].version data_file = path.join(resource_filename('the_project_name_for_this_pkg', '__init__.py'), '_config', 'data_file.txt')
Improve this question
5
  • 3
    Seems like you are mixing up the name of the project and the name of a top-level package. They are often the same, but there are still many cases when they do not match. Commented Feb 22, 2020 at 10:41
  • Within test_foo.py it should already be defined within the package itself, just use the package name directly there. Commented Feb 22, 2020 at 10:50
  • @metatoaster Where is it defined in the package itself? - I couldn't find it in magic variables, inspect, or sys.modules. Commented Feb 22, 2020 at 13:05
  • It would be whatever you defined in setup.py; the package system is bolted onto Python after the fact, and given that string for the package name don't typically change, it's a lot less hassle to just hardcode that same str value into test_foo.py that rather than trying to come up with ways to resolve that in Python. Commented Feb 23, 2020 at 0:10
  • Alternatively, you could leverage EntryPoints - create your own at setup.py that would have values that reference the module that has the resources you need. From your code (or any other package, fro that matter) simply query for that, and use that result to feed into resource_filename to get what is needed. Commented Feb 23, 2020 at 0:15

2 Answers 2

Reset to default
6

Not entirely sure what the larger goal is, but maybe you could be interested in reading about importlib.resources as well as importlib.metadata.

Something like the following:

import importlib.metadata import importlib.resources version = importlib.metadata.version('SomeProject') data = importlib.resources.files('top_level_package.sub_package').joinpath('file.txt').read_text()

And more generally, it is near impossible (or not worth the amount of work) to 100% reliably detect the name of the project (SomeProject) from within the code. It is easier to just hard-code it.

Nevertheless here are some techniques, and ideas to retrieve the name of the project from one of its modules:

Update:

I believe some function like the following should return the name of the installed distribution containing the current file:

import pathlib import importlib_metadata def get_project_name(): for dist in importlib_metadata.distributions(): try: relative = pathlib.Path(__file__).relative_to(dist.locate_file('')) except ValueError: pass else: if relative in dist.files: return dist.metadata['Name'] return None

Update (February 2021):

Looks like this could become easier thanks to the newly added packages_distributions() function in importlib_metadata:

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

A solution I've been working on:

from os import listdir, path from contextlib import suppress import ast def get_first_setup_py(cur_dir): if 'setup.py' in listdir(cur_dir): return path.join(cur_dir, 'setup.py') prev_dir = cur_dir cur_dir = path.realpath(path.dirname(cur_dir)) if prev_dir == cur_dir: raise StopIteration() return get_first_setup_py(cur_dir) setup_py_file_name = get_first_setup_py(path.dirname(__file__))

First pass:

def get_from_setup_py(setup_file): # mostly https://stackoverflow.com/a/47463422 import importlib.util spec = importlib.util.spec_from_file_location('setup', setup_file) setup = importlib.util.module_from_spec(spec) spec.loader.exec_module(setup) # And now access it print(setup)

That option did work. So I returned to the ast solution I referenced in the question, and got this second pass to work:

def parse_package_name_from_setup_py(setup_py_file_name): with open(setup_py_file_name, 'rt') as f: parsed_setup_py = ast.parse(f.read(), 'setup.py') # Assumes you have an `if __name__ == '__main__'` block: main_body = next(sym for sym in parsed_setup_py.body[::-1] if isinstance(sym, ast.If)).body setup_call = next(sym.value for sym in main_body[::-1] if isinstance(sym, ast.Expr) and isinstance(sym.value, ast.Call) and sym.value.func.id in frozenset(('setup', 'distutils.core.setup', 'setuptools.setup'))) package_name = next(keyword for keyword in setup_call.keywords if keyword.arg == 'name' and isinstance(keyword.value, ast.Name)) # Return the raw string if it is one if isinstance(package_name.value, ast.Str): return package_name.value.s # Otherwise it's a variable defined in the `if __name__ == '__main__'` block: elif isinstance(package_name.value, ast.Name): return next(sym.value.s for sym in main_body if isinstance(sym, ast.Assign) and isinstance(sym.value, ast.Str) and any(target.id == package_name.value.id for target in sym.targets) ) else: raise NotImplemented('Package name extraction only built for raw strings & ' 'assigment in the same scope that setup() is called')

Third pass (works for both installed and development versions):

# Originally from https://stackoverflow.com/a/56032725; # but made more concise and added support whence source class App(object): def get_app_name(self) -> str: # Iterate through all installed packages and try to find one # that has the app's file in it app_def_path = inspect.getfile(self.__class__) with suppress(FileNotFoundError): return next( (dist.project_name for dist in pkg_resources.working_set if any(app_def_path == path.normpath(path.join(dist.location, r[0])) for r in csv.reader(dist.get_metadata_lines('RECORD')))), None) or parse_package_name_from_setup_py( get_first_setup_py(path.dirname(__file__)))
Improve this answer

5 Comments

I don't see any use case for this. The setup.py file is never installed, so it simply can't be read. Sure, it looks like it works when it is executed directly from a clone of the source code repository, but once installed, it can't possibly work. Am I missing something? I don't really see the point of this code. If you want to have access to the name of the project, there are better techniques: bitbucket.org/pypa/distlib/issues/102/… - stackoverflow.com/a/22845276/11138259 - stackoverflow.com/a/56032725/11138259 - But why?
If really you want to read the name of the project from setup.py, then maybe use this technique to retrieve the metadata by executing (parts of) the setup. Or place the metadata in a setup.cfg file which is easier to parse than a setup.py.
Good point on the setup.py not being available once installed. So I'll really need someway of if conditioning and use a different solution once installed. I'll try this answer you mentioned… though I am worried that it may get confused between two packages.
You should only ever read from the installed distribution's metadata (develop and/or editable modes, also have this metadata available). If the metadata is not readable, then it's your clue that there is a packaging or installation issue, no reason to fallback to reading from setup.py. I don't see how it could get confused between two installed distributions.
@sinoroc - The fallback to read from setup.py is if the package isn't installed, or if you have cd'd to the directory of the package source and are running python -m <mod name>. Otherwise it won't pick the right file in pkg_resources.resource_filename…

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.