3

I am developing a package that has a file structure similar to the following:

test.py package/ __init__.py foo_module.py example_module.py

If I call import package in test.py, I want the package module to appear similar to this:

>>> vars(package) mapping_proxy({foo: <function foo at 0x…}, {example: <function example at 0x…})

In other words, I want the members of all modules in package to be in package's namespace, and I do not want the modules themselves to be in the namespace. package is not a sub-package.

Let's say my files look like this:

foo_module.py:

def foo(bar): return bar

example_module.py:

def example(arg): return foo(arg)

test.py:

print(example('derp'))

How do I structure the import statements in test.py, example_module.py, and __init__.py to work from outside the package directory (i.e. test.py) and within the package itself (i.e. foo_module.py and example_module.py)? Everything I try gives Parent module '' not loaded, cannot perform relative import or ImportError: No module named 'module_name'.

Also, as a side-note (as per PEP 8): "Relative imports for intra-package imports are highly discouraged. Always use the absolute package path for all imports. Even now that PEP 328 is fully implemented in Python 2.5, its style of explicit relative imports is actively discouraged; absolute imports are more portable and usually more readable."

I am using Python 3.3.

Improve this question
1
  • I just put it here blindly without try: how about from package import *? Commented Jan 20, 2013 at 16:45

2 Answers 2

Reset to default
3

I think you can get the values you need without cluttering up your namespace, by using from module import name style imports. I think these imports will work for what you are asking for:

Imports for example_module.py:

from package.foo_module import foo

Imports for __init__.py:

from package.foo_module import foo from package.example_module import example __all__ = [foo, example] # not strictly necessary, but makes clear what is public

Imports for test.py:

from package import example

Note that this only works if you're running test.py (or something else at the same level of the package hierarchy). Otherwise you'd need to make sure the folder containing package is in the python module search path (either by installing the package somewhere Python will look for it, or by adding the appropriate folder to sys.path).

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

+1: e.g., multiprocessing package from stdlib uses this technique
If I execute example_module.py or __init__.py directly, I get an import error, but test.py works. Getting test.py is obviously the most import one to get working, but is there any way (without a try/catch) to get the other two working as well? I suppose this is why I posted this question in the first place.
This also keeps the modules in the package's namespace. Now that I think of it, is this is a requirement by Python?
@TylerCrompton: In general, no, there is no good way to make it possible to run a module from within a package as a script. It's possible to put something like sys.path.append("..") in example_module.py, but it's a hack and might not work perfectly (you could also try the -m command line option). The Python developers have pretty much said that running scripts within packages is fundamentally broken design and that they don't intend to support it ever (despite it being frequently requested by new Python programmers).
I'm not sure I understand your question about the package namespace. If you put import package.foo_module you will indeed get the foo_module module, but if you just import package, there won't automatically be anything at package.foo_module (only at package.foo and package.bar). Even from package import * (which is usually discouraged) should do the right thing.
2

I want the members of all modules in package to be in package's namespace, and I do not want the modules themselves to be in the namespace.

I was able to do that by adapting something I've used in Python 2 to automatically import plug-ins to also work in Python 3.

In a nutshell, here's how it works:

  1. The package's __init__.py file imports all the other Python files in the same package directory except for those whose names start with an '_' (underscore) character.

  2. It then adds any names in the imported module's namespace to that of __init__ module's (which is also the package's namespace). Note I had to make the example_module module explicitly import foo from the .foo_module.

One important aspect of doing things this way is realizing that it's dynamic and doesn't require the package module names to be hardcoded into the __init__.py file. Of course this requires more code to accomplish, but also makes it very generic and able to work with just about any (single-level) package — since it will automatically import new modules when they're added and no longer attempt to import any removed from the directory.

test.py:

from package import * print(example('derp'))

__init__.py:

def _import_all_modules(): """ Dynamically imports all modules in this package. """ import traceback import os global __all__ __all__ = [] globals_, locals_ = globals(), locals() # Dynamically import all the package modules in this file's directory. for filename in os.listdir(__name__): # Process all python files in directory that don't start # with underscore (which also prevents this module from # importing itself). if filename[0] != '_' and filename.split('.')[-1] in ('py', 'pyw'): modulename = filename.split('.')[0] # Filename sans extension. package_module = '.'.join([__name__, modulename]) try: module = __import__(package_module, globals_, locals_, [modulename]) except: traceback.print_exc() raise for name in module.__dict__: if not name.startswith('_'): globals_[name] = module.__dict__[name] __all__.append(name) _import_all_modules()

foo_module.py:

def foo(bar): return bar

example_module.py:

from .foo_module import foo # added def example(arg): return foo(arg)
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.