Skipping over failed imports until they are needed (if ever)

This answer builds upon the solution presented by Mathias Ettinger

If you want to get rid of the try: ... except: ... boilerplate you could implement some kind of custom loader for your engines.

This is something I wrote quickly, which is likely far from perfect, but it works.

in __init__.py:

import importlib def mock_missing(name, msg): def init(self, *args, **kwargs): raise ImportError( f'The class {name} you tried to call is not importable; ' f'this is likely due to it not being installed. ' f'Original reason: {msg}') return type(name, (), {'__init__': init}) def try_load(engine, module): """Try to load an engine, return a mock class if it was not found Inspired by https://stackoverflow.com/a/10675081 """ try: module = importlib.import_module(module, __name__) return getattr(module, engine) except (ModuleNotFoundError, AttributeError) as ex: return mock_missing(engine, msg=str(ex)) 

To test it:

from engines import try_load PSI4 = try_load("PSI4", ".psi4") a = PSI4() # module does not exist try: PSI5 = try_load("PSI5", ".PSI5") b = PSI5() raise AssertionError("This should have failed!") except ImportError as ex: print(str(ex)) # class does not exist in module try: PSI6 = try_load("PSI6", ".psi4") c = PSI6() raise AssertionError("This should have failed!") except ImportError as ex: print(str(ex)) 

This outputs:

The class PSI5 you tried to call is not importable; this is likely due to it not being installed. Original reason: No module named 'engines.psi5' The class PSI6 you tried to call is not importable; this is likely due to it not being installed. Original reason: module 'engines.psi4' has no attribute 'PSI6' 

That's an odd use of the word "default", which, etymologically speaking, means to remove failure, and generally refers to a fallback setting that "just works". I suggest calling it Missing instead.

Whenever you find yourself trying to replace a standard language feature, the first thing you should think is,

There's probably something I'm missing. What's the standard approach to this kind of problem?

Your code feels inelegant because you're not using a standard approach. When your code starts to feel weird like that, it's time to take a step back and look for the standard approaches and tools for the kind of problem you're facing. Code becomes elegant when it's something you would expect to see, and you can create that expectation by adhering to the norms and paradigm of the language and tools you're using.

In this case, the standard approach is not to import something until you're actually going to use it. In this context, that means that instead of importing all these classes into your package (__init__.py), let callers decide what engines they require and import them. I've run into situations where my code fails late into its execution because of a dependency that isn't properly installed; it is frustrating and wasted my time. You want your design to encourage fail fast. Having your callers explicitly import the engines from your package that they need will save them trouble.

Example calling code:

from Program.engines.psi4 import PSI4 PSI4('example_molecule').generate_input() 

If other parts of your code use the engines, then let the caller inject the engine as a dependency:

class ProgramWorker(object): def __init__(self, engine): if engine is None: raise ValueError('engine must have a value') self.engine = engine def do_something(self, x, y): intermediate = self.engine.calculate_something(x, y) # do more with intermediate return result 

Parameterizing the engine at initialization time like this is an example of dependency inversion, and needing to select one of several implementations is one of the few situations in Python where a more OO approach makes a lot of sense. Instead of having the class find its dependencies, you let callers inject the dependency as an argument. This gives callers greater flexibility, more control, and more transparency into the prerequisites of your code. In particular, they retain all the fail fast properties of normal imports and are given full control over what to load and what not to. Callers would use it like this:

from Program.engines.psi4 import PSI4 from Program.somewhere import ProgramWorker eng = PSI4('example_molecule') w = ProgramWorker(eng) w.do_something() 

This wouldn't work if your engines don't have a common interface, but in that case, you shouldn't be using an inheritance structure. If different engines are required for different functionality, then it would be better just to split up the functionality into separate modules based on dependencies.

run.py Command line

Assuming your run.py is a command line application, then you can control the engine with a command line option. Combined with the above, it would then look something like this:

import click @click.command() @click.options('--engine', type=click.Choice(['psi4', 'rdkit', ...])), default='psi4', help='The engine to use for computations') def main(engine): if engine_name == 'psi4': from .engine.psi4 import PSI4 eng = PSI4('example_molecule') elif engine_name == 'rdkit' from .engine.rdkit import RDKit eng = RDKit('example_molecule') elif engine_name == ...: ... else: raise ValueError('Unknown engine: ' + engine) eng.generate_input() if __name__ == '__main__' main() 

(You don't have to use click; it's just the easiest command line parser I know of.)

Here, you can just import the engine inline because you need to delay the import until the option has been parsed. This is okay since this block of code, being the command line implementation, will necessarily execute almost immediately after being invoked.

If you need this "select an engine" functionality in multiple places, you could encode that in your engines package (the __init__.py) as a function, but it's unlikely that you have several different scripts that need this as its main purpose is to support the command line. If you have multiple commands that use an engine, you can use a click.group to enable them to share these parameters and this code.

Naming

You should probably rename your modules to avoid conflicts with the names of packages you're depending on: psi4engine or psi4_engine, rdkitengine or rdkit_engine

I would also suffix your engine classes with Engine to avoid possible name conflicts and to increase the clarity of your code: PSI4Engine, RDKitEngine.

Optional dependencies

You might already be doing this if you're creating a package from your code, but in case you're not, you should be declaring packages like PSI4 and RDKit as optional dependencies.