Best practice for importing external module in python

Question

Let me ask the question by a simplified example. Assume I have a python2 script called example.py in a directory with other script I wrote called utils.py. In example.py, I need to import an external module which is placed under the path /A/B/external.py. 'A' and 'B' are just folders, not packages (without __ init__.py). Finally, assume that there is a utils.py file under /A/B/ as well, and it is imported by external.py.

If I import the external module by the way I'm familiar with, I'll do:

import sys sys.path.append('/A/B/') import external

the problem is that when I'll try to import my utils module, python will find it in sys.modules dict (since external imported it) and thus won't import my script but only the one under /A/B which was already imported.

Now, there are workarounds. I may change the name of the module, or put the external module in a package, etc. But all of these feel dirty. I couldn't find what is the 'pythonic' best practice to do such things. As some of you might guess, it could lead to very annoying bugs with more complex cases. In addition, the way I'm familiar with is not acceptable by PEP8 as there are import commands not at the top of the module.

In conclusion, I'm looking for the best practice to do such things, thanks.

Why external has to be located in another directory? Could it be a subdirectory of your workspace? — Blownhither Ma
– Blownhither Ma, Commented Oct 19, 2018 at 6:19
@BlownhitherMa no, it is another library coded by other group, which is frequently updated and I want to use their most updated version (so I don't want to copy the code to my workspace) — Ron U
– Ron U, Commented Oct 23, 2018 at 16:21

Ron U · Accepted Answer · 2020-08-01 14:36:34Z

Although almost two years had passed, I want to describe the correct answer as far as I understand it now. TL;DR: The best approach is to avoid manipulating sys.path at all(!) and use distribution-packaging mechanisms instead.

First, let us distinguish between distribution-package and import-package. An import-package is basically a directory that contains pythonic modules (i.e. .py/.pyc files) and an __init__.py file to declare that this folder is indeed a package. In the question's example this is the directory that included example.py and utils.py. Opposed to that, a distribution-package is:

A versioned archive file that contains Python packages, modules, and other resource files that are used to distribute a Release.

sources: PyPA user guide , glossary .

Another term that should be known is the script. A python script is the __main__ python file, that is directly run via the interpreter (while the other python files are called modules). source: python's docs.

In your own import-package you shouldn't edit sys.path because the script file imports modules relative to its file's location, and that's the desired behavior. If you're intend of using another module from outside your import-package (and let me make it clear that this is not the case that you import from another import-package that is inside your current import-package, this is a normal behavior and works good), then you should import it as an installed distribution-package.

What I mean by that, is that the external code (in the example, the external.py module that is located under A/B/) should be available as a distribution-package (its maintainers should upload it as a package to some distribution mechanism, i.e. PyPi, and that could very much be you, but as another package developer). Then - you as the external package user should install it (e.g. with pip install on the wheel file of that external distribution-package) and then it will be added and managed by your environment so you could import external and it would work without collisions.

Collectives™ on Stack Overflow

Best practice for importing external module in python

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related