29

I'm working with a project that contains about 30 unique modules. It wasn't designed too well, so it's common that I create circular imports when adding some new functionality to the project.

Of course, when I add the circular import, I'm unaware of it. Sometimes it's pretty obvious I've made a circular import when I get an error like AttributeError: 'module' object has no attribute 'attribute' where I clearly defined 'attribute'. But other times, the code doesn't throw exceptions because of the way it's used.

So, to my question:

Is it possible to programmatically detect when and where a circular import is occuring?

The only solution I can think of so far is to have a module importTracking that contains a dict importingModules, a function importInProgress(file), which increments importingModules[file], and throws an error if it's greater than 1, and a function importComplete(file) which decrements importingModules[file]. All other modules would look like:

import importTracking importTracking.importInProgress(__file__) #module code goes here. importTracking.importComplete(__file__)

But that looks really nasty, there's got to be a better way to do it, right?

Improve this question
1
  • How do you know you create a circular import? If you're unaware of it, what's the problem? Please be specific about the problems. Commented Mar 9, 2010 at 3:25

4 Answers 4

Reset to default
10

To avoid having to alter every module, you could stick your import-tracking functionality in a import hook, or in a customized __import__ you could stick in the built-ins -- the latter, for once, might work better, because __import__ gets called even if the module getting imported is already in sys.modules, which is the case during circular imports.

For the implementation I'd simply use a set of the modules "in the process of being imported", something like (benjaoming edit: Inserting a working snippet derived from original):

beingimported = set() originalimport = __import__ def newimport(modulename, *args, **kwargs): if modulename in beingimported: print "Importing in circles", modulename, args, kwargs print " Import stack trace -> ", beingimported # sys.exit(1) # Normally exiting is a bad idea. beingimported.add(modulename) result = originalimport(modulename, *args, **kwargs) if modulename in beingimported: beingimported.remove(modulename) return result import __builtin__ __builtin__.__import__ = newimport
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thanks, but: This breaks the original program in the two cases: 1) class Somethink(object): myimport = __import__ ... self.myimport(name) (self object is in modulename and the real module name is in etc, like in django.conf)` 2) try: import some_module \n except: some_module = None (remains in the set and it report false positive next time)
If a package foo_package has a module "foo_package/a.py" which imports from foo_package import b than it is detected as a circular import of "foo_package" even if "foo_package/__init__.py" is an empty file. This is false. The important module names are in the ignored "etc etc" not in modulename in the case of "from <package> import <module>"!
what I see is empty modulename (i.e., it is a string of length 0).
Simple check for circular imports might be imlemented with dedicated file: import main_app; print(id(main_app))
3

Circular imports in Python are not like PHP includes.

Python imported modules are loaded the first time into an import "handler", and kept there for the duration of the process. This handler assigns names in the local namespace for whatever is imported from that module, for every subsequent import. A module is unique, and a reference to that module name will always point to the same loaded module, regardless of where it was imported.

So if you have a circular module import, the loading of each file will happen once, and then each module will have names relating to the other module created into its namespace.

There could of course be problems when referring to specific names within both modules (when the circular imports occur BEFORE the class/function definitions that are referenced in the imports of the opposite modules), but you'll get an error if that happens.

Improve this answer

1 Comment

Hey. Since imports usually happen at the top of files, isn't the situation you describe in the last paragraph the usual case. So circular imports usually throw an exception, if you can be sure you've actually imported one of the participating modules. Or have I misunderstood?
1

Not all circular imports are a problem, as you've found when an exception is not thrown.

When they are a problem, you'll get an exception the next time you try to run any of your tests. You can change the code when this happens.

I don't see any change required from this situation.

Example of when it's not a problem:

a.py

import b a = 42 def f(): return b.b

b.py

import a b = 42 def f(): return a.a
Improve this answer

2 Comments

Well, as I said, it was a poorly designed project. I have a module which holds global settings for the project. I have other modules which configure preset settings. It's possible that those modules trigger a circular import, and don't through any errors, but just don't do their job.
Circular imports will kill your ability to build docs using sphinx and autodoc.
0

import uses __builtin__.__import__(), so if you monkeypatch that then every import everywhere will pick up the changes. Note that a circular import is not necessarily a problem though.

Improve this answer

1 Comment

Circular imports will kill your ability to build docs using sphinx and autodoc.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.