Skip to main content
Added a bit of clarification (and suggestion on using ImportError instead of assert)
Source Link
DavidW
  • 31.2k
  • 7
  • 64
  • 99

According to this really old mailing list post it works if you also have an __init__.py file (the __init__.py file is not used, but seems to be necessary for the directory to be treated as a module, and hence the __init__.so file to be loaded).

If I add __init__.py:

# an assertexception just to confirm that the .so file is loaded instead of the .py file assertraise FalseImportError("__init__.py loaded when __init__.so should have been loaded") 

then your example works on Linux Python 2.7.3:

$ python -c 'import foo; foo.hello_world()' hello world blah 

This has all the signs of a buggy corner case so probably isn't recommended. Note that on Windows this doesn't seem to work for me giving

ImportError: DLL load failed: %1 is not a valid Win32 application. 

Addendum (for a little extra context):

This behaviour doesn't seem to be explicitly documented. In the original description of packages from around Python 1.5 era they say:

without the __init__.py, a directory is not recognized as a package

and

Tip: the search order is determined by the list of suffixes returned by the function imp.get_suffixes(). Usually the suffixes are searched in the following order: ".so", "module.so", ".py", ".pyc". Directories don't explicitly occur in this list, but precede all entries in it.

The observed behaviour is certainly consistent with this — __init__.py needed to treat a directory as a package, but .so file is loaded in preference to .py file — but it's hardly unambiguous.

From a Cython point of view this behaviour seems to be been used to compile the standard library (in which case __init__.py would always have been present), or in the testcases given https://github.com/cython/cython/blob/master/tests/build/package_compilation.srctree (and a few other examples too). In these the "srctree" file looks to be expanded into a variety of folders containing __init__.py (and other files) then compiled. It's possible that only having __init__.so was simply never tested.

According to this really old mailing list post it works if you also have an __init__.py file (the __init__.py file is not used, but seems to be necessary for the directory to be treated as a module, and hence the __init__.so file to be loaded).

If I add __init__.py:

# an assert just to confirm that the .so file is loaded instead of the .py file assert False 

then your example works on Linux Python 2.7.3:

$ python -c 'import foo; foo.hello_world()' hello world blah 

This has all the signs of a buggy corner case so probably isn't recommended. Note that on Windows this doesn't seem to work for me giving

ImportError: DLL load failed: %1 is not a valid Win32 application. 

According to this really old mailing list post it works if you also have an __init__.py file (the __init__.py file is not used, but seems to be necessary for the directory to be treated as a module, and hence the __init__.so file to be loaded).

If I add __init__.py:

# an exception just to confirm that the .so file is loaded instead of the .py file raise ImportError("__init__.py loaded when __init__.so should have been loaded") 

then your example works on Linux Python 2.7.3:

$ python -c 'import foo; foo.hello_world()' hello world blah 

This has all the signs of a buggy corner case so probably isn't recommended. Note that on Windows this doesn't seem to work for me giving

ImportError: DLL load failed: %1 is not a valid Win32 application. 

Addendum (for a little extra context):

This behaviour doesn't seem to be explicitly documented. In the original description of packages from around Python 1.5 era they say:

without the __init__.py, a directory is not recognized as a package

and

Tip: the search order is determined by the list of suffixes returned by the function imp.get_suffixes(). Usually the suffixes are searched in the following order: ".so", "module.so", ".py", ".pyc". Directories don't explicitly occur in this list, but precede all entries in it.

The observed behaviour is certainly consistent with this — __init__.py needed to treat a directory as a package, but .so file is loaded in preference to .py file — but it's hardly unambiguous.

From a Cython point of view this behaviour seems to be been used to compile the standard library (in which case __init__.py would always have been present), or in the testcases given https://github.com/cython/cython/blob/master/tests/build/package_compilation.srctree (and a few other examples too). In these the "srctree" file looks to be expanded into a variety of folders containing __init__.py (and other files) then compiled. It's possible that only having __init__.so was simply never tested.

link to the mail.python.org archive of the message instead of a 3rd party mailing list archive site
Source Link
Richard Hansen
  • 54.8k
  • 20
  • 93
  • 101

According to this really old newsgroup postthis really old mailing list post it works if you also have an __init__.py file (the __init__.py file is not used, but seems to be necessary for the directory to be treated as a module, and hence the __init__.so file to be loaded).

If I add __init__.py:

# an assert just to confirm that the .so file is loaded instead of the .py file assert False 

then your example works on Linux Python 2.7.3:

$ python -c 'import foo; foo.hello_world()' hello world blah 

This has all the signs of a buggy corner case so probably isn't recommended. Note that on Windows this doesn't seem to work for me giving

ImportError: DLL load failed: %1 is not a valid Win32 application. 

According to this really old newsgroup post it works if you also have an __init__.py file (the __init__.py file is not used, but seems to be necessary for the directory to be treated as a module, and hence the __init__.so file to be loaded).

If I add __init__.py:

# an assert just to confirm that the .so file is loaded instead of the .py file assert False 

then your example works on Linux Python 2.7.3:

$ python -c 'import foo; foo.hello_world()' hello world blah 

This has all the signs of a buggy corner case so probably isn't recommended. Note that on Windows this doesn't seem to work for me giving

ImportError: DLL load failed: %1 is not a valid Win32 application. 

According to this really old mailing list post it works if you also have an __init__.py file (the __init__.py file is not used, but seems to be necessary for the directory to be treated as a module, and hence the __init__.so file to be loaded).

If I add __init__.py:

# an assert just to confirm that the .so file is loaded instead of the .py file assert False 

then your example works on Linux Python 2.7.3:

$ python -c 'import foo; foo.hello_world()' hello world blah 

This has all the signs of a buggy corner case so probably isn't recommended. Note that on Windows this doesn't seem to work for me giving

ImportError: DLL load failed: %1 is not a valid Win32 application. 
Bounty Awarded with 50 reputation awarded by Neithrik
Short clarification
Source Link
DavidW
  • 31.2k
  • 7
  • 64
  • 99

According to this really old newsgroup post it works if you also have an __init__.py file (but it loadsthe __init__.py file is not used, but seems to be necessary for the directory to be treated as a module, and hence the .so__init__.so file to be loaded).

If I add __init__.py:

# an assert just to confirm that the .so file is loaded instead of the .py file assert False 

then your example works on Linux Python 2.7.3:

$ python -c 'import foo; foo.hello_world()' hello world blah 

This has all the signs of a buggy corner case so probably isn't recommended. Note that on Windows this doesn't seem to work for me giving

ImportError: DLL load failed: %1 is not a valid Win32 application. 

According to this really old newsgroup post it works if you also have an __init__.py file (but it loads the .so file).

If I add __init__.py:

# an assert just to confirm that the .so file is loaded instead of the .py file assert False 

then your example works on Linux Python 2.7.3:

$ python -c 'import foo; foo.hello_world()' hello world blah 

This has all the signs of a buggy corner case so probably isn't recommended. Note that on Windows this doesn't seem to work for me giving

ImportError: DLL load failed: %1 is not a valid Win32 application. 

According to this really old newsgroup post it works if you also have an __init__.py file (the __init__.py file is not used, but seems to be necessary for the directory to be treated as a module, and hence the __init__.so file to be loaded).

If I add __init__.py:

# an assert just to confirm that the .so file is loaded instead of the .py file assert False 

then your example works on Linux Python 2.7.3:

$ python -c 'import foo; foo.hello_world()' hello world blah 

This has all the signs of a buggy corner case so probably isn't recommended. Note that on Windows this doesn't seem to work for me giving

ImportError: DLL load failed: %1 is not a valid Win32 application. 
Source Link
DavidW
  • 31.2k
  • 7
  • 64
  • 99
Loading