Best way to package a Python library that includes a C shared library?

Question

I have written a library whose main functionality is implemented in C (speed is critical), with a thin Python layer around it to deal with the ctypes nastiness.

I'm coming to package it and I'm wondering how I might best go about this. The code it must interface with is a shared library. I have a Makefile which builds the C code and creates the .so file, but I don't know how I compile this via distutils. Should I just call out to make with subprocess by overriding the install command (if so, is install the place for this, or is build more appropriate?)

Update: I want to note that this is not a Python extension. That is, the C library contains no code to itself interact with the Python runtime. Python is making foreign function calls to a straight C shared library.

Have you got any solution for your problem? I'm stuck with the same question and desperate for an answer ;) — MrLeeh
– MrLeeh, Commented Jul 15, 2017 at 15:00
This is a very long time ago but it looks like I solved this by shelling out to make. This is the setup.py from that library: gist.github.com/obeattie/3d491ec4c29b1d4b46387b285c91ca89 — obeattie
– obeattie, Commented Jul 17, 2017 at 14:52
I've tried doing what you have in this gist, but... how did you get the .so you lined with to get installed in any location Python / ld will look for it? — wvxvw
– wvxvw, Commented Jan 24, 2018 at 16:24

Sam Hartman · Accepted Answer · 2014-06-05 17:28:16Z

I'd consider building the python module as a subproject of a normal shared library build. So, use automake, autoconf or something like that to build the shared library, have a python_bindings directory with a setup.py and your python module.

Community · Accepted Answer · 2017-05-23 12:34:22Z

I had a similar need and found this answer helpful: Python setup.py call makefile don't include binaries.

I have an ANSI C library in the src directory of my distribution. In the src directory is a Makefile that builds a file called liblsd.so in my package directory (lsd). I call this in setup.py and then tell setup to include the library file using the package_data argument.

import os.path import subprocess from setuptools import setup with open(os.path.join(os.path.dirname(__file__), 'README.rst')) as f: readme = f.read() subprocess.call(['make', '-C', 'src']) setup(name='LSD', version='0.0.1', description='Python bindings for the LSD line segment detector.', long_description=readme, author='Geoff Hing', author_email='[email protected]', url='https://github.com/ghing/python-lsd', packages=['lsd'], package_data={'lsd': ['liblsd.so']}, include_package_data=True, classifiers=[ 'Development Status :: 1 - Planning', 'Intended Audience :: Developers', 'License :: OSI Approved :: MIT License', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Programming Language :: C', ], )

You should at least sub class distutils.command.build and override the run method, otherwise the make call will happen for every type of call to setup.py where you may not want to recompile the library (e.g. 'check' or 'upload')

ElmoVanKielmo · Accepted Answer · 2014-06-05 15:29:07Z

Given that you followed the instructions on how to create Python extensions in C, you should just enlist the extension modules like in this documentation.
So the setup.py script of your library should look like this:

from distutils.core import setup, Extension setup( name='your_python_library', version='1.0', ext_modules=[Extension('your_c_extension', ['your_c_extension.c'])], )

and distutils knows how to compile your extension to C shared library and moreover where to put it.

Of course I have no further information about your library, so you probably want to add more arguments to setup(...) call.

Seems like... pythonhosted.org/Distutils2/distutils/… ...exactly the same :)
It is not a Python extension. This is a pure C library (which contains no code itself to interact with the Python runtime), but Python makes calls to it via ctypes. It's potentially possible to wrap the C shared library in a Python extension module, but I'd prefer to keep the current ctypes approach.

Collectives™ on Stack Overflow

Best way to package a Python library that includes a C shared library?

3 Answers 3

Comments

1 Comment

3 Comments

Linked

Hot Network Questions