Is there an easier way to package with Python?

Question

I tried to package a django app today. It's a big baby, and with the setup file, I have to manually write all packages and sub packages in the 'package' parameter. Then I have to find a way to copy fixtures, htmls / Css / image files, documentations, etc.

It's a terrible way to work. We are computer scientists, we automatize, doing this makes no sense.

And what when I change my app structure ? I have to rewrite the setup.py.

Is there a better way ? Some tool to automate that ? I can't believe a language than value developer time like Python makes packaging such a chore.

I want to be able to eventually install the app using a simple pip install. I know about build out, but it's not much simpler, and is not pip friendly.

jathanism · Accepted Answer · 2010-10-07 21:13:15Z

5

At the very least if you use setuptools (an alternative to the stdlib's distutils) you get an awesome function called find_packages() which when ran from the package root returns a list of package names in dot-notation suitable for the packages parameter.

Here is an example:

# setup.py from setuptools import find_packages, setup setup( #... packages=find_packages(exclude='tests'), #... )

p.s. Packaging sucks in every language and every system. It sucks no matter how you slice it.

edited Oct 7, 2010 at 21:13

answered Oct 7, 2010 at 21:02

jathanism

33.9k9 gold badges72 silver badges89 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Bite code Over a year ago

Do you have such a tip for satic files such as po, html/css and doc that I have to include manually as well ? Maybe setuptools.findall ? If you do, I accept your anwser.

jathanism Over a year ago

You can specify those using the package_data (docs.python.org/distutils/…) or data_files.(docs.python.org/distutils/…) parameters. For the example, if data_files is set to [('share/mypackage', ['test.html'])], upon installation test.html will be copied into /usr/share/mypackage (assuming your prefix is /usr/share).

Bite code Over a year ago

It's not a silver bullet, but it's the best anwset so I'll accept it.

Jonny Buchanan · Accepted Answer · 2010-11-25 19:36:55Z

I've been through this pain myself today. I used the following, yoinked straight from Django's setup.py, which walks the app's filesystem looking for packages and data files (assuming you never mix the two):

import os from distutils.command.install import INSTALL_SCHEMES def fullsplit(path, result=None): """ Split a pathname into components (the opposite of os.path.join) in a platform-neutral way. """ if result is None: result = [] head, tail = os.path.split(path) if head == '': return [tail] + result if head == path: return result return fullsplit(head, [tail] + result) # Tell distutils to put the data_files in platform-specific installation # locations. See here for an explanation: # http://groups.google.com/group/comp.lang.python/browse_thread/thread/35ec7b2fed36eaec/2105ee4d9e8042cb for scheme in INSTALL_SCHEMES.values(): scheme['data'] = scheme['purelib'] # Compile the list of packages available, because distutils doesn't have # an easy way to do this. packages, data_files = [], [] root_dir = os.path.dirname(__file__) if root_dir != '': os.chdir(root_dir) myapp_dir = 'myapp' for dirpath, dirnames, filenames in os.walk(myapp_dir): # Ignore dirnames that start with '.' for i, dirname in enumerate(dirnames): if dirname.startswith('.'): del dirnames[i] if '__init__.py' in filenames: packages.append('.'.join(fullsplit(dirpath))) elif filenames: data_files.append([dirpath, [os.path.join(dirpath, f) for f in filenames]])

This is brillant ! I will try this. If it works, I'll accept the anwser.

ewall · Accepted Answer · 2010-10-10 02:01:24Z

0

I think the tool you are looking for is Buildout. There lots of places where you can learn more about it, from SlideShare to Pycon videos.

Other similar or related tools which you might want to check out include virtualenv, Fabric, and PIP.

edited Oct 10, 2010 at 2:01

answered Oct 8, 2010 at 3:34

ewall

28.3k16 gold badges74 silver badges85 bronze badges

3 Comments

Bite code Over a year ago

Can you pip install something packaged with buildout ? I don't want to provide a package that need more that a pip install in a virtualenv.

Bite code Over a year ago

Searched. Yes you can. But my, buildout is everything but easy to use. It's huge, with spare documentation and based on text file instead of python files.

ewall Over a year ago

I can't disagree with your description, I think you're spot on. I just don't know what else to recommend that fits any better :(

Yuval Adam · Accepted Answer · 2010-11-25 19:57:03Z

I've been doing a bit of research myself on Django deployment methods recently.

I found these two resources very useful:

http://www.caktusgroup.com/blog/2010/04/22/basic-django-deployment-with-virtualenv-fabric-pip-and-rsync/ - A basic overview of how to use virtualenv, pip and fabric to do easy Django deployment from dev machine to production.
https://github.com/jacobian/django-deployment-workshop/ - Jacob(ian) did a Django deployment workshop, with all the code posted on GitHub. All the necessary links are there as well.

This is not helpfull at all for this question. Deployment is perfectly ok to me. Packaging is the annoying part.
The links I posted give some good insight on packaging methods as well. I'm sorry you didn't find the links useful and decided to downvote.

Collectives™ on Stack Overflow

Is there an easier way to package with Python?

4 Answers 4

3 Comments

1 Comment

3 Comments

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

1 Comment

3 Comments

2 Comments

Related