[sugar] Python: distutils, setuptools, packages, etc

[Dan Williams]

On Wed, 2006-09-27 at 15:23 -0500, Ian Bicking wrote:
> So, the other topic about package layout, is if and how OLPC should use 
> distutils and/or setuptools.
> 
> Right now, honestly, I don't understand the Sugar build process (not 
> jhbuild so much as the sugar package itself).  I assume it makes sense 
> to people familiar with auto* and the configure/make/make install 
> pattern.  Honestly I'm not one of those people, so maybe the current 
> system handles requirements I'm unaware of.
> 
> Anyway, the conventional way to distribute and install a package in 
> Python is with distutils.  This involves a file setup.py in the root of 
> the package, which describes the package and anything in the build 
> process (for pure-Python packages it's very short).  It handles 
> compiling extensions as well, but without as much flexibility as 
> configure, or as efficient as make.  But in practice it seems to be 
> enough flexibility, and certainly enough for OLPC.  And you just have a 
> setup.py file, without any other build-related files, which I personally 
> appreciate.

I'd rather not use distutils.  Any build system we make here hides a lot
of complexity behind the scenes, but from what I've used distutils for,
it hides way more than a standard autotools setup.  We also have to
build some C bits (bindings/ directory) and we should definitely be
using automake for that so that we pick up the correct CFLAGS and such.

I'm not really sure what the issue with autotools here is (other than
that autotools sucks in a general sense?)  Plenty of other projects make
use of it, and I don't think that just because every other python thing
uses it makes it the right choice.

Too often language-specific stuff like distutils doesn't take local
system specifics into account, and you end up having to hack around
stuff anyway.  At least with autotools we know how to do that.  We
picked autotools because lots of people know it (including us) and we
had to pick at least one evil from the pile :)

Dan

> Then the build process is:
> 
>    python setup.py install
> 
> This will build anything that needs building, and copy the result into 
> site-packages.  (If using virtual-python.py, it will install it in the 
> right location; with the current setup you'd have to give it a --prefix 
> option.)
> 
> Anyway, you guys probably all know this.  Is there a reason Sugar 
> doesn't use distutils?  I don't want to rehash old discussions, but I 
> don't know what discussions are old yet ;)
> 
> Then the second question is about using Setuptools.  Setuptools is a 
> distutils extension that's quite popular; it didn't quite make it into 
> Python 2.5 as a standard library module, but probably will be in 2.6 and 
> is widely used now.
> 
> Some of the features of Setuptools aren't necessarily useful to OLPC. 
> Some might be quite useful; the egg format that setuptools is a possible 
> application distribution mechanism, for instance.  Setuptools also has 
> .pth file handling, as I mentioned in a previous email, which would be 
> useful for our development.  It also improves the situation for using 
> zip files for Python packages, which may be useful.  And a plugin 
> system.  And maybe some other things I can't remember.  All of which is 
> a fair amount of Python buy-in; not every OLPC application will be a 
> Python application, for instance, so the egg format isn't ubiquitous 
> (though it has some similarities to Mac bundles, so it certainly brings 
> Python apps closer to what OLPC may implement in a more general form).
> 
> Zip files, .pth files, eggs, and how sys.path is setup is all another 
> important discussion, but somewhat separate.  How exactly sugar is setup 
> is also to some degree separate, as it's a core library and how it gets 
> on the system is kind of incidental.  But something like the chat 
> application should be a model for how other applications are installed 
> and distributed, so it should probably be split off entirely, and then 
> these questions do apply there.
> 
> This email is too long.  Sorry.
>