[Sugar-devel] Bundles with binary requirements (Was: The ARM is near)

[Benjamin M. Schwartz]

Gary C Martin wrote:
> Hi Benjamin,
> 
> On 28 Aug 2009, at 03:58, Benjamin M. Schwartz wrote:
> 
>> Bobby Powers wrote:
>>> I think having something like:
>>>
>>> example.activity
>>> |-arch/
>>> |-arch/x86/
>>> |-arch/x86/bin/
>>> |-arch/x86/lib/
>>> |-arch/armel/
>>> ...
>>>
>>> could work.  Sugar could set an environmental variable ARCH to the
>>> relevant value, and we could have a reference activity_startup.sh
>>> which adds the correct lib path to LD_LIBRARY_PATH and launches the
>>> appropriate executable (or maybe a flag in activty.info which has
>>> sugar do this).  This is still somewhat kludgy, but I'm not sure of a
>>> better way.
>> Which solution we should choose is a technical discussion that  
>> deserves
>> its own thread.  I'm personally not enthusiastic about the "fat  
>> packages"
>> approach, in which binaries for many architectures are included in  
>> one .xo
>> bundle, because:
> 
> What would be your recommendation?  As a long time Mac user (and  
> developer) I was/am all for "fat packages" (universal binaries), and  
> we certainly don't have the ability to compile binaries on demand for  
> the significant portion of target sugar HW. Activity bundles are a  
> major plus, from where I'm standing, vs. the traditional ./configure,  
> make install, and dependancy requirements.
> 
> With my activity author hat on, I've gone out of my way to avoid the  
> hell of binary blobs, it breaks the whole idea of providing code in  
> Python source for others to easily edit, modify, learn from. But I  
> understand (having adopted maintenance of some old projects) that this  
> has not always been possible for authors (though it woul would always  
> be my goal when writing code).

I am very much with you here: I think writing Activities in pure Python is
very valuable to minimize the barrier to entry for software modification,
and maximize the incentive to learn a bit of software engineering.  Sugar
now has a number of officially supported languages: python, eToys/Squeak,
HTML+Javascript, and bash/shell, at least, and the benefits of
"editability" extend at least somewhat to all of these.

When authors ship binary code, not written in one of the above languages,
it is either because
1. they are depending on pre-existing software, written in another
language, whose presence is not guaranteed by Sugar, or
2. they are writing new software in another language.
(or both)

These cases seem quite different to me.  In particular, we might be able
to make use of some existing packages for case 1, whereas no packages yet
exist in case 2.

Virtually all binaries on Linux are produced by gcc.  One way we could
resolve our binary issue is to declare gcc to be a "blessed dependency" of
Sugar, and then demand that all XO bundles include all their code and
dependencies as source files, to be compiled on the target machine.  This
would essentially resolve the cross-platform problem, in a way that is
nicely aligned with Sugar's emphasis on software freedom and hands-on
engineering.  I also think that the performance overhead would not be
totally outrageous.  We are not talking about compiling browsers, kernels,
GUI toolkits, or X.  Those things are already provided, and developers
should not duplicate them.  The projects being compiled should be
relatively small.

Unfortunately, the situation is not quite so nice.  A few problems:
- Pre-existing applications use a variety of build systems, like autotools
and cmake.  Unless we bless them all, Activity bundles in Case 1 will have
to compile the source code for the build system before they can build the
project itself.  This is not easy for the bundler to get right.
- We will need to provide all the headers against which to compile, the
-devel packages in many distros.  This may require a significant amount of
disk space
- Different distros have different library versions.  Depending on how we
phrase our library version guarantees, we could find that Activities need
to ship an obscenely deep dependency tree to be safe.
- Building large projects is not always easy.  Sometimes complicated,
fragile, poorly documented steps are required.

I am not sure how to resolve all these problems.  My favorite solution so
far most nearly resembles Gentoo's Portage.  Portage is a sort of
"dependency-resolving build management system" that processes "ebuilds"
which are shell scripts that control package compilation.  Portage also
supports precompiled binary packages if they are available, as an
optimization.  "Gentoo Prefix" provides a set of several thousand ebuilds
that can be processed, installed, and run entirely without root
privileges.  I imagine a system in which
For case 1: Activity bundles are required to include source tarballs and
ebuilds for any dependencies.  Upon installation, these dependencies will
be installed from binary packages if matching binaries can be found.
Otherwise, they will be compiled from the provided source, using the
provided ebuild.
For case 2: Authors writing activities in a gcc language must package the
activity as a source tarball and an ebuild to compile it.   For simple
programs, the ebuild is trivial.

The most critical flaw I see in this design is that it creates redundancy
against the underlying distribution.  Gentoo Prefix cannot make use of any
packages installed "outside the prefix", because it doesn't trust outside
packages to be compatible.  This is understandable, especially since
Gentoo Prefix runs natively on Linux, Solaris, AIX, Mac OS, and Windows.
However, it means that in order for Sugar to share dependencies (e.g.
Python, GTK, glib, Telepathy, xlib, ...) with its Activities, all of Sugar
would have to be installed "inside the prefix", and even then some hackery
would be needed to avoid recopying all dependencies for every Activity.  I
don't yet understand how difficult it would be to resolve this issue.

--Ben

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
Url : http://lists.sugarlabs.org/archive/sugar-devel/attachments/20090828/71efc643/attachment.pgp