[Sugar-devel] Introducing sugar-build

[Daniel Narvaez]

Hi Gonzalo,

thanks a lot for your feedback.

To answer your general point:

* I think these tools are used by few people because they don't work well.
I know despite having a lot of experience with GNOME and linux builds I was
highly frustrated by sugar-jhbuild complexity and unreliability. I'm sure a
lot of people have been in that situation and gave up. This is not to pick
on sugar-jhbuild, as I said in rationale it's a very difficult issue to
solve. Mine is another try.and I agree with who said competition is good
here.
* Maintenance of these tools has an high cost but losing talented
contributors because they are unable to get the thing to build is a much
much higher one. Also, many of the design choices in sugar-build intends to
reduce maintenance cost (which I think in turn will favor reliability).

Why I didn't just contribute to sweets?

My understanding is that sweets has a very different goal. It's a
distribution tool, not a development one. It's probably useful for
activities development, in the same way of distributions packages. But
that's not what I'm interested in. I want to make it possible for more
people to hack on the sugar core modules.

Why I didn't just contribute to sugar-jhbuild?

Let me answer by describing the very few code sugar-build is composed of.

Makefile (32 lines)

This is what exposes the minimal set of commands we provide. I think it's a
simpler  interface compared to jhbuild and will also allow us to use a
different set of scripts under the hoods if we want (my dream is to be able
to build only sugar-* stuff in the future and we could just use submodules
if that ever happens). There is nothing like that in sugar-jhbuild.

scripts/check-system (181 lines, expected to grow a lot)

I wanted checks to be cross platform. It's a good start to have those if
you want to add support for another distro. You just need to add mappings
to the distro specific packages which is tedious but easy. And it's easier
to maintain because whenever you add a dependency you just need to make
sure there is a package for every supported distributions in a single
aggregated file, rater than having to go through a bunch of xml files.
Also I'd like to reuse the script to check if sugar distributions are
compliant, in the sense they provide all the dependencies activities are
allowed to depend. This is probably the only high maintenance cost part of
sugar-build, the most use we make of it the better.
Again nothing like this in sugar-jhbuild.

scripts/build-activity (8 lines)

sugar-jhbuild uses a custom jhbuild module for this purpouse. I'm just
building a couple of activities (for the reasons I explained in the
rationale) and i didn't think it was worth to maintain an out-of-tree
jhbuild module for this. We could add the module to jhbuild in the future
though, if the GNOME maintainers would take it.

scripts/sugar.modules (46 lines)
scripts/system.modules (27 lines)

Somewhat duplicated with sugar-jhbuild but the logic is different. We
assume a certain base system and only specific the minimal number of
absolutely required modules. We don't try to list a complete list of
dependencies and then skip the ones that are not required on a certain
distribution.

So I could have started from sugar-jhbuild but, to achieve my goals, I
would have had to basically to delete most of it and replace with new code.
The only sizable piece of code they share is jhbuild and that's something
we just pull as is.

Daniel

On 14 June 2012 03:50, Gonzalo Odiard <gonzalo at laptop.org> wrote:

> From one side, it's great to have more people interested in the build
> tools, thanks!
> But is really bad start again with another solution (sugar-jhbuild,
> sweets, sugar-build...),
> this tools are not used by too much people, then the cost of maintain it,
> is very high.
> If you are working in a solution based in sugar-jhbuild, please try to
> identify
> how can be improved the actual situation (I know is not very good)
> and try to work with the people already involved.
>
> Gonzalo
>
>
> On Wed, Jun 13, 2012 at 7:45 PM, Daniel Narvaez <dwnarvaez at gmail.com>wrote:
>
>> Hello,
>>
>> I'm trying to figure out how to make building sugar from source easier
>> and more reliable. I wrote a new set of build scripts that I'm hoping will
>> improve the situation. They are not too dissimilar from sugar-jhbuild but
>> there a few key differences.
>>
>> The code is here:
>> https://github.com/dnarvaez/sugar-build
>>
>> I'm pasting the README which explains the rationale and how the thing
>> works in practice.
>>
>> Daniel
>>
>>
>> = Setup a supported distribution =
>>
>> For all the distributions your user need admin rights (i.e. it should be
>> able to run the su command).
>>
>> == Fedora 17 ==
>>
>> Works out of the box!
>>
>> == Ubuntu 12.04 ==
>>
>> By default only root can start the X server. You need to change that with
>>
>>   sudo dpkg-reconfigure x11-common
>>
>> I'm hoping we can get rid of this step in the next Ubuntu version when a
>> working nested X server implementation should become available.
>>
>> = Getting started =
>>
>> Run the commands
>>
>>     make build
>>     make run
>>
>> That should be all you need to have sugar running!
>>
>> = Commands reference =
>>
>> make build        Build everything.
>> make run        Run sugar.
>> make build-[module]    Build a single module.
>> make clean        Delete build artifacts and sources.
>>
>> = Report bugs =
>>
>> For now just send an email to Daniel Narvaez <dwnarvaez at gmail.com>. I
>> expect a lot of tweaks will be necessary before this is stable. In the
>> future distributions upgrades will also be problematic. We need the build
>> to work out of the box 99% of the time for everyone, so if stuff breaks
>> don't please don't be shy and always report the bug (patches are
>> appreciated of course but absolutely not required). If you install an
>> activity and it doesn't work, that might also be a sugar-build bug, in
>> doubt just report it.
>>
>> = Rationale =
>>
>> == The current situation ==
>>
>> * We need more people to hack on the core sugar modules. A lot of
>> maintenance is necessary and so many features that are a fundamental part
>> of the initial vision are still unimplemented.
>> * Building sugar is currently too complicated and unrealiable. While this
>> is just an annoyance for an experienced hacker, it can a blocker for
>> newcomers. If they can't even get the build to work they are likely to give
>> up and never contribute.
>> * It is a difficult problem to solve because we are depending on system
>> components that, while having reasonable ABI policies, are moving very fast
>> (recently with gobject-introspection and gtk3). The number and the level of
>> our dependencies is also much higher than the average project.
>> * It's not just about building and running glucose. A good development
>> environment needs to include all the system components activities are
>> allowed to expect.
>> * Unrealiability brings more unrealiablity. Build breakages are
>> apparently going unnoticed for several days. I suspect part of the reason
>> is that people just expect things to be broken and deal with it, without
>> reporting or fixing the issues for everyone.
>>
>> == How sugar-build tries to improve the situation ==
>>
>> * Rather then trying to adapt to the underlining system, it ensures we
>> have a consistent system on all the distributions we support, so that the
>> same identical build process works on all of them. The downside is obvious,
>> we can't expect to work on old distributions. But I think that's a price we
>> can pay for easier maintenance and especially for higher reliability. It's
>> better to support only a few distributions and work flawlessly 99% of the
>> time on the time, then promise support for everything but never work
>> realiably.
>> * To verify that all the system components we need are installed we use
>> cross-distribution checks. Though to make life easier for developers we map
>> these to distribution-specific packages and install all of them
>> automatically before starting to build.
>> * We only build master of the sugar modules. That's what we want
>> developers to work with. The less build configurations we need to maintain
>> the more likely they are to be realiable. For the same reason we only
>> support the most recent version of a distribution.
>> * We build only a couple of essential activities, browse and terminal, to
>> improve the initial develop experience. The rest can be downloaded from
>> activities.sugarlabs.org and they should work because we are ensuring
>> consistent system dependencies.
>> * We are using jhbuild internally to keep the amount of custom code to
>> the minimum, but we are not exposing it in the build commands, which are a
>> just few make targets. jhbuild is very flexible but too complicated for us.
>> People should be able to contribute to large parts of the sugar code base
>> without much linux or GNOME experience.
>> * Ideally we would only build sugar modules and depend on distributions
>> for everything else. And I hope that's what we will do at some point in the
>> future. Unfortunately in the current development cycle we will need a few
>> components that are not yet packaged, not even written or upstreamed. I
>> believe we can reduce the impact on realiability at the minimum by building
>> only modules that are absolutely required and by checking out a specific
>> commit rather than master. We don't want to get new code from upstream
>> without first verifying that it works well on all the distributions we
>> support. The same is true for jhbuild itself.
>> * To ensure reliability of the build scripts and of the sugar code
>> itself, we are going to setup buildbot slaves for all the supported
>> distributions. They will notify the irc channel and perhaps the committers
>> when a build breaks. When it happens I think fixing it should be everyone
>> top priority. Ideally we would also improve our review process so that no
>> code is committed before it has gone through buildbot, gerrit works very
>> well for that but there might be less invasive ways to do it. Regardless of
>> the exact process and tools, patches that add new dependencies to the
>> codebase should come with a corresponding sugar-build patch. Nothing that
>> is pushed should break the buildbots.
>> * Ensuring the build works is an important first step. Though integration
>> tests, even if just to verify that stuff actually run, will go a much
>> longer way.
>>
>>
>> _______________________________________________
>> Sugar-devel mailing list
>> Sugar-devel at lists.sugarlabs.org
>> http://lists.sugarlabs.org/listinfo/sugar-devel
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20120614/9a06a26b/attachment-0001.html>