[Sugar-devel] Bug tracking Vs Patch review

[Bernie Innocenti]

El Tue, 31-08-2010 a las 10:33 +0200, Tomeu Vizoso escribió:
> Yes, we agreed on that, but then, just for the sake of argument,
> several unchecked statements were made that unfairly represented the
> current Sugar development model and the work of several members of our
> community.
>[...]

I'm criticizing the development process, not the community.

While we seem to have diametrically opposed positions on what's
engulfing Sugar's development, we seem to agree that we indeed have a
problem of some kind.

Believe it or no, I highly respect you for all the work you've done for
Sugar and for taking the time to analyze the situation with me. I'm
actually sorry that few others seem interested in having this
discussion.

Moving forward: how are we going to reach consensus on changing (or not
changing) our development model? Shall we have a vote among all current
Sugar Labs members? Only people who ever contributed a patch? Only those
who have patches in git? Shall we ask the oversight board to make a
decision?

When I started to work on Dextrose, I immediately felt the need to fix
the upstream code acceptance process, starting from how we do code
reviews. Six months later nothing has changed, and I had to do most of
the work outside the mainline tree, causing problems for everyone.

It's not too late: some deployments still intend to contribute some of
their work upstream. They simply need someone from within Sugar Labs to
make a few steps towards them.


> In case it needs to be said, I'm extremely grateful at Christian,
> Gary, Eben, Eduardo and the others that put their experience, effort
> and time thinking about our UI, discussing it with others, drawing
> mockups for other people's features, etc

Me too. The dedication of our community is not being questioned.


> Unfortunately, I have seen people spend months coding a feature and
> not having found 30 seconds to write a heads-up email to the list
> asking for user experience feedback. At some point I took a minute to
> write such emails for 3 other people's features. Obviously it helps if
> you start the discussion just before you start coding, and not just
> after.

Ha! I've also seen the exact opposite happen: Tincho's initial design
proposals for the backup/restore feature and for the resource meter were
mostly ignored... Then we went on implementing those designs, test,
integrate and finally submit patches. At this point, people started
asking to redesign both the UI and the implementation.

So we gave up. Both Uruguay and Paraguay find the backup/restore feature
really useful. It would be extremely useful also for other deployments
and SoaS users. But it won't be in mainline until someone rewrites it,
which probably means never.

And here's where we should keep into account non-technical factors. Why
would a deployment that is already happy with how the feature works
today take the extra time to rewrite it according to how an upstream
wants it?

The long-term benefit of upstreaming patches is always unclear,
especially to managers. If we put too much of a burden on contributors,
we're simply going to loose them. We could go around saying it was their
fault for not following procedures X, Y and Z, but it's ultimately our
loss.


> > Sometimes UI designers come up with an idea that nobody would implement.
> > In other cases, programmers come up with UI changes that designers
> > reject. When you get both, the work needs to go through maintainer
> > approval of each affected module.
> 
> Again, you are misrepresenting me just for the sake of argument,

You? This thread is not about you, don't take it personally. You happen
to be another person who cares to re-discuss our processes, this is why
I'm having this conversation with you.


> there's a wide chasm between giving others the chance to opine and
> taking those opinions seriously, and having to reach unanimity on a
> perfectly defined design.

So you agree that the final decision to include code in Sugar should be
in the hands on just one person who takes full responsibility for the
evolution (or lack thereof) of Sugar?

Because, to me, this is exactly what Sugar is missing today.


> > Having some form of reviews is great, but we broke up Sugar in tightly
> > coupled modules, so that any non-trivial change often spans across 2 or
> > 3. Typically it's sugar + sugar-toolkit + sugar-artwork.
> 
> One person is working on merging them, several others have preferred
> just to complain about it.

Once you've established a lightweight workflow, merging patches is in
itself a no-brainer! All the maintainer has to do is "git am
patches_to_apply.mbox" and resolve any conflicts. If we managed to
approve patches in a timely fashion, they would apply cleanly almost all
the time.

As you know, in 6 months I've accumulated in Dextrose over 100 patches
for Sugar. Even though managing a patch queue is somewhat harder than
simply merging them, only a tiny amount of my time went in it. The
actual time sinks were testing and interaction with the various
upstreams.

Interaction with OLPC, Fedora and other projects actually went very
smoothly. Only Sugar was problematic.


> > Different people may be competent on different parts of the code, but
> > I'd rather have one incompetent decision maker rather than many
> > conflicting ones.
> 
> That's not true, and actually, may be interesting for you to try to
> find out why our maintainers don't waste their time arguing with each
> other and seem to have an easy time finding agreement.

What??? Our mailing-list archives and tickets are *full* of people
arguing in favor or against this or that patch!

This is absolutely normal in any project. In our case, most of these
threads die off with no clear action being taken. Like in the case of
the backup/restore feature.


> Visible because someone has invested a lot of time giving it
> visibility in the mailing list and made bombastic statements so other
> people were forcedly dragged into the discussion?

If you think this discussion is just about defending the current process
against bombastic statements, there's no hope we can make any progress.
Better stop it here.

I thought you agreed with me that we have a big problem of downstreams
not contributing that we wanted to solve in one way or another.

Please drop the "personal attack" argument; it just derails us off any
constructive track.


> > Very few people look daily at activity in the bug tracker and even fewer
> > participate in discussions of tickets filed by others. As a result, we
> > often learn about a patch only when (and if) it lands in git... when
> > it's too late for a public review.
> 
> You are again mixing unrelated things just for the sake of argument.
> We could have very well patches tracked in trac with the queue exposed
> in the mailing list and with reviews happening in the mailing list. We
> don't have this setup because someone needs to invest time on setting
> it up.

We could, theoretically, but we don't have this. And even if we did,
people won't easily see the thread going on in each ticket. The only
sensible way to follow development done in a bug tracker is manually
subscribing to each new bug, which makes a very poor replacement for a
mailing list (no threading, no ability to reply from the email client,
no proper editor for writing long replies...).


> > After a good night of sleep, I realize that my post sound too much like
> > a rant and too little like a constructive proposal. I apologize about
> > that.
> 
> And this email I'm replying to is constructive in which way?

Like I did several months ago, I'm proposing an organizational change
that would (in my mind) solve the problem. We're not moving forward
because you took it personally and became really defensive.

Being open to experimentation on the development process was another
factor that drove Linux to success. If the kernel hackers stuck to the
same methods of 15 years ago, development wouldn't have scaled up past a
certain point.

So, why don't we just *try* my proposal for some time, to see if
something good would happen? If I were wrong, we can simply revert to
the old method.


> Contributors are the first ones that benefit from having a clear
> submission process. They can check if they are being treated fairly
> and can influence the way their work is accepted. You weren't around
> when Sugar moved from an ad-hoc patch acceptance to have a process but
> that doesn't mean that we have to start again just so you can better
> understand why.

I can't speak for all the Sugar contributors, but I met a large number
of them who were *totally* pissed off by the interaction with us. I'm
sure you know who they are.

Granted, we can't make everyone happy, but Sugar has an unusually high
number of very unhappy contributors and, hence, an unusual number of
lost contributions. This is why I felt the urge to propose a drastic
change.


> > You say that there are not enough people to cover all the roles defined
> > by our processes, but the more people you add, the more development will
> > get engulfed.
> 
> Again misrepresenting what I have ever said, we only have two roles
> and we have more than two people. We need more people doing more work
> but I don't see what it has to do with our processes.

Are we talking about different things? I see *plenty* of maintainers
over here:

  http://wiki.sugarlabs.org/go/Development_Team/Release/Modules

Plus, we have an RM, a Team Coordinator (now vacant) and several folks
for the Design team.

None of the above mentioned people can make a final decision on whether
or not a non-trivial patch like the sort-by-size feature can land in
git. Add to this that the discussion happens mostly in a bug tracker and
it takes *months* to make a decision. This time we were lucky, Andres
managed to get it through just in time for 0.90 (after spending several
months off-tree, of course).


> I think despite the hard moments I have been successful at that and if
> I had to make a recommendation it would be to not whine, do good work
> and to try harder to understand the others. Once you have gotten
> respect because of your work *and* you have gotten a fair
> understanding of what's going on, then you will be in a good position
> to propose improvements.

You make it sound like I've never contributed anything to Sugar or any
other FLOSS project. Is this what you meant or am I misinterpreting the
sense of your last paragraph?


> > The bad news is that we often introduce serious bugs that would be
> > easily spotted by a glance at the patch
> 
> What do you mean, that nobody glanced at it?

No, I mean that an insufficient number of eyeballs have glanced at it.
Have you ever heard of Linus' "many eyes" law?

  http://en.wikipedia.org/wiki/Linus'_Law


> > One example. For the last 2 weeks, code in jhbuild has been crashing on
> > startup for several people (anish, tch, myself) on several distros
> > (Fedora 13, Fedora 14, Ubuntu Karmic).
> 
> Patches were promptly made available in the mailing list and some have
> already been accepted by Sascha into jhbuild.

Still doesn't work here, neither with jhbuild nor with Fedora packages.
It's not a mere dependency issue:

  http://bugs.sugarlabs.org/ticket/2269
  http://bugs.sugarlabs.org/ticket/2270

A few days ago I was also seeing another error in the logs about a
missing changed-activity signal, but it seems to have disappeared now.


> > Regardless, we proceeded to
> > release 0.89.5 tarballs that downstreams promptly turned into broken
> > packages (tested on Fedora).
> 
> You clearly had different results than me when testing on F14, have
> you filed bugs or reported them in some place?

What distro did you use? It also fails on Ubuntu and Fedora 13 according
to other people on #sugar. Did you really hear about this for the first
time for me?


> By the way, this is called integration testing and it happens every
> cycle in GNOME's jhbuild and again when stuff gets packaged at first
> in Fedora.

We're not talking about a minor integration glitch. It's broken on 3
distros (minimum).


> The problems found during the integration phase are due to the
> unexpected effects of putting together for the first time specific
> versions of hundreds of different dependencies built with specific
> sets of build options (such as the gnome-keyring issue with
> mission-control). How did you expected to catch those by reading Sugar
> patches?

Not by reading patches, but by having the maintainer test that at least
jhbuild still runs before pushing, or at least before releasing
tarballs.

But I'm assuming that Sugar is currently broken for everyone, which does
not seem to be the case at least for you.


> > We seem to be missing the equivalent of a "product manager"... someone
> > who cares for the product as a whole. In a different occasion, Walter
> > noted the need for an "architect", but in a project like ours these are
> > very much the same thing, just with an emphasis on the present or on the
> > future.
> 
> It would help if you described that role in some details, otherwise
> can mean too many different things.

This is more or less what I'm thinking of:

 http://en.wikipedia.org/wiki/Product_manager


This is common business practice. If you prefer the agile software
development terminology, we need a "Product Owner" for Sugar.

Walter some time ago called for an Architect, which is something close,
but with a focus on technical aspects of the product and long-term
evolution of the codebase.


> > We merged a total of 355 patches in sugar over the last 365 months.
> > Exactly one patch per day. Over the same period, Linus merged 100 times
> > this number of patches,
> 
> Looks like you are onto something big here. After you fix Sugar you
> can go fix GNOME and the rest of FOSS projects, then with 100 times
> more productivity Microsoft, Apple and Google will disappear in a puff
> of smoke.
>
> > meaning that maintaining Sugar should be
> > feasible even as a part-time job.
>
> And we have how many full-time maintainers?

We have several, no? You, erikos, alsroot... plus several part-time
ones.

So complaining that we don't have enough maintainers for merging 350
patches per year is clearly missing the point. The problem we have is
one of organization, not one one of resources.

The number of new features and bugfixes in Dextrose proves that there
are abundant resources for Sugar development. We're just blocking them
from contributing effectively because the processes we have in place
don't work well.

-- 
   // Bernie Innocenti - http://codewiz.org/
 \X/  Sugar Labs       - http://sugarlabs.org/