[sugar] "Garden": A (currently hypothetical) Sugar library for sharing state between Activity participants

[Ian Bicking]

Andrew Clunis wrote:
> It seems to me that OLPC development would be much friendlier for
> students, educators, and wizened geeks alike if Sugar provided a means
> for an author to mesh-enable their Activities in an elegant,
> non-obtrusive and declarative way, *without* the need for a traditional
> client/server topology.  Right now it appears that sugar.p2p only
> provides a very simple foundation, thus requiring authors to essentially
> devise their own network protocol.
> 
> "Garden", as I have termed the hypothetical system in the interim,
> could be the answer to this problem.  It would be an amalgam, of sorts,
> of a network object synchronisation system and the high-level Python
> marshalling library sitting on top of it.
> 
> Requirements:
> 
> 1) It needs to be easy for Python activity developers to write their
>    activities without having to spend significant amounts of time
>    devising an activity-specific network protocol;
> 
> 2) The syntax for using it should be as declarative and intuitive as
>    possible;

I think the Python portion of this will be relatively easy to implement, 
compared to the basic algorithm design.  Also, "declarative and 
intuitive" can be a bit subjective and domain-specific.  All of which is 
just to say that I think we can defer this portion of the design until 
later, and focus on the language/runtime-neutral portion of this problem 
space.

> 3) It should also at least be partially available (at least the network
>    syncing component) to developers using a language other than Python; 
> 
> Threat Model:
> 
> Garden earns its name and threat model from the clever "Walled Garden"
> metaphor mentioned in the OLPC Human Interface Guidelines.  The general
> idea is that all participants enjoy equal and nearly absolute trust
> within the context of an activity.  That said, there should be a clear
> boundary between the activity context and the local personal context of
> a user.  Everyone can draw and muck about in the soil, but you don't
> want some other kid coming up and giving you a wet willy (read: explode
> your laptop).

Additionally, just because you are sharing one object/document, you 
don't want users to have access to all your objects/documents created by 
the same activity.  The activity itself will probably have access to all 
these.  So we need a stronger boundary than just the activity itself.

Mostly this is okay if we don't exchange code.  Other possible holes are 
application-specific (though still quite possible).

> Protocol:
> 
> Changes would be transmitted as "differentials".  Transmitting the
> entire object for each change would be prohibitively expensive in terms
> of both network and CPU, and would also make debugging network logs more
> difficult.
> 
> The mechanism of the network component could be this: all the laptops
> in the Activity are joined together in a single multicast group.
> This will likely be done via the existing facilities in Sugar
> (sugar.p2p) for many-to-many messaging (reliable multicast, etc.).
> However, apparently that component of sugar is likely going to
> rewritten or heavily refactored, so naturally any development will need
> to follow the changes made there.
> 
> In order to manage the inevitable conflict race condition, at
> negotiation time the participants could assign themselves a random
> "rank" value.  In the event that a laptop receives a second changeset
> with a sequence number equal to the one it received last, it would elect
> to drop or use the changeset if the difference between ranks of both is
> negative.  If it does decide to keep the newly received changeset, then
> it will have to drop current object from the prior version (including
> any local changes that may have occurred) and replace it with the new one.
> This method has the added benefit of removing the need for a timeout,
> replacing it with a limited number of copies of the object in
> memory.  However, the extra memory use could prove expensive.
> 
> Unfortunately, none of this takes the "island case" into account.  If
> the group is split in two, then it would continue as two separate
> groups with no rejoin capability (actually, that is debatable because
> the Activity would still be listed in Avahi with the same ID. do I
> have that right?).
> 
> The above system is of course only one method of many to tackle these
> problems.  Perhaps a preexisting system like "Fleet" will provide a
> good basis for designing the protocol instead (suggested by Ivan K.).
> 
> The entire network component could be exposed over d-bus for Activities
> written in languages other than Python to take advantage of, since all
> it does is provide a generic interface for ensuring a piece of data
> in memory is kept synchronised with a group of other machines over the
> network.  This would make the system meet requirement #3.
> 
> High-Level Python Component:
> 
> The language-specific component, at least for Python (requirement #1),
> could be implemented something like this: Every time one laptop makes
> a change to the shared object, the library is instructed to propagate
> the change.  This could be done by requiring the Activity developer to
> explicitly specify the changes they make by calling some special
> method for every change they make.  A more implicit method could
> involve introspection, whereby the library checks each instance
> attribute against older values in memory. Then, the library would
> transmit the modified fields to the multicast group, along with an
> incremented sequence number.  Both methods introduce some non-obvious
> complications.  Lists and Dictionaries would have to be replaced with
> special Garden-specific versions. Whatever method is chosen, it is
> important to keep requirement #2 in mind.
> 
> A thread would probably be necessary to wait for new messages coming in
> from the network, and a Lock for the entire shared object as well.

I imagine the message listener could also be part of the GTK event loop.

> Best Practices for an Activity Author:
> 
> - - Never trust any data inside a shared object outside the scope of the
>   Activity (be wary of things like filenames, turing-complete code such
>   as scripts, data that you might pass into code other than your own,
>   etc.);
> 
> - - Make the rest of your Activity code as state-independent as possible,
>   because for several reasons the library may replace the entire state
>   of the shared object(s)
>   (conflict rollbacks, joining a new activity, etc.).

This is difficult, because essentially you'll have to poll your object 
to see if it changed.  Polling is a big no-no for OLPC.

So then you get a situation where you will have, I suppose, events and 
listeners, with one kind of event: "updated".  Then the UI listens for 
an event from the model object, and updates the UI based on that.  At 
which point one wonders if the model object is needed at all, or if the 
events are sufficient?

> Other Possible Features:
> 
> - - A "private" mode, whereby setting up a symmetric session
>   key by doing some sort of key negotiation via DH.  This feature, if it
>   exists at all, should be made easy to disable because many jurisdictions
>   in the world take a jaundiced eye to cryptography.

Aren't all shared objects private until other users are specifically 
invited?

Potentially this framework could implement the UI around making shared 
objects available/public, inviting users, etc.  It seems like a common 
UI around this could be nice.

> - - A "proxy" mode to allow someone to bring a friend in who is not on the
>   local network but rather on the public Internet somewhere.
> 
> This is a really hard problem, so I elected to post this brain-dump
> instead of write any code.  There will almost certainly be many differing
> opinions about how to go about doing this.
> 
> Is my take on this utterly hare-brained or does it have some merit? :)

It's definitely important to consider this, so thanks for bringing the 
topic up.  I think there's a lot of discussion yet to come around this ;)


-- 
Ian Bicking | ianb at colorstudy.com | http://blog.ianbicking.org