[sugar] "Garden": A (currently hypothetical) Sugar library for sharing state between Activity participants

[Andrew Clunis]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

It seems to me that OLPC development would be much friendlier for
students, educators, and wizened geeks alike if Sugar provided a means
for an author to mesh-enable their Activities in an elegant,
non-obtrusive and declarative way, *without* the need for a traditional
client/server topology.  Right now it appears that sugar.p2p only
provides a very simple foundation, thus requiring authors to essentially
devise their own network protocol.

"Garden", as I have termed the hypothetical system in the interim,
could be the answer to this problem.  It would be an amalgam, of sorts,
of a network object synchronisation system and the high-level Python
marshalling library sitting on top of it.

Requirements:

1) It needs to be easy for Python activity developers to write their
   activities without having to spend significant amounts of time
   devising an activity-specific network protocol;

2) The syntax for using it should be as declarative and intuitive as
   possible;

3) It should also at least be partially available (at least the network
   syncing component) to developers using a language other than Python; 

Threat Model:

Garden earns its name and threat model from the clever "Walled Garden"
metaphor mentioned in the OLPC Human Interface Guidelines.  The general
idea is that all participants enjoy equal and nearly absolute trust
within the context of an activity.  That said, there should be a clear
boundary between the activity context and the local personal context of
a user.  Everyone can draw and muck about in the soil, but you don't
want some other kid coming up and giving you a wet willy (read: explode
your laptop).

Protocol:

Changes would be transmitted as "differentials".  Transmitting the
entire object for each change would be prohibitively expensive in terms
of both network and CPU, and would also make debugging network logs more
difficult.

The mechanism of the network component could be this: all the laptops
in the Activity are joined together in a single multicast group.
This will likely be done via the existing facilities in Sugar
(sugar.p2p) for many-to-many messaging (reliable multicast, etc.).
However, apparently that component of sugar is likely going to
rewritten or heavily refactored, so naturally any development will need
to follow the changes made there.

In order to manage the inevitable conflict race condition, at
negotiation time the participants could assign themselves a random
"rank" value.  In the event that a laptop receives a second changeset
with a sequence number equal to the one it received last, it would elect
to drop or use the changeset if the difference between ranks of both is
negative.  If it does decide to keep the newly received changeset, then
it will have to drop current object from the prior version (including
any local changes that may have occurred) and replace it with the new one.
This method has the added benefit of removing the need for a timeout,
replacing it with a limited number of copies of the object in
memory.  However, the extra memory use could prove expensive.

Unfortunately, none of this takes the "island case" into account.  If
the group is split in two, then it would continue as two separate
groups with no rejoin capability (actually, that is debatable because
the Activity would still be listed in Avahi with the same ID. do I
have that right?).

The above system is of course only one method of many to tackle these
problems.  Perhaps a preexisting system like "Fleet" will provide a
good basis for designing the protocol instead (suggested by Ivan K.).

The entire network component could be exposed over d-bus for Activities
written in languages other than Python to take advantage of, since all
it does is provide a generic interface for ensuring a piece of data
in memory is kept synchronised with a group of other machines over the
network.  This would make the system meet requirement #3.

High-Level Python Component:

The language-specific component, at least for Python (requirement #1),
could be implemented something like this: Every time one laptop makes
a change to the shared object, the library is instructed to propagate
the change.  This could be done by requiring the Activity developer to
explicitly specify the changes they make by calling some special
method for every change they make.  A more implicit method could
involve introspection, whereby the library checks each instance
attribute against older values in memory. Then, the library would
transmit the modified fields to the multicast group, along with an
incremented sequence number.  Both methods introduce some non-obvious
complications.  Lists and Dictionaries would have to be replaced with
special Garden-specific versions. Whatever method is chosen, it is
important to keep requirement #2 in mind.

A thread would probably be necessary to wait for new messages coming in
from the network, and a Lock for the entire shared object as well.

Best Practices for an Activity Author:

- - Never trust any data inside a shared object outside the scope of the
  Activity (be wary of things like filenames, turing-complete code such
  as scripts, data that you might pass into code other than your own,
  etc.);

- - Make the rest of your Activity code as state-independent as possible,
  because for several reasons the library may replace the entire state
  of the shared object(s)
  (conflict rollbacks, joining a new activity, etc.).

Other Possible Features:

- - A "private" mode, whereby setting up a symmetric session
  key by doing some sort of key negotiation via DH.  This feature, if it
  exists at all, should be made easy to disable because many jurisdictions
  in the world take a jaundiced eye to cryptography.

- - A "proxy" mode to allow someone to bring a friend in who is not on the
  local network but rather on the public Internet somewhere.

This is a really hard problem, so I elected to post this brain-dump
instead of write any code.  There will almost certainly be many differing
opinions about how to go about doing this.

Is my take on this utterly hare-brained or does it have some merit? :)

- -- 
Regards,
Andrew Clunis
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFFaptzALkUMXSNow8RAkYXAKC+1jkQWuldiWKqeUELAEibkNxudACfd6Ed
XKygV5x7KrSbeqYfElwSQLQ=
=7Nbk
-----END PGP SIGNATURE-----