[sugar] Question about clipboard service

[Eben Eliason]

On Tue, Aug 5, 2008 at 3:44 AM, Greg Smith <gregsmitholpc at gmail.com> wrote:
> Hi Eben,
>
> Interesting spec! Thanks for writing it up. If you want to deliver this
> in 9.1.0, can you put a link to it on the 9.1.0 page:
> http://wiki.laptop.org/go/9.1.0

Done.  I'll add some related tickets there shortly.

> 1 - Not to be pedantic, but can you explain why we need a clipboard? I
> want to define what problem it solves for the user.

The clipboard is a crucial element of the learning environment we want
to provide in Sugar.  Activities are all fullscreen, which makes it
impossible to show two side by side at once.  There is also no
standard desktop onto which files can be placed.  To get around these
issues, we introduce a multi-item clipboard, which serves as a
combination clipboard and/or temporary desktop, onto which objects and
clippings can be placed and "carried" among the various zoom levels
and activities.  Having this facility is crucial to providing a rich
environment in which kids are encouraged to explore, create, and most
importantly /combine/ things they create in separate activities.

On a more technical note, our synchronous/asynchronous collaboration
paradigm also depends on the clipboard, since it requires that it be
possible to open multiple versions of an activity and copy/paste
between them to perform manual merges.

> 2 - I can't tell the difference between clipping and object. Can you
> give more examples and spell out any specific differences in functionality?

Basically, as I mentioned, it serves as a combination clipboard and
temporary storage space.  The only real difference between a clipping
and an object is that the latter has a name.  As a specific example,
an object would be a Write document entitled "Great White Sharks",
while a clipping would be a snippet of text - say, the first paragraph
- from that document.

The need for differentiation is solely to provide better (the best we
can offer) representations of items on the clipboard, such that it's
possible to readily distinguish them from each other, making its use
natural and intuitive.  It's largely inconsequential whether or not
kids or teachers understand the difference between clippings and
objects; all that matters is that they see titles, icons, previews,
and other info consistent with their idea of what they've placed on
the clipboard.

> 3 - Can you link to documentation on the relevant APIs and .py scripts
> or any other information an activity would need to use this?

Heh, no.  That's the whole point.  We haven't yet built an API; the
specification is my design cue for the engineers, so that we can come
to something more usable.

> 4 - Can you move something from the journal to the clipboard?

Absolutely.  In fact, there's already a button for this: "Copy to clipboard".

> 5 - Is drag and drop definitely supported or not?

It needs to be.  And it needs to be ubiquitous.  Sugar is meant for
kids all over the world, with different backgrounds, and little to no
computer experiences.  Drag'n'drop is a physical metaphor that quite
literally moves objects from one place to another.  Unlike a magic
button which performs some magic incantation on objects or state, this
action allows an intimate "physical" connection with the virtual
objects in the UI.

> 6 - Should there be an option to copy and paste from a menu in
> activities or only via control-c/v and drag & drop.

Absolutely!  I think that exposing the buttons for these actions
remains necessary.  They are readily discoverable; it provides a hint
to children to know when an activity supports copy and/or paste; and
it is required for accessibility, since drag'n'drop isn't an operation
that all can perform.

There is already a (perhaps too weak) mention in the HIG about this:
http://wiki.laptop.org/go/OLPC_Human_Interface_Guidelines/The_Sugar_Interface/Toolbars#Edit_Toolbar

> 7 - Is cut supported? How do you remove things from the clipboard? How
> many items can it hold?

Cut is definitely supported, and will remain mapped to the Ctrl-X
shortcut familiar to many, however it will not retain the same name.
Instead, we've applied the notion of "cut" as "copy and erase", which
makes clear that the result of the action is twofold: first it
performs a copy action, and second it erases the item selected.  Since
these semantics make "cut" an alternate form of the copy action, "copy
and erase" will actually appear as a secondary option in the palette
for the "copy" button itself.

Though I'm introducing a multi-item clipboard, very special care was
taken to ensure that, in all possible ways, the clipboard behaves
*exactly* like the single-item clipboards on other computer platforms.
 Every time a copy (or copy & erase) operation is performed, the
resulting clipping appears on the "top" of the clipboard stack, and is
selected as the latest "paste target".  That is, if you were to then
press a paste button, or invoke the Ctrl-V shortcut, the most recent
item added would be pasted.

However, it's also possible to access the clipboard contents randomly.
This is possible via drag'n'drop, of course, but it's also possible to
simple click an item to select it as a new paste target.  The current
selection is clearly indicated.  It's also possible to manually remove
items from the clipboard by selecting the erase action from their
palettes.  It's not yet possible, but I also want to support
drag'n'drop reordering of the clipboard itself.

Finally, as the clipboard is a transient space, we don't want it to
retain items permanently.  Instead, the clipboard will be limited to
either a) some fixed number of items or b) the number of items which
fit in the edge of the Frame without scrolling (this option seems
logical, but is screen-size dependent).  In other words, similar to
the Journal, old clippings eventually "fall off" to make way for new
ones.

> 8 - Are there any performance implications (e.g. when greater than 3
> items on clipboard XO runs low on RAM)?

It's possible, but I don't know the details.  My suspicion is that it
only becomes a real issue when the activities which the clippings were
created from are closed, as otherwise (I think, but I really don't
know) the activities actually advertise their output formats and then
supply the actual data upon a paste operation.  I suppose we could
hack together some solution which actually stored clipboard items as
files on disk (in /tmp/clipboard, perhaps), but that might be
overkill.

> 9 - Can you include some workflow/use case examples? Here are two to get
> you warmed up:

We've did many of these, over a year ago.  I'm sure I have some old
use case scenarios lying around somewhere...

> a - User creates a document in write, then wants to upload it (or part
> of it) to a web site. What are the exact steps and how does the
> clipboard help?
> b - User takes a picture, then uses paint to modify it then puts it in
> write to add text. What are the exact steps and how does the clipboard
> help?
>
> 10 - Are there any collaboration related uses for this? How does it work
> with collaboration, if at all?

Of course.  As mentioned, one prime use case is manual merge.  If
Alice painted a great picture of a lion and Bob painted a great
picture of a lamb, they could share one or both of those activities
and use the clipboard to bring the lion and lamb together in one
image.

If 5 kids are collaborating on a science report, they might all go
home and do independent work on their subtopic as homework.  The next
day they come together in a single shared Write session and use the
clipboard to aggregate their individual efforts into a final report.

The clipboard is absolutely necessary for *asynchronous* collaboration
to happen.

> 11 - Does it do any conversion of formats? What happens if I try to put
> one type of file in an activity that doesn't understand it?

Now you're asking technical details.  In this regard, we will behave
just like the GTK clipboard (which we are building on), and it does
allow (as long as activities support it) some conversions.  Of course,
there will be UI consequences, which I can speak to.  Foremost, we'd
want to make sure that activities are informed of the mime type(s) of
the currently selected clipping, such that they can make the paste
icon insensitive when they don't support any of them.  This, actually,
will go hand in hand with the system which colors the paste icon in
the colors of the currently selected clipping, to further the
metaphor.

> 12 - Can you include more detail on the software design? e.g. How many
> lines of code, what code included where, how many hours work, where does
> the code fit in the overall architecture, what dependencies does it have
> and what other areas is it likely to affect? Is it fully backward
> compatible with existing activities or would activities need to be
> re-written, even if they don't want to use the new functinality?

I imagine it can be made backward compatible.  Some of the new
features might require changes to the activity base class, and yet
others will require activity participation, but the sooner we
introduce these kids of APIs, the better, so that the platform can
then grow over time.  This one has just kept slipping by.

But I'm just UI. You'll need to ask the devs about the true
implications, API, man hours, and overall amount of code necessary.

> 13 - How does it fit with the journal? That is, when would a user pick
> things off the journal and when would they use the clipboard?

Well, many activities will provide object choosers which allow one to
select an item from a Journal-like interface.  Many others won't, but
most all will have copy/paste.  If an activity supports paste, it will
also be possible to go to the Journal, copy an image onto the
clipboard, transition to their activity of choice, and then paste it
into their workspace.

The inverse is also true.  One may select to "Keep in Journal" any
item on the clipboard, which transitions that item from the transient
space into permanence.

In any case, the primary use for the clipboard is as a means to port
items around among activities for the purposes of embedding.

> 14 - Can you copy things directly to a USB drive?

Definitely.  Anywhere we have a "Keep in Journal" option we can also
have a "Keep in > External Storage" menu, which allows one to keep an
item directly onto USB or SD.  This option could also be added to
items on the clipboard.

- Eben


> I hope that's not too many questions. You don't need to address every
> one, just wanted to throw out some suggestions to get to the next level
> of detail.
>
> I appreciate the specifying in advance and I think you are on the right
> track.
>
> Since the journal abstracted the file system, its not easy to move files
> between activities. I think we need an overall strategy for file sharing
> between activities, HW elements (NAND, USB, SD card), computers (XO -
> XO, XO - PC), schools (XO <-> XS <-> XS) and beyond. Clipboard may be a
> key part of it.
>
> Thanks,
>
> Greg S
>
>> Date: Mon, 4 Aug 2008 14:59:56 -0400
>> From: "Eben Eliason" <eben.eliason at gmail.com>
>> Subject: Re: [sugar] Question about clipboard service
>> To: "Tomeu Vizoso" <tomeu at tomeuvizoso.net>
>> Cc: devel at lists.laptop.org, sugar at laptop.org
>> Message-ID:
>>       <948b197c0808041159n21f232f7tac136e05f8ae9191 at mail.gmail.com>
>> Content-Type: text/plain; charset=ISO-8859-1
>>
>> There is a fairly comprehensive specification [1] for the clipboard on
>> the wiki.  Most importantly it discusses the use of titles, icons,
>> colors, and previews, which are the 4 elements of clippings that we
>> need to support in various combination to make the clipboard
>> successful.
>>
>> This spec isn't word for word perfect anymore, as some minor details
>> have changed, but it gives the high level picture of the API I want to
>> support.  Please ask any questions you may have about what's there so
>> far, and I'll try to clean up some details at some point soon.
>>
>> - Eben
>>
>> [1] http://wiki.laptop.org/go/Specifications/Clipboard
>>
> _______________________________________________
> Devel mailing list
> Devel at lists.laptop.org
> http://lists.laptop.org/listinfo/devel
>