[sugar] Automated testing of activities
Sun Jul 22 15:30:32 EDT 2007
First, sorry for taking a while to respond to this thread. I think
Fernando has already eloquently stated a good set of priorities of us:
what is most important is that the basic Sugar interface and key
text-based applications are accessible to at least a basic quality
level; games and high quality speech and numerous other things that we
have in the West are nice, but not the bar we should be striving toward
at this time.
To your questions:
> When you say "accessibility" you mean "support for vision-impaired
> users"? Are there other accessibility requirements?
> What in particular are the requirements? Support for a screen reader?
> Large fonts? Zoomable screen? Will there be a screen reader application
> for the system? Where is it coming from, and how will it interact with
A good place to start is http://wiki.laptop.org/go/Accessibility which
contains both some background on the needs, and also an enumeration of
the key things we need to do to support various disabilities (physical
impairments, the full spectrum of vision impairments, hearing
impairments, etc.). There is also an accessibility section in the
Design Fundamentals page at:
Fundamentally for vision impairments (short of near or total blindness),
we need either some level of theming (with a high contrast theme as we
have in desktop GNOME), or a zoomable interface. This includes scaled
fonts, scaled images, etc. For severe vision impairments we need an
add-on screen magnifier, and for blindness we need an add-on screen
reader. For both of these, we need API hooks implemented in the
applications that allow these add-on applications to get the information
they need - this is ATK that is already in GNOME/GTK+, and some version
of AT-SPI for inter-process communication. We have an existing GNOME
application that does a good job for screen reading in Orca. It also
supports screen magnification, though we are looking for smoother
magnification with more features to come with a compositing window
manager. Beryl has some nice magnification features for accessibility,
and that remains a likely path for future magnification on the desktop.
For other impairments like severe physical impairments, we also address
this via an add-on application that allows someone to generation input
from other than a keyboard or mouse. This is more than simple keyboard
substitution; for an effective and efficient user interface for someone
with a severe physical impairment, you really want an application like
the GNOME On-screen Keyboard (GOK) that extracts user interface elements
and places them on a special, scanning keyboard for rapid selection by
someone who can only move their shoulder or make some other simple
muscular gesture. Like Orca, GOK uses ATK/AT-SPI and so is an option
for OLPC once we figure out the IPC mechanism to use.
> How does the accessibility interact with the intent to support
> localizable or localization-free activities, in large part by leaving
> text out of the interface entirely? What is a textless application
> supposed to do in this environment?
> What parts of the system are going to have to comply with this
> requirement? The mesh view, for example? The clipboard? What about, say,
> drawing activities? The record application? Games?
For blind users with text-to-speech, we will need to associate some sort
of text representation to our icons, or at least an "earcon" - audio WAV
file. Since images come from files with filenames, we have a
quick-and-dirty text string we can use for these images. As Fernando
pointed out in his reply, even English text (so long as it sounds
different than other English text for other icons) is better than no
text for someone who doesn't otherwise speak English. Also a lot
smaller than having a bunch of WAV files in flash...
Please note: I am not suggesting we need to render text to the screen;
just that we have text associated with every image a user can interact
with (of every "important" application) which can be retrieved by our
screen reader and then spoken via text-to-speech. The GNOME On-screen
keyboard also uses this text, by the way. And automated testing will
find that text useful too.
We need to make the key tasks in the mesh view accessible. It isn't
important to convey the spatial relationships of the mesh view to a
blind person; it is important that a blind person be able to find his
friends, to have a sense of the access points (how many are there?), etc.
Don't worry about drawing activities at this point. There are a number
of things we do in the desktop to make most functions of most drawing
applications accessible to most people. But please, let's focus on the
critical things first. The same is even more the case for games.
Sun Microsystems, Inc.
> Is automated testing intended for more than just battery life testing?
> If not, is it really necessary for every activity to support it? If so,
> what do you expect to accomplish? Will it actually save more than the
> amount of time taken to implement it for a given activity?
> What are the time constraints?
> The potential scope is huge...it would be nice to understand the actual
> Jim Gettys wrote:
>> I want to give everyone a heads up to start thinking about the following
>> 1) we need to be able to do basic automated smoke testing of activities.
>> 2) we need to support accessibility in activities. Some of you may not
>> be aware of it, but this is a non-optional requirement of some
>> jurisdiction, most of which roughly follow US government laws in this
>> 3) we need to be able to quantify improvements/regressions in activity's
>> energy usage, as we optimize various code in our system.
>> 4) we need to be able to automate testing of realistic workloads (or
>> play loads :-), of our systems that roughly simulate the use of a child
>> in school, so we can see how we're doing when we change various knobs
>> that we have for controlling power usage, from backlight, to use of the
>> dcon, to blanking the screen, to suspending aggressively, etc.
>> Applications adding hints in key locations that suspending might be a
>> good thing to do are also becoming possible, as our power management
>> infrastructure improves.
>> But if we can't reproduce results, we'll be in fog, unable to see what
>> direction to go.
>> We'll therefore need to be able to script applications. So long as
>> we're on an XO-1 with its resolution screen *and* you don't change the
>> UI, it's not all that hard. But we expect all of you to want to tune
>> your UI's, and also we need to ensure accessibility needs get met.
>> Future systems our software runs on may have different screens; this
>> model will break down pretty quickly.
>> Note that the hooks required for accessibility hooks makes it possible
>> to script activities by name, rather than by X/Y coordinates on the
>> screen, and wait for results, and that this technology therefore can
>> remove the screen resolution dependence of such scripting. Custom
>> widgets you build will need such hooks, in any case.
>> We'll shortly be setting up a battery life testing infrastructure in a
>> revived Tinderbox; with the machine we have with instrumented with > 20
>> measurement points we can gather great amounts of useful information on
>> the behavior of our system and of individual activities.
>> At some point, we'll start asking you to generate a workload for an
>> activity, which should be able to address many of the issues above.
>> More when the infrastructure work is further along.
>> - Jim
More information about the Sugar-devel