[Sugar-devel] Test protocol for "Resume" vs. "Start new"

[Christian Marc Schmidt]

Thanks Gary, I'll have a look at this. I'm worried that conducting tests
with novice users might be inconclusive, given that we are testing
schoolchildren. But I am also not an educator. I'd be happy to work with
someone with an educational background to put something together that might
be suitable as a test for new users.

For now, though, I'd suggest we stick to our plan to see where it gets us.

Christian
On Mon, Jan 25, 2010 at 11:20 AM, Gary C Martin <gary at garycmartin.com>wrote:

> Hi Christian,
>
> On 23 Jan 2010, at 15:01, Christian Marc Schmidt wrote:
>
> > Hi all:
> >
> > For our meeting at 10:30pm EST today, I put together a draft of a test
> protocol (in email body below). Please look it over and feel free to edit or
> add to. Thanks,
> >
> > Christian
> >
> > --
> > Sugar Home View Concept Testing
> > Draft 01/23/10
> >
> > Goal: Evaluate the Home view for "Resume activity" and "Start new
> activity" functionality.
> > Protocol: Alternate between two builds, one per student: The current
> build [Build A], which makes "Resume" the default action, and a test build
> [Build B] making "Start new" the default action.
> > Duration: 5 mins per student
> > Student Profile: Should be a current user (not a first-time user). Age
> and grade should be recorded, along with duration of overall Sugar
> experience.
> >
> > 1. Begin at Home screen after boot up
> >
> > 2. Test cognitive model
> >       • "Describe what you see in this view."
> >       • "What are the symbols in the ring?"
> >       • "Describe what each symbol in the ring is."
> >       • "Do you know why the symbols are different colors?"
> >       • "What activity did you last work on?"
> > 3. Test pathways
> > [Alternate between two pathways, one per student]
> >
> > Pathway 1:
> >       • "Show how you would go back to the activity you last worked on.
> Describe what you are doing." [Student shows pathway to resume activity.]
> >       • "Now, show what you would do if you wanted to start a new
> [Write/Draw/Browse, etc.; should be of the same kind] activity. Describe
> what you are doing." [Student shows pathway to start a new activity of the
> same kind.]
> >       • "Now, show what you would do if you wanted to start a new
> [Write/Draw/Browse, etc.; should be a different activity than the current
> one] activity." [Student shows pathway to start a new activity different
> from the current activity.]
> > Pathway 2:
> >       • "Show how you would start a new activity. Describe what you are
> doing." [Student opens a new activity.]
> >       • "Now, show what you would do if you wanted to go back to the
> activity you last worked on. Describe what you are doing." [Student shows
> pathway to resume a previous activity.]
> >       • "Now, show what you would do if you wanted to start a new
> [Write/Draw/Browse, etc.; should be of the same kind] activity. Describe
> what you are doing."[Student shows pathway to start a new activity of the
> same kind.]
> > 4. General questions
> > "If you could change anything about Sugar, what would make it easier for
> you to go back to a previous activity?"
> > "What would make it easier for you to start a new activity?"
> > "Is there anything else about Sugar that you would change?"
>
> I know we've drifted from a usability study model, to a focus group model
> for practical classroom reasons – but I though Jakob Nielson's latest
> article was a timely piece worth a quick read through given we are going to
> be getting feedback from existing Sugar users.
>
> "Summary: It's more difficult to conduct usability studies with experienced
> users than with novices, and the improvements are usually smaller. Still,
> improving expert performance is often worth the effort."
>
>        http://www.useit.com/alertbox/experienced-users.html
>
> Regards,
> --Gary




-- 
anything at christianmarcschmidt.com
917/ 575 0013

http://www.christianmarcschmidt.com
http://www.linkedin.com/in/christianmarcschmidt
http://twitter.com/cms_
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.sugarlabs.org/archive/sugar-devel/attachments/20100125/b0cd0e59/attachment.htm