[Sugar-devel] Test protocol for "Resume" vs. "Start new"

[Tomeu Vizoso]

On Wed, Jan 27, 2010 at 13:17, Wade Brainerd <wadetb at gmail.com> wrote:
> If possible, it might be helpful to record the students completing
> this test, and then post the results to the wiki (as links to YouTube
> etc).
>
> We video tape focus testing sessions all the time in the video game
> industry.   You just have to be careful about interpreting the
> results, and it does take some time to go through watching them all.
> But it reveals a lot of hesitations, failed attempts, etc. that don't
> come out in a survey.
>
> Anyway, either way I agree that this test would be helpful in
> determining the best home view behavior.

This may be relevant:

http://mairin.wordpress.com/2009/08/26/open-source-portable-usability-testing-lab/
http://mairin.wordpress.com/2009/10/17/open-source-portable-usability-testing-lab-part-2-the-parts/

Regards,

Tomeu

> Wade
>
>
> On Mon, Jan 25, 2010 at 11:35 AM, Christian Marc Schmidt
> <christianmarc at gmail.com> wrote:
>> Thanks Gary, I'll have a look at this. I'm worried that conducting tests
>> with novice users might be inconclusive, given that we are testing
>> schoolchildren. But I am also not an educator. I'd be happy to work with
>> someone with an educational background to put something together that might
>> be suitable as a test for new users.
>>
>> For now, though, I'd suggest we stick to our plan to see where it gets us.
>>
>> Christian
>> On Mon, Jan 25, 2010 at 11:20 AM, Gary C Martin <gary at garycmartin.com>
>> wrote:
>>>
>>> Hi Christian,
>>>
>>> On 23 Jan 2010, at 15:01, Christian Marc Schmidt wrote:
>>>
>>> > Hi all:
>>> >
>>> > For our meeting at 10:30pm EST today, I put together a draft of a test
>>> > protocol (in email body below). Please look it over and feel free to edit or
>>> > add to. Thanks,
>>> >
>>> > Christian
>>> >
>>> > --
>>> > Sugar Home View Concept Testing
>>> > Draft 01/23/10
>>> >
>>> > Goal: Evaluate the Home view for "Resume activity" and "Start new
>>> > activity" functionality.
>>> > Protocol: Alternate between two builds, one per student: The current
>>> > build [Build A], which makes "Resume" the default action, and a test build
>>> > [Build B] making "Start new" the default action.
>>> > Duration: 5 mins per student
>>> > Student Profile: Should be a current user (not a first-time user). Age
>>> > and grade should be recorded, along with duration of overall Sugar
>>> > experience.
>>> >
>>> > 1. Begin at Home screen after boot up
>>> >
>>> > 2. Test cognitive model
>>> >       • "Describe what you see in this view."
>>> >       • "What are the symbols in the ring?"
>>> >       • "Describe what each symbol in the ring is."
>>> >       • "Do you know why the symbols are different colors?"
>>> >       • "What activity did you last work on?"
>>> > 3. Test pathways
>>> > [Alternate between two pathways, one per student]
>>> >
>>> > Pathway 1:
>>> >       • "Show how you would go back to the activity you last worked on.
>>> > Describe what you are doing." [Student shows pathway to resume activity.]
>>> >       • "Now, show what you would do if you wanted to start a new
>>> > [Write/Draw/Browse, etc.; should be of the same kind] activity. Describe
>>> > what you are doing." [Student shows pathway to start a new activity of the
>>> > same kind.]
>>> >       • "Now, show what you would do if you wanted to start a new
>>> > [Write/Draw/Browse, etc.; should be a different activity than the current
>>> > one] activity." [Student shows pathway to start a new activity different
>>> > from the current activity.]
>>> > Pathway 2:
>>> >       • "Show how you would start a new activity. Describe what you are
>>> > doing." [Student opens a new activity.]
>>> >       • "Now, show what you would do if you wanted to go back to the
>>> > activity you last worked on. Describe what you are doing." [Student shows
>>> > pathway to resume a previous activity.]
>>> >       • "Now, show what you would do if you wanted to start a new
>>> > [Write/Draw/Browse, etc.; should be of the same kind] activity. Describe
>>> > what you are doing."[Student shows pathway to start a new activity of the
>>> > same kind.]
>>> > 4. General questions
>>> > "If you could change anything about Sugar, what would make it easier for
>>> > you to go back to a previous activity?"
>>> > "What would make it easier for you to start a new activity?"
>>> > "Is there anything else about Sugar that you would change?"
>>>
>>> I know we've drifted from a usability study model, to a focus group model
>>> for practical classroom reasons – but I though Jakob Nielson's latest
>>> article was a timely piece worth a quick read through given we are going to
>>> be getting feedback from existing Sugar users.
>>>
>>> "Summary: It's more difficult to conduct usability studies with
>>> experienced users than with novices, and the improvements are usually
>>> smaller. Still, improving expert performance is often worth the effort."
>>>
>>>        http://www.useit.com/alertbox/experienced-users.html
>>>
>>> Regards,
>>> --Gary
>>
>>
>> --
>> anything at christianmarcschmidt.com
>> 917/ 575 0013
>>
>> http://www.christianmarcschmidt.com
>> http://www.linkedin.com/in/christianmarcschmidt
>> http://twitter.com/cms_
>>
>> _______________________________________________
>> Sugar-devel mailing list
>> Sugar-devel at lists.sugarlabs.org
>> http://lists.sugarlabs.org/listinfo/sugar-devel
>>
>>
> _______________________________________________
> Sugar-devel mailing list
> Sugar-devel at lists.sugarlabs.org
> http://lists.sugarlabs.org/listinfo/sugar-devel
>