There are a few things I think need to be clarified:<br><ol><li>What are we looking as a community to get out of this Fedora Test Day event?</li><li>What is the purpose of a "test case" and/or "test plan"? How detailed should they be? Presuming we want them, how should they be stored in general, and for this particular scenario?<br>
</li></ol><br>For #1, I disagree that having users start out for this occasion by learning how to use Sugar on their own is appropriate for a few reasons:<br><ul><li>The amount of time that any given tester will be available to help us out likely is limited. Time spent doing discovery will not be spent on other tasks.</li>
<li>Not everyone will start at the same time. Perhaps calling this a "Test Day" is a misnomer because there is no guarantee that testers will be in the time zone or country.</li><li>Sugar is a relatively stable platform with a few known recurring UI disputes. It is not Nell, the Helicopter experiment, or another one where the user interface could potentially require major changes.<br>
</li></ul>With this test day, my personal view is that we need to get feedback verifying basic Sugar and activity functionality in Fedora. When Peter Robinson, Kalpa Welivitigoda, or someone else updates a Sugar software package in Fedora, these packages often go through the verification process without a single person commenting on if the proposed update worked or not.<br>
<br>Mind you, usage feedback is appreciated; but is more of a secondary concern to me. Fedora had "Fit and Finish" test days during Fedora 12 cycle where they asked for general usage feedback; perhaps we can propose that they do another round of those aimed at the various desktop environments with Fedora 18.<br>
<br><br><br>For #2, I have used similar test templates to the the New Zealand's to verify activities in the past, and was thinking of making one available in this case. Translating one into a Wiki template would make it straightforward to clarify which activities support sharing, webcam usage, etc. <br>
<br>The reason I am interested in maintaining test cases with a system to keep a historical log of who did what when is because I want to be able to parallelize tasks. Although I recognize I could be more efficient, there simply is too much material in Sugar and the XO platform for one person to focus on. And yet I get regularly asked "do you remember bug #123" or "when was the last time someone looked at Q"?<br>
<br>I am not looking for detailed test cases to the point of listing which buttons to click when; but rather simple ones like "Does it install?" and "Can it open a saved document?".<br><br>Fedora takes a curious approach to this in that they write a series of test cases which could be parallelized, but then offer to have everyone run the same set of test cases. And usually, pretty much everyone runs most of the available items. <br>
<br>For comparison, look at the last GNOME 3 test day (<a href="http://fedoraproject.org/wiki/Test_Day:2011-04-21_GNOME3_Final">http://fedoraproject.org/wiki/Test_Day:2011-04-21_GNOME3_Final</a>) versus the last Sugar one (<a href="http://fedoraproject.org/wiki/Test_Day:2010-08-19_Sugar">http://fedoraproject.org/wiki/Test_Day:2010-08-19_Sugar</a>).<br>
<br>I'm open to taking suggestions back to the Fedora Testing mailing list if someone has an idea on how we could do things better, but I'm trying to avoid cross-posting too much. We could also inquire on the Fedora QA mailing list as to who might participate and what their skill levels are so we can better tune our approach.<br>
<br><br><br><div class="gmail_quote">On Sat, Feb 18, 2012 at 5:04 AM, Tabitha Roder <span dir="ltr"><<a href="mailto:tabitha@tabitha.net.nz">tabitha@tabitha.net.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On 17 February 2012 08:36, Samuel Greenfeld <span dir="ltr"><<a href="mailto:greenfeld@laptop.org" target="_blank">greenfeld@laptop.org</a>></span> wrote:<br></div><div class="gmail_quote"><div class="im">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On March 22 there will be a Sugar test day for Fedora 17. This means that the Fedora community in general will be gathering to look at Sugar and see what issues we have close to the end of the Sugar 0.96 cycle.<br><br></blockquote>
</div><div><br>While test cases can be useful, I always try to start with some discovery time as this is when you can get some feedback on design and intuitive behaviour (though this is impacted by use of other systems with many users). Something like:<br>
"Find a friend. Work together to discover how to open the laptop if you have an XO, or start Sugar. Together try clicking on things and see if you can learn how to play any games or complete any activities. Can you find ways to take photos, write stories, make music."<br>
After that, get their feedback on how that went before giving them a test case. First time users of Sugar can also give you feedback on their experience of first use of an activity while following testing instructions. There have been a number of occasions when I have said "oh, you have to click on that first and then click on that other thing" and they have said "why is it designed like that?" which really makes us rethink about the design of activities. <br>
Our basic activity testing template (written a long time ago) is here - <a href="http://wiki.laptop.org/go/Activity_testing_template#The_NZ_activity_test" target="_blank">http://wiki.laptop.org/go/Activity_testing_template#The_NZ_activity_test</a><br>
</div></div><br>On the topic of tracking testing, we have looked at a number of options here in NZ and I think Australia also looked at a number of options. In NZ we tried writing them on <a href="http://wiki.laptop.org" target="_blank">wiki.laptop.org</a> but that didn't really work. My personal method of managing test requests is to try to tag the requests (or potential requests) in my email inbox when they arrive and then test them on Saturday, archiving off emails as things are tested. This only works for us because we meet in one place, not a suitable solution for multiple testing locations. I personally don't think we should add anymore systems, but look at ways to use existing systems to manage testing - such as the two bug trackers we already use or the <a href="http://activities.sugarlabs.org" target="_blank">activities.sugarlabs.org</a> site. <br>
<br>Hope this helps<span class="HOEnZb"><font color="#888888"><br>Tabitha<br>
</font></span></blockquote></div><br>