[Sugar-devel] [Testing] Fedora Sugar Test Day - Test case content, location

[Thomas C Gilliard]

On 03/03/2012 07:35 PM, S Page wrote:
> Hey all,
>
> @Tabitha, I made your template prominent in
> http://wiki.laptop.org/go/Activity_testing_template and continued the
> archivist job of marking old stuff as "{{obsolete|from 2008}}"...
>
> Is _anything_ under http://wiki.laptop.org/go/Tests still a useful
> test case?  Maybe a mass delete or rename to Tests/old crap/Xyz is in
> order?
> A lot of those tests are linked from activity pages, e.g.
> http://wiki.laptop.org/go/Memorize has a TST link, but most activities
> have moved to sugarlabs.org where none (?) of them have testing
> information.
>
> @Sridhar, re:
>>> The main piece of advice I'm getting is that we need a good test case
>>> management system that allows for linking with defects. The links need
>>> to be able to be tracked, searched and sorted.
> "Passive voice is to be avoided".  Before spending time on features
> like that, be very clear who exactly is going to use them. You can do
> powerful querying on the 2008 w.l.o test cases, but... AFAICT nobody
> cared! Maybe all you actually need is to add links to bug reports in
> testing results, and in bug reports add links to the test case (if
> any) the tester was following when the bug occurred.
>
> On Sun, Feb 26, 2012 at 11:34, Samuel Greenfeld<greenfeld at laptop.org>  wrote:
>> ...[lots]...
> Thanks for the background, very perceptive.  As always I'm eager to
> assist in the wiki side of things.
>
> == Comments on various approaches ==
>
>> In my case I've been told to use a spreadsheet.
> (More passive voice, love it ;-)
> Did you see the spreadsheet in
> <http://wiki.laptop.org/go/Community_testing_meetings/2008-11-06/Displaying_testing_metrics_in_motivating_ways>
> ?
> One nice thing about using a Google Docs spreadsheet is you can
> publish a form front-end that anyone can use to add a row for a test
> result, see Carl Klitscher's form front-end
> <https://spreadsheets.google.com/viewform?key=pyBIsSK_3IlsHBpwk1EFNcQ>
> ? @Tabitha, were you among the Wellington testers that supposedly used
> this? If so, did you like it?
>
> I think the Sugar activity testing spreadsheet
*1-)Tests-Sugar-0.94.1* resulted from a need to find 0.94 sugar 
compatible activities
Authored by Alan Jhonn Aguiar Schwyn It is flexable as a spreadsheet. 
with most of the features a spreadsheet offers.
> <https://docs.google.com/spreadsheet/ccc?key=0AntaXnq4oy2_dHpvZHhLeGRQYzc1cDlRZU9Mc1NldGc#gid=0>
> appends columns for each test result, which is not amenable to a form
> front-end.
2-) full Open Office spreadsheet
  http://people.sugarlabs.org/Tgillard/Activities-Index-ASLO-f13-Mirabelle-f14-rawhide-Soas-tests.ods
and posting it on sugarlabs.wiki with links. This is hard use for a 
number of reporters

3-)Another approach for activity testing are this wiki tables split list 
on the  sugarlabs wiki that I have been maintaining:
I was guided to the wiki table approach early on as a good compromise.
Activity tests A-I
http://wiki.sugarlabs.org/go/Sugar_Creation_Kit/sck/Activity_Matrix%28A_to_I%29
Activity tests J-Z
http://wiki.sugarlabs.org/go/Sugar_Creation_Kit/sck/Activity_Matrix%28J_to_Z%29

An older first attempt is listed here:
  http://wiki.sugarlabs.org/go/Sugar_on_a_Stick_QA

fedora for their test day testing uses wiki tables. . Here they do use 
links to subpages of wiki tables for the actual test descriptions and 
testing results.

  http://fedoraproject.org/wiki/Test_Results:Fedora_17_Alpha_RC4_Base  
(is a sample from fedora 17  testing.)

These can be complex structures. I am not certain that anything is 
gained here by the complexity. But you can link to a large number of 
different tests and list their results

This format can be more difficult to use as the reporters have to be 
familiar with wiki table editing.
but it does have sortable columns and can include links to other 
information.

4-)Fedora also uses a master "tracking bug" in bugzilla that lists all 
of the outstanding bugs for a release and close them as issues are 
fixed. this does include links and attachments.
.
> The Fedora Test Day approach on the //fedoraproject.org wiki
> encourages tabular reporting of test results, but I haven't found a
> page with any results,  e.g.
> <http://fedoraproject.org/wiki/Test_Day:2010-08-19_Sugar>. It's
> similar to the external spreadsheet approach. The choice between them
> depends whether wiki linking (e.g. "What links here" to find out what
> test days exercised a particular test case) is more important than
> spreadsheet jockeying.
>
> I think<http://wiki.sugarlabs.org/go/Community/Distributions/Fedora-SoaS#Testing_Results>
> is pretty unusable. It would be much better to create a subpage for
> each testing result, like SL.o used to e.g.
5-)

http://wiki.sugarlabs.org/go/Community/Distributions/Fedora-SoaS#Testing_Results

a single wiki page as it is more flexible for reporters to edit and for 
the kind of information it contains.
The only indexing is via the table of contents and linking.

I hope that this summary of the way I have looked at test result 
reporting is helpful.

Tom Gilliard
satellit_ on #sugar IRC freenode
> <http://wiki.sugarlabs.org/go/SoaS/Testing/Soas2-200905031329>  , and
> then use {{subst:}} or Special:PrefixIndex to pull in or reference
> these from other pages.
>
> I summarized the four (so far!) ways sl.o has collected test results
> in<http://wiki.sugarlabs.org/go/SoaS/Testing>  .
>
> Good luck,
> --
> =S Page
> _______________________________________________
> Testing mailing list
> Testing at lists.laptop.org
> http://lists.laptop.org/listinfo/testing
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20120303/a4f1cd47/attachment-0001.html>