[Sugar-devel] Unit testing an activity

[Sascha Silbe]

"Taylor Rose (RIT Student)" <tjr1351 at rit.edu> writes:

> Does anyone have advice about how to run unit tests for an activity? I am
> working on a video conferencing activity called Open Video
> Chat<https://github.com/FOSSRIT/Open-Video-chat>.
> I'd like to be able to run some basic unit tests on the activity but I'm
> not sure how to instantiate an activity in a testing environment or if it's
> even possible. Can anyone offer advice or point me to an example? I'd like
> to be able to mock a connection and make sure the activity reacts properly,
> starting up gstreamer and such.

While the test support infrastructure for Activities is in better shape
than that for Sugar, it's mostly a set of general components, prototypes
and hacks that you have to tape together yourself. Especially automated
testing of shared Activities is going to hit some rather rough edges. If
you're willing to dive in nonetheless and get it all working well, we'd
appreciate you documenting the process so other Activity authors benefit
From your effort.

I'll describe the parts starting on the system layer.

As you want to include Collaboration in your tests (which makes sense
for Open Video Chat), you'll need two different systems. There are
various hacks to test Sugar with just a single system, but we don't
really know when they work and when they break, so I'd recommend
sticking with two entirely separate systems for now. Start the test
suite inside the Terminal activity on each system, distinguishing
between them by passing an option (Initiator vs. someone joining the
shared instance). That will bypass all the pitfalls of trying to remote
control a Sugar session from a different system. The two systems can
also be VMs on one physical host if they share a multicast-capable
network connection (e.g. private network between the VMs or bridging to
the physical network, but no NAT setup like the KVM user mode network
stack). Synchronisation between the two instances happens naturally in
your use case of testing Collaboration.

The Python unit testing framework shipped with Python 3 and backported
to Python 2.7 (as "unittest2") is a natural choice for any Python based
application. It includes support for skipping tests (e.g. for
functionality that's not available in older versions of Sugar or for
tests that are hardware-specific) and marking tests as being expected to
fail, either always (e.g. for test-driven development, where you write
the test first) or based on some condition (e.g. if some versions of
Sugar contain a bug that causes the test to fail). There are hooks
(TestCase.setUp() and TestCase.tearDown()) to run code before and after
running each individual test to prepare the environment for the test
(e.g. start resp. stop the Activity). If you need to run code before and
after complete sets of tests (e.g. if you want to split up a sequence of
steps into individual tests, keeping the Activity running between the
tests), please contact me. I've already implemented this for several
test suites and can show you how to use it.

Launching the activity is where it gets tricky. The initiator side would
be easy enough using sugar-launch, but it has no support for joining a
shared activity. It'll also assign the activity_id randomly, so you'd
either need to determine the activity_id on the joiner side by scanning
for shared activities or control the Sugar platform UI (the "Shell") by
scanning for the shared activity icon in the Neighbourhood (using a
special icon for the Activity running in test suite mode, so it doesn't
try to connect to somebody else instead) and simulating a click. The
third and probably best option is to tap into the semi-privates of
Sugar, giving you more control of the Activity session creation
process. Assign a fixed, hard-coded activity_id (use
sugar.activity.activityfactory.create_activity_id() to generate it) so
that the both sides know it. For the initiator side, set
sugar.activity.activity.Activity._activity_id to the fixed activity_id
and call Activity.share() on start-up. See below for how to tell your
activity it should do this magic. For the joining side, copy the code of
sugar-launch that you need to start you activity and enhance it to pass
the activity_id to sugar-activity (CLI option "-a").

If you're going to do the tests on the code level (rather than on the UI
level) you're almost set if you get all of the above working. You can
phone in to your activity by setting an environment variable in your
test suite when calling sugar-activity to tell your activity to enter
"test mode". There's a number of ways to remote control the activity if
you want to the test code to run outside the activity, or you could
set the environment variable to the name of the test you want to run and
have the activity execute it.

For tests on the UI level, we're entering less than firm ground
again. You could try SugarBot [1-4]. Given the way it works [5], I have
a feeling it would be a major effort to port SugarBot to GTK3. If you
don't get dogtail [6] working (it requires several additional tools
running in the background in order to work at all), SugarBot may be a
good intermediate step, however. You can always switch to a different
framework later when we've made the necessary preparations in Sugar,
which is likely to happen during the migration to GTK3.

Thanks for your attention. I'll be happy to elaborate if you want to
know more about any of the above, or anything else that comes up in this
context.

Sascha

[1] http://wiki.sugarlabs.org/go/BugSquad/SugarBot
[2] http://code.google.com/p/sugarbot/w/list 
[3] http://gsoc-sugarbot.blogspot.com/
[4] http://code.google.com/p/sugarbot/downloads/detail?name=ScreenCast2.mp4&can=2&q=#makechanges
[5] http://code.google.com/p/sugarbot/wiki/HowDoesSugarbotWork
[6] https://fedorahosted.org/dogtail/
-- 
http://sascha.silbe.org/
http://www.infra-silbe.de/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 489 bytes
Desc: not available
URL: <http://lists.sugarlabs.org/archive/sugar-devel/attachments/20120507/0982f94a/attachment.pgp>