[Systems] api.sugarlabs.org

[David Farning]

On Fri, Oct 9, 2009 at 7:39 PM, Bernie Innocenti <bernie at codewiz.org> wrote:
> David,
>
> some time ago you asked for an account on one of the Fedora buildslaves
> to revive the buildbot for api.sugarlabs.org.  Sorry it took so long, I
> had a tricky network issue with the build slaves.
>
> Once you have IPv6 working on your computer or on another host, try
> logging in with this command:
>
>  ssh -6 dfarning at buildslave-fedora-11-64bit.sugarlabs.org.sugarlabs.org
>
> You should also be able to sudo as root.
>
> Q.:
>
> 1) does it have to be Fedora for some reason?

Anything jhbuild runs on is fine.  The original choice of Fedora was
because jhbuild tended to build more frequently on fedora in the early
days--- this constraint is no longer true.

> 2) how much time does the generator take to complete a run?

On the slave It take the same time as a jhbuild rebuild + about 20
seconds to parse the new tree + a couple of seconds to rsync the newly
generated documentation against the documentation on sunjammer.  If a
build fails for any reason it just stops and waits for a the next
iteration to try again

> 3) can we update the documentation once per day, with a low
>   priority job?

Yes, it is scheduled for once a day now.  it is just a matter of add
nice to the script

> 4) is the process of generating the documentation relatively secure,
>   or does it involve running python code checked out from the
>   repositories?

It is currently set up as slave-> insecure and master-> secure.  All
of the building and parsing happens on the slave.  The master runs a
cron job to triggers the build, scp's the buildscript to the slave,
and waits for the slave to rsync the results back.

> If we didn't have any of these constraints, I'd suggest to use sunjammer
> for this task rather than the buildslaves, which should be seen as
> relatively insecure, throw-away systems.

When we set this up a little over a year ago the jhbuild process was
little flaky.  I didn't want to do run it on sunjammer.  It is
embarrassingly simple:)

david

--api-server.sh --------------------------------------------------
#!/bin/bash

SERVERDIR="."

SLAVEACCT="apislave"
SLAVENAME="sugarlabs1.xen.prgmr.com"
SLAVEDIR="apidocs"

# bash check if directory exists
if [ ! -d $SERVERDIR ]; then
       mkdir $SERVERDIR
fi

scp api-slave.sh $SLAVEACCT@$SLAVENAME:api-slave.sh

ssh $SLAVEACCT@$SLAVENAME './api-slave.sh > api-slave.log'

scp  $SLAVEACCT@$SLAVENAME:api-slave.log api-log

rsync   --archive \
        --verbose \
        --compress \
        --rsh=ssh \
        $SLAVEACCT@$SLAVENAME:$SLAVEDIR $SERVERDIR

--api-slave.sh --------------------------------------------------
#!/bin/bash
set DISPLAY=:103

BUILDDIR="sugar-jhbuild"
DOCDIR="apidocs"

# bash check if directory exists
if [ -d $BUILDDIR ]; then
        pushd $BUILDDIR
        git-pull
        popd
else
        git-clone git://git.sugarlabs.org/sugar-jhbuild/mainline.git $BUILDDIR
fi

pushd $BUILDDIR
./sugar-jhbuild --no-interact
popd


epydoc  $BUILDDIR/install/lib/python2.5/site-packages/sugar \
        --verbose \
        --html \
        --docformat="restructuredtext" \
        --show-private \
        --show-sourcecode \
        --include-log \
        --name="Sugar learning platform" \
        --css="white" \
        --url="http://api.sugarlabs.org" \
        --navlink="<a href='www.sugarlabs.org'>Sugar Learning Platfrom</a>" \
        --graph="all" \
        --dotpath="/usr/bin/dot" \
        --fail-on-error \
        --output=$DOCDIR

-----------------------------------------------------------------------------------