[Systems] Trac maintenance

[Tomeu Vizoso]

On Tue, Sep 15, 2009 at 07:39, Bernie Innocenti <bernie at codewiz.org> wrote:
> == Hostname change ==
>
> Our trac instance is now reachable as bugs.sugarlabs.org.  The old
> hostname dev.sugarlabs.org is not going away anytime soon, but
> please update your links for consistency.
>
> == Plugin updates ==
>
> I refreshed all plugins with the latest versions and generally cleaned
> up the installation as much as I could.  This is what we're presently
> running:
>
>  IniAdmin-0.2-py2.5.egg
>  TracAuthOpenId-0.2.1dev-py2.5.egg
>  TracCustomFieldAdmin-0.2.2-py2.5.egg
>  TracHTTPAuth-1.1-py2.5.egg
>  TracSpamFilter-0.2.1dev_r8330-py2.5.egg
>  TracXMLRPC-1.0.6-py2.5.egg
>
> If nobody is relying on it, I'd like to disable the XMLRPC and its
> dependency HTTPAuth to simplify the configuration.

Sounds good to me.

> == Spam ==
>
> I installed the TracSpamFilter plugin in the hope to reduce spammers who
> have been riddling our database with bogus tickets.
>
> A while ago I disabled wiki editing even for normal users, but the logs
> are still full of failed attempts by spammers to create new spam pages:
>
>  75.147.59.54 - - [15/Sep/2009:00:26:45 -0400] "GET /wiki/30_buy_vitamin_b_12?action=delete&version=2&old_version=1&delete_version=Delete+version+2 HTTP/1.1" 200 4322 "http://bugs.sugarlabs.org/wiki/30_buy_vitamin_b_12?action=diff&version=2" "Mozilla/5.0 (X11; U; Linux x86_64; es-AR; rv:1.9.1.3) Gecko/20090911 Fedora/3.5.3-1.fc12 Firefox/3.5.3"
>  75.147.59.54 - - [15/Sep/2009:00:26:56 -0400] "POST /wiki/30_buy_vitamin_b_12 HTTP/1.1" 303 - "http://bugs.sugarlabs.org/wiki/30_buy_vitamin_b_12?action=delete&version=1" "Mozilla/5.0 (X11; U; Linux x86_64; es-AR; rv:1.9.1.3) Gecko/20090911 Fedora/3.5.3-1.fc12 Firefox/3.5.3"
>
> At this point, these are just a big waste of cpu time.  I'm not sure
> it's worth filtering the offending IPs.

It could be if it would make Trac more usable.

> == Database cleanup ==
>
> We should come up with an SQLite recipe to get rid of all the pages they
> created when the editing policy was more permissive:
>
>  http://bugs.sugarlabs.org/wiki/TitleIndex
>
> And there are dozens of bogus user accounts to kill.  The older ones are
> easy to spot because they're all named similarily: alexqz3518,
> burnessct4097, ethanvj5914...
>
> The newest ones are subtler:
>
>   AmuttCaralirm AmuttCaralirm almeidababich69674 at gmail.com
>   BliZSaRdChiCK BliZSaRdChiCK user1196 at dorogen.in
>   ...
>
>
> == Performance ==
>
> In an attempt to solve the longstanding performance issue with our trac
> instance, I've rebuilt and installed the latest versions of all the
> plugins we were using.
>
> It didn't help much:
>
>  bernie at solarsail:~$ time wget http://bugs.sugarlabs.org/ticket/17 2>/dev/null
>  real  0m12.149s
>  user  0m0.004s
>  sys   0m0.008s
>  bernie at solarsail:~$ time wget http://bugs.sugarlabs.org/ticket/17 2>/dev/null
>  real  0m4.107s
>  user  0m0.008s
>  sys   0m0.008s
>  bernie at solarsail:~$ time wget http://bugs.sugarlabs.org/ticket/17 2>/dev/null
>  real  0m6.097s
>  user  0m0.012s
>  sys   0m0.004s
>
> :-(
>
> It seems that 4sec is the lowest time it takes to display an ordinary
> ticket.  The slowest responses are most probably caused by concurrent
> requests of spammers, search engines and regular users.  Solarsail has
> 32 processors, but our Trac setup with mod_wsgi is single threaded.

If search engines are having a big impact, what about a robots.txt file?

> There's also a buildbot instance running on solarsail, but it's running
> with nice 19 and disk I/O shouldn't affect trac too much.
>
> If we can't figure out how to optimize Trac at least 500% by next week,
> I propose moving it to a new VM on treehouse, which has "just" 8 cores,
> but each one should be 10 times faster.

Sounds good to me.

> Another solution would be migrating our ticket datbaase to
> launchpad.net.  From recent discussions with the Launchpad folks, it
> seems we'll to wait for their next release cycle to get some features we
> need to integrate LP's bug tracker nicely with the rest of our
> infrastructure.  If we were really desperate, we could give up on some
> of our requirements and rush a migration within next month.

Well, I expect that other Trac users aren't having to wait so much for
a query to return and they should have similar spam and search engines
load, right?

Thanks a lot for looking at this,

Tomeu

> (please, let's not discuss the LP migration in this thread.  Actually,
> please let's not discuss it at all yet.  We're still talking with
> Canonical and others to explore what long-term hosting possibilities
> make sense for a project of our size.
>
> --
>   // Bernie Innocenti - http://codewiz.org/
>  \X/  Sugar Labs       - http://sugarlabs.org/
>
> _______________________________________________
> Systems mailing list
> Systems at lists.sugarlabs.org
> http://lists.sugarlabs.org/listinfo/systems
>



-- 
«Sugar Labs is anyone who participates in improving and using Sugar.
What Sugar Labs does is determined by the participants.» - David
Farning