[Systems] Excessive scraper queries

[Bernie Innocenti]

Hello Ignacio,

while I was investigating a logs bloat issue on sunjammer, I noticed
that apache is being hit by about 1 million requests per day from a
python script running locally:

  activities.sugarlabs.org:80 2001:4830:134:7::11 - -
[23/Mar/2014:01:59:36 -0400] "GET /es-ES/sugar/addon/4461 HTTP/1.1" 200 3
7522 "-" "Python-urllib/2.6"
 activities.sugarlabs.org:80 2001:4830:134:7::11 - -
[23/Mar/2014:01:59:38 -0400] "GET /es-ES/sugar/addon/4467 HTTP/1.1" 200 8
55 "-" "Python-urllib/2.6"
 [...]

This is not necessarily forbidden, but I'd like to understand what this
script does and whether it *really* needs to run so aggressively.

Note that the files are available on the local filesystem, there's no
need to read them periodically over http.

-- 
Bernie Innocenti
Sugar Labs Infrastructure Team
http://wiki.sugarlabs.org/go/Infrastructure_Team