[IAEP] Downloads

[Bernie Innocenti]

C. Scott Ananian wrote:

> We have plenty of disk space on http://download.laptop.org as well;
> it's just looking for content.  For example, I'd be willing to mirror
> sugar stuff in http://download.laptop.org/mirrors/ , which would let
> you locally administer the content on sugarlabs.  But if you've got
> plenty of bandwidth, there might not be a point to excessively
> mirroring this stuff; it might just make it confusing if/when the
> mirrors drift out of date.

Yeah, download.sugarlabs.org has a big pipe too, and the number of
downloads at this time is not overwhelming.


> On the other hand, I've found that mirroring combined with round-robin
> DNS is a great way to distribute load and improve uptime, as machines
> can be added/dropped from the DNS pool without affecting availability
> of the content.  Let me know if you're interested in that.  I've got a
> HOWTO guide somewhere I wrote up in the freesklyarov.org days that I
> can dig up.

The HA setup is pretty simple to setup both sides, and beneficial.
I'd like to setup a "push" style mirror, so we know they're all up
to date, and I can centrally change the frequency of the updates
and serialize them on the mirrors.

I'll do the DNS trickery and rsync cron job on this side,
but first download.laptop.org needs to setup like this:

 1) name-based virtualhost download.sugarlabs.org on port 80,
    with documentroot pointing to the mirror.  For example:

     <VirtualHost *:80>
         ServerName download.sugarlabs.org
         ServerAlias download2.sugarlabs.org
         DocumentRoot /pub/sugarlabs
         <Directory /pub/sugarlabs>
             Options +SymLinksIfOwnerMatch +Indexes
         </Directory>
         ErrorLog "logs/sugarlabs_error_log"
         CustomLog "logs/sugarlabs_access_log" combined
         CustomLog "logs/access_log" vcombined
     </VirtualHost>

 2) a user (bernie or whatever) with ssh access, to the box.
    Please use this key set:
      http://www.codewiz.org/pub/authorized_keys.trinity

 3) put bernie in a group such as "sugarlabs" or sugardl"

 4) make the documentroot writable by that group, with
    BSD groups semantics (+s).

The rsync job will be bandwidth limited at 64KB/s.  For simplicity,
I won't bother making the update atomic (given your background,
I think you don't need further explanations :-)

Also, the problem of discrepancy between multiple mirrors is
bothersome enough for users that we may end up not enabling
transparent DNS load balancing, and put an explicit
download2.sugarlabs.org on the wiki.  What do you think?


> (Note that download.laptop.org is currently a B2 connected to a 1TB
> external hard drive, so it might be worth adding a sugarlabs machine
> as a mirror for it and doing round-robin DNS in that direction as
> well, if you're willing.)

I think Ivan's machine is the most indicated to act as a mirror
for download.l.o.  Mine here does not have the bandwidth to
qualify.

-- 
   \___/  Bernie Innocenti - http://www.codewiz.org/
  _| X |  Sugar Lalbs Team - http://www.sugarlabs.org/
  \|_O_|  "It's an education project, not a laptop project!"