[Systems] mirror testing request

[David Farning]

On Mon, Sep 21, 2009 at 7:52 PM, Adrian Chadd <adrian.chadd at gmail.com> wrote:
> I'd be happy if Cacheboy was handling all of your downloads. My
> distribution system is slightly more flexible than bouncer and is
> network topology aware.

This takes us back to the questions.
Is cacheboy stable enough that we could host the entire infrastructure on it.
What would we need to do to run sugarlabs.cdn.cacheboy.net at our
server facility?

I remember you saying that cacheboy required root on mirror nodes
because it runs as a cluster.

david



> Adrian
>
> 2009/9/22 David Farning <dfarning at sugarlabs.org>:
>> On Mon, Sep 21, 2009 at 3:54 PM, Marten Vijn <info at martenvijn.nl> wrote:
>>> On Mon, 2009-09-21 at 16:35 -0400, Luke Faraone wrote:
>>>> On Mon, Sep 21, 2009 at 16:29, Marten Vijn <info at martenvijn.nl> wrote:
>>>>         What is A.SL.O ?
>>>>
>>>
>>> ah clear, these links do not point to download as far i can see or are
>>> redicted.
>>
>> Currently a.sl.o downloads point to activities.sugarlabs.org.  We can
>> set it up to point to dl.sl.org if there is an advantage.
>>
>>> Anyway, currently only:
>>>
>>> http://download.sugarlabs.org/ is mirrored. to all mirros.
>>>
>>> So if there is copy of activities@ on download@ it will be
>>> picked up soon 1-24 hrs.
>>
>> Ok got it.  Mozilla uses a system called bouncer to control its mirror
>> net work.  Whenever a file is written to downloads.mozilla.org.
>> Bouncer starts immediately propagating the file across the mirror
>> system.  Once bouncer has verified that a particular mirror has a good
>> copy of the file, it starts redirecting requests to that mirror.
>>
>> It seems that that is similar to how things work within cacheboy.  But
>> we currently have a manual step of rsyncing from dl.sl.o to cacheboy,
>> dl2.sl.o, and the european mirrors you have set up.  This gives us a
>> two tier mirror system
>>
>> dl.sl.o
>>  dl2.sl.o
>>  ftp.nluug.nl
>>  ftp.nluug.nl
>>  sugarlabs.cdn.cacheboy.net
>>
>> A user manually chooses between one of the above mirrors.  Rsync runs
>> on a cron job.  Health is monitored by Marten.
>>
>> sugarlabs.cdn.cacheboy.net
>>  *.cdn.cacheboy.net
>>
>> If a user choses sugarlabs.cdn.cacheboy.net , they are redirected
>> around the cacheboy network based on location and load.  Files are
>> pushed to mirrors as needed.  cacheboy monitors individual mirror's
>> health and removes them as necessary.
>>
>> This leads to two questions:
>> Is cacheboy stable enough that we could host
>> sugarlabs.cdn.cacheboy.net at Gnaps and alias dl.sl.o to
>> sugarlabs.cdn.cacheboy.net ?  This would put the entire download
>> infrastructure on cacheboy.
>>
>> or
>> Should we look into Bouncer or Fedora Mirror Manager to manage and
>> redirect within the first level of mirrors?
>>
>> david
>>
>>> There is no system yet to do geoip based redirections(as far as i know).
>>>
>>> Yes it should be there, IHMO it should be in some
>>> kind of cgi?
>>>
>>> apt-get install  libapache2-mod-geoip?
>>>
>>> I never played with it, and I don't trust myself
>>> to write sane cgi's. So if we have talents on board,
>>> it is time to use them.....
>>>
>>> Feel more than free to take over from here. I would prefeer to chase
>>> some more mirrors.....
>>>
>>> cheers,
>>> Marten
>>>
>>>>
>>>>
>>>> http://activities.sugarlabs.org/.
>>>>
>>>>
>>>> --
>>>> Luke Faraone
>>>> http://luke.faraone.cc
>>> --
>>> this email is sent from my mainframe.....
>>> http://martenvijn.nl                Sugar on a Stick
>>> http://bsd.wifisoft.org/nek/        The Network Event Kit
>>> http://opencommunitycamp.org         26th Jul - 2nd August
>>>
>>>
>>
>