[Systems] Clustering Remora

[Bernie Innocenti]

El Thu, 27-08-2009 a las 19:59 +0200, Marten Vijn escribió:
> As posted some ago download2 does not always have a .stamp file.
> 
> Currenty I am decreasing verbosity.

I had no time to investigate yet.  Do you already have a shell account
on sunjammer?  And would you like to have a look yourself?


> Funny see my posting on cacheboy asked for bandwidth. Getting this for
> them would also help us distributing iso's.  

Answered that.  Let's see what we can get from the GNAPs/MXP folks.


> Think getting more data from trend and monitoring i next important step
>
> Also moving my mirror_check script should be moved to a monitoring
> machine. 

Are you missing a VM to do this?  Me and Dogi are setting up a new KVM
host that should be ready for production within days.

If you'd prefer to use your own resources, go on and do what you think
would work best.  Only, please also document what you do in the wiki and
give access to a few more Infrastructure Team people.  As we grow
bigger, I think it's important that we transition from being a bunch of
individuals to a coordinated team of interchangeable players.


> This could help finding bottle neck. At some point we could need a setup
> with loadbalancers / failover mechs, but I don't think we are there yet.

At this point, bandwidth is not /yet/ a big bottleneck for us.  It will
certainly become a problem at some point, so it's good to start planning
ahead.  Besides, regardless of how much bandwidth we have at the source,
offering geo-located mirrors can always be a good service to users
located far away from the main distribution site.

My #1 concern to scale software distribution would be properly
clustering a.sl.o.  Remora is a fairly large and slow LAMP application,
and no matter how much we can optimize away inefficiencies, we'll
eventually reach a point where no individual machine will be able to
cope with the traffic.

Unfortunately, I'm afraid the application servers will have to sit near
the only database server, or performance will suck.  Perhaps Remora
supports a master-slave MySQL scenario, I haven't investigated.

This would also be another good project waiting for a volunteer sysadmin
to explore.  I can provide all the necessary resources and support.
activities-devel.sl.o. would be a good playing ground, just ask me to
get access and get in contact with Aleksey for coordination.

-- 
   // Bernie Innocenti - http://codewiz.org/
 \X/  Sugar Labs       - http://sugarlabs.org/