[Systems] Discovery One load balancing testing

[dfarning at sugarlabs.org]

On Wed, Dec 2, 2009 at 2:44 PM, Aleksey Lim <alsroot at member.fsf.org> wrote:
> On Wed, Dec 02, 2009 at 02:27:23PM -0600, dfarning at sugarlabs.org wrote:
>> It looks like it is time to enter a feature freeze on Discovery One and spend some time debugging and tuning.
>>
>> Recent Features:
>> Log parser - Yesterday alsroot moved the log parser from sunjammer(acting as a web node) to aslo-proxy.  Our download count has been under recording for the past couple of weeks.  By parsing the squid logs we can catch everthing.
>
> I guess we can disable apache logs to /srv/www-sugarlabs/activities on
> sunjammer

I think so. Is to log parser on aslo-proxy pointing a the database on aslo-database or sunjammer?  I think we are ready to point it at sunjammer if it is not already there.

Because the new stack is not yet in production, activities.sl.o is pointing at sunjammer.  I abused the new stack pretty hard yesterday with no ill effects.  It can sustain a little over 40 transactions per second; about three times what sunjammer can handle alone. 

As soon as Bernie gets a chance to point activities.sl.o at aslo-proxy, it is ready to go into production.  Because we are using the database and NFS mounts from sunjammer, it will be trivial to point activities.sl.o back at sunjammer in case of problems. 

david

>> Awstats - Yesterday we moved an awstats instance to aslo-proxy to help us get a better sense of the actual aslo traffic.  The awstats instance on sunjammer gave us information about total traffic on sunjammer.
>>
>> HAproxy - This morning we enabled HAproxy on aslo-proxy so we can load balance across multiple web-node.
>>
>> Stress testing - I am currently stress testing the new stack with siege.  Sunjammer is still in production.  To prevent interference with normal aslo users, the LB is sending 90% of the requests to aslo-web and only 10% of the requests to sunjammer.
>>
>> It is pretty cool to be able to kill apache on aslo-web and have the benchmark keep running -- albeit much slower.  On the other hand, we need to run siege from a different machine.  We are using a quarter of treehouse's total cpu just to run siege.
>>
>> david
>>
>
> --
> Aleksey
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 271 bytes
Desc: OpenPGP digital signature
Url : http://lists.sugarlabs.org/private/systems/attachments/20091203/b01b2fa9/attachment.pgp