[Systems] runaway apache processes on sunjammer

[dfarning at sugarlabs.org]

On Tue, Nov 24, 2009 at 3:27 AM, Ivan Krstić <krstic at solarsail.hcs.harvard.edu> wrote:
> David pointed out a large number of Apache processes on sunjammer
> coming up earlier tonight and then again after a restart. Most of
> these were in I/O sleep:
>
> root at sunjammer:/srv/home/krstic# ps ax -o pid,state,command | grep
> apache | grep D | wc -l
> 239
>
> My first guess is that NFS is doing something stupid. A look at
> mount(8) output shows several overlapping NFS mounts, some of which
> are obviously misspelled, and others are just weird (why are there
> loopback NFS mounts mounted to actual directories?).
>
> Anyway, whoever set this up ought to clean up the NFS configuration
> and eliminate the stray mounts. My guess is Apache will keep building
> up dead processes until then.
>

I umounted _all_ of the nfs mounts and replace them with symlinks on sunjammer.

The overall goal is that each aslo-web node must have access the same data:
1. Common database 
2. Common activities/[files/staging] directories

The common data base is pretty straight forward.  The common directories are somewhat more problematic.   All of the distributed file systems seem like overkill.  I have been trying NFS shares.  My biggest concerns with NFS are the single point of failure of the server and security implications.

Upstream, mozilla handles the issues with a high availability SAN with private network to the various aslo nodes.  We don't have that sort of money yet. So we need a somewhat cheaper solution.  Bernie is working on an interconnect between sunjammer and treehouse.  But we we still have the issues of sharing files between multiple aslo-web nodes.

There is no real hurry on this anymore. I think we have a several weeks before sunjammer starts to crumble under the load.... again.

david
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 271 bytes
Desc: OpenPGP digital signature
Url : http://lists.sugarlabs.org/private/systems/attachments/20091124/a83794af/attachment.pgp