[Systems] Migration to SN testing node urgent

[Aleksey Lim]

On Wed, Oct 15, 2014 at 11:34:01AM -0500, Sebastian Silva wrote:
> Alsroot,
> Greetings,
> We're observing downtime about twice a day now in production instance
> of Sugar Network central node.
>
> Every time I have to log into jita and issue:
>
>  sudo /etc/init.d/sugar-network stop
>  ps -o pid,comm,user,thcount -u www-data | wc -l
>  # ^^ is useful to give an idea of traffic
>  # goes down to ~12 when all SN threads die after a few seconds
>  sudo /etc/init.d/sugar-network start
>
> It's probably all the traffic, but also it seems to have gotten worse
> after the downtime jita had some time ago.
>
> It's stressful for editors/admins and annoying for users.
>
> As I understand it, new node implementation does not have this problem.
>
> I can do the migration myself, if you provide me with some details:
>
> * procedure for migrating database
> * current release tarballs/sources for putting in production
>
> I think it's even better if I do it, then I will have a better sense of
> how the clockwork ticks. I'll attempt to document as I go. Maybe we can
> setup some uptime monitoring this time around (cc: systems@ for this
> purpose).
>
> It would be helpful to coincide when you are online on this task. For
> me a good time would be starting Friday 17th at 21:00 (UTC -5 / Bogota)
> - but I'm open to accomodate to your timezone/schedule/convenience.
> This way we can test over the weekend and have a working service by
> Monday 20th.
>
> Let me know so we can announce the planned maintenance downtime.
>
> We've gotten this far and have engaged some active users. I think there
> is a bright future for Sugar Network. We just need to keep rowing.
> Thanks for your commitment.

The issue is not w/ SN node in particular but with Apache connections
pool, next time restart Apache. Last time we decided to not migrate
node.sl.o to intermediate code base release. So, if the only issue is
unavailable node, it could be started on separate IP out of Apache.

If I got it right, it is possible to grant jita new external IP,
so, we need to ask Bernie. Then,

1.  node.sugarlabs.org DNS should be re-pointed to the new IP

2.  /srv/sugar-network/.config/sugar-network/config should be tuned:

        [node]
        host = <NEWIP>
        port = 80

3.  /etc/apache2/sites-enabled/node.sugarlabs.org should be tuned:

    <VirtualHost *:80>
            ServerName node.sugarlabs.org
            ProxyPass /  http://<NEWIP>:80/
            ProxyPassReverse /  http://<NEWIP>:80/
    </VirtualHost>

4.  sugar-network-node restarted and Apache reloaded.

--
Aleksey
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: Digital signature
URL: <http://lists.sugarlabs.org/private/systems/attachments/20141016/0eac0b31/attachment.pgp>