[Systems] ALERT JUSTICE DOWN [24 hours without response] (!)

[Bernie Innocenti]

+systems@

I rebooted justice from the management console and it's now responding
to pings.

I couldn't view the screen capture and I had no time to go to the Media
Lab to physically inspect the machine, so I don't understand the root cause.

As reported by Dogi, Justice seems to crash every 1-2 months. I suggest
we try the following steps:

1. upgrade justice to Ubuntu 14.04 (like we did with freedom 1yr ago)

2. if crashes continue, go to the server room and swap the drives with
freedom (which is our hot-swap server and doesn't currently run anything
critical)

3. Ask again the ML to give one of us physical access to the server
room. I work nearby, but I have trouble leaving during office hours on a
personal errand and if anything happens over a week-end we're in trouble.

Sebastian: you should at least get access to the management console.
Ping me on IRC and I'll send you the credentials on a secure channel.


On 06/18/2015 10:40 AM, Sebastian Silva wrote:
> Hello Sugar Oversight Board, Sugar Labs Members,
> 
> Our main production server virtual machine host is down and I can't
> reach it.
> We have several systems that depend on this infrastructure, including
> pootle server which was actively being used by translators of Aymara and
> Awajun native languages.
> 
> I respectfully request that you call on the phone whoever has physical
> access to this machine and we try to bring it back online. I think this
> should be either Bernie Inocenti or Stefan Unterhauser.
> 
> Also, I would like to request for more volunteers from infrasctucure
> team to have virtual terminal access to these machines (not just ssh),
> or to put them in a proper collocation service where we can get some
> support.
> 
> Thanks in advance for your help.
> Sebastian
> 
> On 17/06/15 20:55, Sebastian Silva wrote:
>> Affected services:
>> translate.sugarlabs.org
>> git.sugarlabs.org
>> packages.sugarlabs.org
>>
>>
>>
>> On 17/06/15 20:48, Sebastian Silva wrote:
>>> We can't reach it.
>>>
>>> Anybody with physical access to the machine please respond.
>>>
>>>
>>> Regards,
>>> Sebastian

-- 
Bernie Innocenti
Sugar Labs Infrastructure Team
http://wiki.sugarlabs.org/go/Infrastructure_Team