CERN Accelerating science

[CVI] Virtual Machine Issues on Central Service and BE-CO

 
Date & time of incident: 
Tuesday, November 20, 2012 - 13:34
Post date: 
Tuesday, November 20, 2012 - 13:37
Incident Description: 

We are having some issues with a specific set of hypervisors deployed in Central Service and BE-CO.

More details soon...

Service Element Affected: 
Server Provisioning Service
Specific Service detail: 
CERN Virtual Infrastructure
Impact: 
Service is degraded
Status: 
Resolved
Resolution date: 
Fri, Nov 23, 13:00
Posted by: 
IT-OIS
Unit responsible for resolution: 
IT Department

Updates

Posted November 23, 2012 - 1:25pm

The hypervisors are working without any problems since Wednesday morning. We continue to monitor them and the tools for catching the issue will be in place if it reappears again.

Posted November 22, 2012 - 8:41am

All the virtual machines are fine. There is no hypervisor affected since yesterday afternoon. We make some changes in the hypervisors in order to detect and catch the underlying issue.

Posted November 21, 2012 - 10:02am

Another server is now down (CERNVS104)

Posted November 20, 2012 - 5:10pm

The hypervisors affected are the following

Central Service (CERNVS101, CERNVS102 and CERNVS106)

BE-CO (CERNVS117, CERNVS118 and CERNVS120)

Spares (CERNVS109, CERNVS112, CERNVS114)

The VMs are back online, but we are doing more investigation on the underlying problem.