CERN Accelerating science

Cleared Incidents 2012

Summary Date & time of incident Service Element Affected Impact Resolved on Post date
Multiple services affected after network switch repair Wed, Dec 19, 18:15 Multiple Services Service is unavailable Wednesday, December 19, 2012 - 20:15 Wed, Dec 19, 19:23
Network device down affecting AFS, Web, CDS and Indico Wed, Dec 19, 17:30 Multiple Services Service is degraded Wednesday, December 19, 2012 - 18:15 Wed, Dec 19, 17:48
EOSCMS restart Wed, Dec 19, 09:45 EOS Service Service is unavailable Wednesday, December 19, 2012 - 10:08 Wed, Dec 19, 10:09
lxplus reinstallation Tue, Dec 18, 16:29 LXPLUS Service Service is degraded Friday, December 21, 2012 - 14:40 Tue, Dec 18, 16:34
76 virtual machines rebooted due to storage problems Tue, Dec 18, 08:30 Multiple Services Some applications linked to services are unavailable Tuesday, December 18, 2012 - 10:30 Tue, Dec 18, 10:50
EOSATLAS unavailable/failover Mon, Dec 17, 10:41 EOS Service Service is unavailable Monday, December 17, 2012 - 12:10 Mon, Dec 17, 10:53
Network incident on RAC8 service at SafeHost Thu, Dec 13, 17:00 Datacenter Network Service Service is unavailable Thursday, December 13, 2012 - 17:05 Thu, Dec 13, 17:17
TWiki service showing intermittent problems Thu, Dec 13, 14:00 Twiki Service Service is degraded Thursday, December 13, 2012 - 15:00 Thu, Dec 13, 17:19
AFS volume creation via afs_admin fails Mon, Dec 10, 18:13 AFS Service Service is degraded Monday, December 10, 2012 - 18:58 Mon, Dec 10, 18:14
Power outage in Safehost affecting CVI Virtual Machines Sat, Dec 8, 09:02 Server Provisioning Service Service is degraded Saturday, December 8, 2012 - 10:41 Sat, Dec 8, 09:05
EOSATLAS unavailable after crash Fri, Dec 7, 22:05 Storage Service for Projects & Experiments Service is unavailable Friday, December 7, 2012 - 22:45 Fri, Dec 7, 22:44
EOSATLAS unavailable Fri, Dec 7, 15:23 EOS Service Service is unavailable Friday, December 7, 2012 - 16:00 Fri, Dec 7, 16:00
Intermittent disconnections on espace.cern.ch and groups.cern.ch Wed, Dec 5, 15:00 Sharepoint Service Service is degraded Wednesday, December 5, 2012 - 16:00 Wed, Dec 5, 16:54
Delays in e-mail delivery Tue, Dec 4, 14:33 E-Mail Service Service is degraded Tuesday, December 4, 2012 - 16:30 Tue, Dec 4, 16:45
Emails not reaching ServiceNow Mon, Dec 3, 14:34 Workflow Development Service Service is degraded Monday, December 3, 2012 - 17:30 Mon, Dec 3, 16:26
Problem with mail notifications from ServiceNow Fri, Nov 30, 07:30 Service Management Service Service is degraded Monday, December 10, 2012 - 20:00 Fri, Nov 30, 10:51
CVI Website unavailability Thu, Nov 29, 08:49 Server Provisioning Service Service is unavailable Thursday, November 29, 2012 - 09:03 Thu, Nov 29, 08:51
Server Disk Error affecting DB On Demand Service Wed, Nov 28, 11:58 Database on Demand Service Some applications linked to services are unavailable Wednesday, November 28, 2012 - 13:05 Wed, Nov 28, 12:04
Problem with apex sso Tue, Nov 27, 17:02 General Purpose Database Service Service is degraded Tuesday, November 27, 2012 - 17:20 Tue, Nov 27, 17:04
Problem with SDB best efforts schedule page Tue, Nov 27, 10:05 SDB Service Service is degraded Thursday, November 29, 2012 - 11:30 Wed, Nov 28, 10:06
Atlas websites unavailable Mon, Nov 26, 06:59 AFS Web Hosting Service Service is degraded Monday, November 26, 2012 - 10:15 Mon, Nov 26, 10:04
afs154 outage Sun, Nov 25, 13:00 AFS Service Service is degraded Sunday, November 25, 2012 - 18:00 Sun, Nov 25, 18:10
AFS daily backup snapshots unavailable for 20,21,22 November Thu, Nov 22, 15:45 Multiple Services Service is degraded Friday, November 23, 2012 - 11:45 Sun, Nov 25, 19:05
CASTOR becoming unresponsive Wed, Nov 21, 11:00 CASTOR Service Service is degraded Wednesday, November 21, 2012 - 12:45 Wed, Nov 21, 12:39
Loss of db services during storage ugprade Tue, Nov 20, 18:00 Multiple Services Service is unavailable Tuesday, November 20, 2012 - 18:20 Tue, Nov 20, 18:04
[CVI] Virtual Machine Issues on Central Service and BE-CO Tue, Nov 20, 13:34 Server Provisioning Service Service is degraded Friday, November 23, 2012 - 13:00 Tue, Nov 20, 13:37
SLS performance degraded Thu, Nov 15, 17:10 Service Information Service Service is degraded Thursday, November 15, 2012 - 19:20 Thu, Nov 15, 17:21
Vidyo service interruption Thu, Nov 15, 11:05 Conference Rooms Service Service is unavailable Thursday, November 15, 2012 - 11:06 Thu, Nov 15, 11:10
Switchboard unreachable from some internal phones Wed, Nov 14, 08:55 Fixed Line Phone Service Service is degraded Wednesday, November 14, 2012 - 09:20 Wed, Nov 14, 09:09
Certificate problem in Account website Tue, Nov 13, 00:00 Multiple Services Service is degraded Friday, November 16, 2012 - 15:00 Fri, Nov 16, 14:40
Atlas ADCR active standby database available with stale data Fri, Nov 9, 20:00 Experiment Database Service Service is degraded Sunday, November 11, 2012 - 19:00 Sun, Nov 11, 15:32
Storage connection problems affecting Virtual Machines hosted in SafeHost Fri, Nov 9, 13:30 Multiple Services Some applications linked to services are unavailable Friday, November 9, 2012 - 16:00 Fri, Nov 9, 14:32
ACRON service degraded Fri, Nov 9, 12:02 Multiple Services Service is degraded Friday, November 9, 2012 - 15:20 Fri, Nov 9, 14:04
EOSATLAS unavailable/failover Tue, Nov 6, 12:02 EOS Service Service is unavailable Tuesday, November 6, 2012 - 12:51 Tue, Nov 6, 12:19
EOSALICE crash Tue, Nov 6, 10:25 EOS Service Service is unavailable Tuesday, November 6, 2012 - 11:50 Tue, Nov 6, 11:29
afs200 unavailable Tue, Nov 6, 10:23 AFS Service Service is degraded Wednesday, November 7, 2012 - 17:09 Tue, Nov 6, 10:25
EDH service degraded Mon, Nov 5, 11:12 IT Database Service Service is degraded Tuesday, November 6, 2012 - 20:00 Mon, Nov 5, 17:34
afs158 problem Sat, Nov 3, 18:00 AFS Service Some applications linked to services are unavailable Saturday, November 3, 2012 - 21:15 Sat, Nov 3, 21:29
Hardware problem on afs104 Mon, Oct 29, 16:49 Storage Service for Projects & Experiments Service is degraded Monday, October 29, 2012 - 17:48 Mon, Oct 29, 16:50
EOSATLAS unavailable/failover Fri, Oct 26, 16:45 Storage Service for Projects & Experiments Service is unavailable Friday, October 26, 2012 - 18:45 Fri, Oct 26, 18:29
Recurrent bookings on Vidyo from Indico Fri, Oct 26, 12:12 Conference Management Application Support Service is degraded Friday, October 26, 2012 - 19:30 Fri, Oct 26, 12:15
Private/test/development Drupal sites unavailable. Fri, Oct 26, 08:45 Web Service Service is unavailable Friday, October 26, 2012 - 11:15 Fri, Oct 26, 09:21
DFS home folders (s, v letters) not available Fri, Oct 26, 02:00 Multiple Services Service is degraded Friday, October 26, 2012 - 06:00 Fri, Oct 26, 07:35
Several databases and application servers unavailable (DBoD, Oracle, etc.) Thu, Oct 25, 18:58 DB & App Platform for AIS, DB & Application Platform Service for Projects & Experiments Service is unavailable Monday, October 29, 2012 - 19:00 Thu, Oct 25, 19:01
afs200 down Thu, Oct 18, 09:54 Storage Service for Projects & Experiments Service is degraded Thursday, October 18, 2012 - 10:05 Thu, Oct 18, 09:55
EOSCMS unavailable Wed, Oct 17, 17:10 Storage Service for Projects & Experiments Service is unavailable Wednesday, October 17, 2012 - 17:45 Wed, Oct 17, 17:45
Indico slow because of AFS service problems Mon, Oct 15, 09:12 Conference Management Application Support, Conference Rooms Service Service is degraded Monday, October 15, 2012 - 09:20 Mon, Oct 15, 09:14
afs42 down Mon, Oct 15, 08:57 Storage Service for Projects & Experiments Service is degraded Monday, October 15, 2012 - 09:27 Mon, Oct 15, 08:59
SLS certificate expired Mon, Oct 15, 00:00 IT Support Services for IT Operations Some applications linked to services are unavailable Monday, October 15, 2012 - 15:21 Mon, Oct 15, 11:58
EOSATLAS unavailable after crash Sat, Oct 13, 16:00 Storage Service for Projects & Experiments Service is unavailable Saturday, October 13, 2012 - 18:15 Sat, Oct 13, 18:01
Trac unavailable Fri, Oct 12, 10:59 Multiple Services Service is degraded Tuesday, October 16, 2012 - 09:00 Fri, Oct 12, 11:01
DB On Demand Service instances rebooting Thu, Oct 11, 15:54 Multiple Services Service is unavailable Wednesday, October 17, 2012 - 10:00 Thu, Oct 11, 16:02
EOSATLAS degraded Wed, Oct 10, 13:00 Storage Service for Projects & Experiments Service is degraded Wednesday, October 10, 2012 - 16:30 Wed, Oct 10, 16:23
EOSALICE unavailable Mon, Oct 8, 07:44 Storage Service for Projects & Experiments Service is unavailable Monday, October 8, 2012 - 09:10 Mon, Oct 8, 08:40
CASTORPUBLIC outage Sat, Oct 6, 12:30 Storage Service for Projects & Experiments All applications linked to service are unavailable Sunday, October 7, 2012 - 11:30 Sun, Oct 7, 14:35
Solved - Various AIS Application unavailable until further notice Fri, Oct 5, 14:48 Multiple Services Service is degraded Saturday, October 6, 2012 - 23:48 Fri, Oct 5, 14:48
DB On Demand Service sites down Fri, Oct 5, 14:40 Multiple Services Some applications linked to services are unavailable Friday, October 5, 2012 - 15:38 Fri, Oct 5, 15:46
Drupal database instance for production sites down Fri, Oct 5, 14:25 DB & Application Platform Service for Projects & Experiments Service is unavailable Friday, October 5, 2012 - 14:40 Fri, Oct 5, 14:52
Problems with some DBoD instances Fri, Oct 5, 11:10 DB & Application Platform Service for Projects & Experiments Service is unavailable Friday, October 5, 2012 - 11:25 Fri, Oct 5, 11:14
OHR database problem causes disturbances in several AIS applications Fri, Oct 5, 10:57 Multiple Services Service is unavailable Friday, October 5, 2012 - 12:36 Fri, Oct 5, 11:03
IT-DB Virtualisation Infrastructure degraded - same problem as last time (October) Fri, Oct 5, 10:20 Multiple Services Service is degraded Monday, November 12, 2012 - 19:00 Fri, Oct 5, 11:38
Drupal database instance for dev/test/personal sites down Fri, Oct 5, 10:20 DB & Application Platform Service for Projects & Experiments Service is unavailable Friday, October 5, 2012 - 11:00 Fri, Oct 5, 10:45
Multiple drupal hosted sites service outages Fri, Oct 5, 10:00 Multiple Services Service is unavailable Friday, October 5, 2012 - 18:00 Fri, Oct 5, 18:03
EOSALICE outage Wed, Oct 3, 22:30 Storage Service for Projects & Experiments Service is unavailable Thursday, October 4, 2012 - 04:30 Thu, Oct 4, 10:02
Router instability in Building 864 Wed, Oct 3, 17:51 Network Infrastructure Service Service is degraded Wednesday, October 3, 2012 - 18:06 Wed, Oct 3, 17:56
EOSALICE outage Wed, Oct 3, 16:00 Storage Service for Projects & Experiments Service is unavailable Wednesday, October 3, 2012 - 17:30 Wed, Oct 3, 16:13
Intermitent storage connection problems affecting Virtual Machines Tue, Sep 25, 18:00 Multiple Services Some applications linked to services are unavailable Wednesday, September 26, 2012 - 15:00 Tue, Sep 25, 19:08
Mobile services issue (GSM/UMTS) Tue, Sep 25, 08:00 Mobile Telephone Service Service is degraded Tuesday, September 25, 2012 - 11:30 Tue, Sep 25, 10:16
Database service issue with ACCMEAS Fri, Sep 21, 07:07 DB & App Platform for Accelerators Service is degraded Friday, September 21, 2012 - 08:07 Fri, Sep 21, 08:09
CASTOR dlf databases affected by storage upgrade Wed, Sep 19, 11:15 DB & Application Platform Service for Projects & Experiments Service is unavailable Wednesday, September 19, 2012 - 12:25 Wed, Sep 19, 11:30
Instability on Oracle HR Wed, Sep 19, 10:45 Multiple Services Service is degraded Wednesday, September 19, 2012 - 11:15 Wed, Sep 19, 10:49
CASTORCMS overload Tue, Sep 18, 09:06 Storage Service for Projects & Experiments Service is degraded Tuesday, September 18, 2012 - 12:36 Tue, Sep 18, 09:08
Technical network router in building 212 restarted unexpectedly Tue, Sep 18, 08:40 Network Infrastructure Service Service is unavailable Tuesday, September 18, 2012 - 08:45 Tue, Sep 18, 10:20
EOSATLAS overload/DoS Mon, Sep 17, 19:00 Storage Service for Projects & Experiments Service is degraded Tuesday, September 18, 2012 - 00:30 Tue, Sep 18, 11:29
A set of hypervisors lost connection to shared storage Fri, Sep 14, 14:00 Multiple Services Some applications linked to services are unavailable Friday, September 14, 2012 - 14:30 Fri, Sep 14, 15:33
Some Mac.Mail have difficulties in connecting to mailbox Fri, Sep 14, 09:00 E-Mail Service Service is degraded Friday, September 14, 2012 - 10:30 Fri, Sep 14, 10:26
16 hypervisors lost connection to share storage Thu, Sep 13, 11:00 Multiple Services Some applications linked to services are unavailable Thursday, September 13, 2012 - 14:00 Thu, Sep 13, 11:54
No connection to network service S513-C-IP650 (Diskserver RC56) Wed, Sep 12, 03:00 Network Infrastructure Service Service is unavailable Wednesday, September 12, 2012 - 16:30 Wed, Sep 12, 09:23
EOSATLAS unavailable Mon, Sep 10, 15:25 Storage Service for Projects & Experiments Service is unavailable Monday, September 10, 2012 - 16:25 Mon, Sep 10, 16:14
EOSCMS unavailable Sat, Sep 8, 14:00 Storage Service for Projects & Experiments Service is degraded Wednesday, September 12, 2012 - 17:30 Sat, Sep 8, 20:41
Primary access to Geant (European Research and Education networks backbone ) down Wed, Sep 5, 06:40 Network Infrastructure Service Service is degraded Wednesday, September 5, 2012 - 09:30 Wed, Sep 5, 08:09
TWiki server swap full incident Thu, Aug 30, 14:45 Twiki Service Some applications linked to services are unavailable Thursday, August 30, 2012 - 14:55 Thu, Aug 30, 16:30
EOSATLAS degraded Wed, Aug 29, 17:00 Storage Service for Projects & Experiments Service is degraded Thursday, August 30, 2012 - 12:30 Thu, Aug 30, 11:01
Vidyo connection problems for IN2P3 users Wed, Aug 29, 09:00 Remote Collaborative Service Service is degraded Friday, August 31, 2012 - 17:00 Thu, Aug 30, 20:18
EOSATLAS unavailable Mon, Aug 27, 17:07 Storage Service for Projects & Experiments Service is unavailable Monday, August 27, 2012 - 17:35 Mon, Aug 27, 17:50
MyTeamWork audio-conference system problems Wed, Aug 22, 11:03 Conference Management Application Support Service is degraded Wednesday, August 22, 2012 - 16:00 Wed, Aug 22, 11:15
The GPN router serving building 31 and 600 crashed Tue, Aug 21, 13:30 Network Infrastructure Service Service is unavailable Tuesday, August 21, 2012 - 13:35 Tue, Aug 21, 13:45
Intermittent connection problems to ITCORE DB Tue, Aug 21, 12:30 General Purpose DB & App Platform Some applications linked to services are unavailable Tuesday, August 21, 2012 - 14:50 Wed, Aug 22, 09:49
SLS is grey Tue, Aug 21, 12:27 IT Support Services for IT Operations Service is unavailable Tuesday, August 21, 2012 - 12:31 Tue, Aug 21, 12:38
Major incident with SVN repositories Tue, Aug 21, 09:06 Multiple Services Service is degraded Tuesday, August 28, 2012 - 09:05 Tue, Aug 21, 11:10
EOSCMS unavailable Mon, Aug 13, 10:16 Storage Service for Projects & Experiments Service is unavailable Monday, August 13, 2012 - 10:43 Mon, Aug 13, 11:27
Problem with AIS Login Mon, Aug 13, 08:50 Multiple Services Service is degraded Monday, August 13, 2012 - 09:50 Mon, Aug 13, 08:52
Authentication SOAP Service random issues Sun, Aug 12, 18:00 Multiple Services Service is degraded Monday, August 13, 2012 - 09:50 Tue, Aug 14, 08:15
Vector Fields Opera licenses unavailable Fri, Aug 10, 08:30 Mechanical Design Software Service is unavailable Friday, August 10, 2012 - 10:45 Fri, Aug 10, 08:53
CMSR, CMSONR_ADG, LCGR database partial unavailibity due to storage issues Thu, Aug 9, 12:40 DB & Application Platform Service for Projects & Experiments Service is degraded Thursday, August 9, 2012 - 14:15 Thu, Aug 9, 14:10
afs100 unavailable affecting CMS Scratch volumes Wed, Aug 8, 09:23 Not specified Service is degraded Wednesday, August 8, 2012 - 09:50 Wed, Aug 8, 09:26
EOSCMS unavailable Tue, Aug 7, 13:50 Storage Service for Projects & Experiments Service is unavailable Tuesday, August 7, 2012 - 16:30 Tue, Aug 7, 14:21
Degraded network performance in building 27 Mon, Aug 6, 14:15 Network Infrastructure Service Service is degraded Monday, August 6, 2012 - 15:45 Mon, Aug 6, 17:01
Problems with E-Groups web interface Wed, Aug 1, 07:20 Identity and Resource Management Application Support Service is degraded Wednesday, August 1, 2012 - 08:15 Wed, Aug 1, 07:40
EOSATLAS namespace readonly Tue, Jul 31, 09:10 Storage Service for Projects & Experiments Service is in read-only mode Tuesday, July 31, 2012 - 11:05 Tue, Jul 31, 09:18
Drupal Infrastructure unavailable Thu, Jul 26, 11:02 Web Service Service is unavailable Thursday, July 26, 2012 - 12:15 Thu, Jul 26, 13:05
Partial power failure at SafeHost Thu, Jul 26, 11:00 Multiple Services Service is unavailable Thursday, July 26, 2012 - 12:00 Thu, Jul 26, 12:41
PABX de Prévessin Tue, Jul 24, 09:20 Fixed Line Telephone Service Some applications linked to services are unavailable Tuesday, July 24, 2012 - 11:30 Tue, Jul 24, 10:00
SSO Incident following update Tue, Jul 24, 08:30 Identity and Resource Management Application Support Some applications linked to services are unavailable Tuesday, July 24, 2012 - 08:50 Tue, Jul 24, 09:25
LCG service IP290 unavailable Wed, Jul 18, 17:40 Not specified Service is unavailable Wednesday, July 18, 2012 - 18:00 Wed, Jul 18, 18:08
E-mails to a domain desy.de were delayed due to problems at Desy Mon, Jul 16, 10:00 E-Mail Service Service is degraded Monday, July 16, 2012 - 14:00 Mon, Jul 16, 16:08
Short disruption of one of the GPN DNS servers Tue, Jul 10, 11:41 Not specified Service is degraded Tuesday, July 10, 2012 - 11:44 Tue, Jul 10, 13:44
Indico - Search feature unavailable Mon, Jul 9, 16:32 Conference Management Application Support Service is degraded Monday, July 9, 2012 - 16:58 Mon, Jul 9, 16:34
CASTOR storage intervention Mon, Jul 9, 11:53 DB & Application Platform Service for Projects & Experiments Service is degraded Monday, July 9, 2012 - 12:00 Mon, Jul 9, 11:57
afs206 unavailable (AFS work spaces) Mon, Jul 9, 11:33 Storage Service for Projects & Experiments Service is degraded Monday, July 9, 2012 - 12:49 Mon, Jul 9, 11:34
Instabilities affecting updates of Service Now tickets Mon, Jul 9, 10:15 Process Application Support Service is degraded Monday, July 9, 2012 - 17:30 Mon, Jul 9, 10:41
afs158 unreachable Sun, Jul 8, 18:00 Storage Service for Projects & Experiments Service is degraded Sunday, July 8, 2012 - 20:43 Sun, Jul 8, 19:35
No network in building 60 for 20 minutes Tue, Jul 3, 08:42 Network Infrastructure Service Service is unavailable Tuesday, July 3, 2012 - 09:00 Tue, Jul 3, 10:24
afs102 unavailable (scratch, compass) Mon, Jul 2, 16:48 Storage Service for Projects & Experiments Service is degraded Monday, July 2, 2012 - 17:58 Mon, Jul 2, 16:50
Many services in the CERN computer centre currently unavailable (open the link for updates) Fri, Jun 29, 08:20 Under Investigation Some applications linked to services are unavailable Friday, June 29, 2012 - 16:30 Fri, Jun 29, 08:23
Flexlm licence servers down Fri, Jun 29, 08:02 Multiple Services Service is unavailable Friday, June 29, 2012 - 11:00 Fri, Jun 29, 10:06
Phonebook web interface degraded Thu, Jun 28, 10:00 CERN Telephone Directory Application Support Service is degraded Thursday, June 28, 2012 - 10:35 Thu, Jun 28, 10:12
CMSONR DB overload due to faulty patch Tue, Jun 26, 15:30 DB & Application Platform Service for Projects & Experiments Service is degraded Tuesday, June 26, 2012 - 17:00 Tue, Jun 26, 16:03
Drupal Infrastructure unavailable Tue, Jun 26, 10:30 Multiple Services Service is unavailable Tuesday, June 26, 2012 - 11:05 Tue, Jun 26, 11:16
Problems to connect to E-Groups (CRA) Web Service using Perl client Tue, Jun 26, 08:30 Identity and Resource Management Application Support Service is unavailable Tuesday, June 26, 2012 - 17:00 Tue, Jun 26, 13:26
SVN affected by egroups changes Mon, Jun 25, 20:00 Multiple Services Service is degraded Tuesday, June 26, 2012 - 18:00 Tue, Jun 26, 13:34
ACCCON (acc_settings_db) unavailable Sun, Jun 24, 05:58 DB & App Platform for Accelerators Service is degraded Sunday, June 24, 2012 - 06:20 Sun, Jun 24, 06:33
Impossible to activate accounts because of EDH password service unavailability Fri, Jun 22, 14:30 Multiple Services Service is degraded Friday, June 22, 2012 - 17:30 Fri, Jun 22, 16:05
Foundation, Phonebook and E-groups not available Fri, Jun 22, 14:10 Multiple Services Service is unavailable Friday, June 22, 2012 - 14:35 Fri, Jun 22, 14:33
Wrong leave credits transferred in EDH Tue, Jun 19, 17:10 Leave Request and Personal Schedule Application Support Service is degraded Wednesday, June 20, 2012 - 00:42 Tue, Jun 19, 17:11
CASTORATLAS diskservers unreachable Mon, Jun 18, 17:15 Storage Service for Projects & Experiments Service is unavailable Monday, June 18, 2012 - 17:30 Mon, Jun 18, 17:32
Slow job submission response time on LXBATCH Thu, Jun 14, 09:00 Batch Service Service is degraded Friday, June 15, 2012 - 17:00 Thu, Jun 14, 10:02
sft.its.cern.ch jira instance not available Tue, Jun 12, 12:26 Under Investigation Service is degraded Tuesday, June 12, 2012 - 13:02 Tue, Jun 12, 12:29
CVICLR04FC cluster failure Tue, Jun 12, 09:00 Multiple Services Service is degraded Tuesday, June 12, 2012 - 10:08 Tue, Jun 12, 10:09
SLS display for some databases is incorrect due to a problem with Oraweb Tue, Jun 12, 07:00 DB & App Platform for Accelerators Service is unavailable Tuesday, June 12, 2012 - 07:30 Tue, Jun 12, 08:07
Drupal Web Sites Unavailable Mon, Jun 11, 15:50 Web Service Service is unavailable Monday, June 11, 2012 - 16:10 Mon, Jun 11, 16:16
EOSATLAS namespace problem Fri, Jun 8, 18:25 Storage Service for Projects & Experiments Service is unavailable Friday, June 8, 2012 - 20:50 Fri, Jun 8, 19:35
Possible network problem in bldg 513 Thu, Jun 7, 04:50 Multiple Services Service is degraded Thursday, June 7, 2012 - 07:20 Thu, Jun 7, 09:47
Problems on afs45 (LHCb project volumes) Tue, Jun 5, 16:53 Storage Service for Projects & Experiments Service is degraded Tuesday, June 5, 2012 - 18:23 Tue, Jun 5, 16:55
CASTOR nameserver issue Tue, Jun 5, 12:45 Storage Service for Projects & Experiments Service is degraded Tuesday, June 5, 2012 - 13:45 Tue, Jun 5, 13:40
Slow response from lxbatch to user commands Tue, Jun 5, 09:00 Batch Service Service is degraded Friday, June 8, 2012 - 00:00 Tue, Jun 5, 17:39
Account Management service running slowly Thu, May 31, 23:00 Multiple Services Service is degraded Friday, June 1, 2012 - 13:00 Fri, Jun 1, 12:07
Indico Search Disabled Wed, May 30, 15:49 Conference Rooms Service Service is degraded Wednesday, May 30, 2012 - 17:10 Wed, May 30, 15:53
Access problems to SVNWEB & Trac Fri, May 25, 16:00 Multiple Services Service is degraded Wednesday, May 30, 2012 - 14:16 Fri, May 25, 16:17
Problem with SVNWEB cluster affecting SVNWEB and Trac Tue, May 22, 22:00 Multiple Services Service is degraded Wednesday, May 23, 2012 - 12:00 Wed, May 23, 08:03
Printer problem on 513-HP-A0 and A0GLOSSY last night. Solved now. Thu, May 17, 09:31 Printing Service Some applications linked to services are unavailable Thursday, May 17, 2012 - 10:00 Thu, May 17, 09:46
EDMS unavailable Wed, May 16, 22:28 DB & App Platform for EDMS Service is unavailable Wednesday, May 16, 2012 - 23:39 Wed, May 16, 22:30
Short interruption LCG services in the vault Wed, May 16, 15:20 Not specified Service is unavailable Wednesday, May 16, 2012 - 15:25 Wed, May 16, 16:39
Problem with EDH this morning Wed, May 16, 07:00 Not specified Service is unavailable Wednesday, May 16, 2012 - 08:35 Wed, May 16, 08:54
Post-mortem update for incident (For SSB contributors: do NOT use "force" until further notice) Wed, May 16, 07:00 Not specified Service is degraded Monday, May 21, 2012 - 08:00 Wed, May 16, 09:05
EDH unavailable Mon, May 14, 16:53 Multiple Services Service is unavailable Monday, May 14, 2012 - 17:30 Mon, May 14, 16:56
Indico Search Disabled Mon, May 14, 13:40 Conference Management Application Support Some applications linked to services are unavailable Monday, May 14, 2012 - 21:43 Mon, May 14, 13:52
Delay in publication of CRLs Fri, May 11, 17:45 Multiple Services Service is degraded Wednesday, May 16, 2012 - 11:45 Wed, May 16, 12:06
SVNWEB Problems related to Trac Fri, May 11, 14:44 Multiple Services Service is degraded Monday, May 14, 2012 - 08:30 Fri, May 11, 14:47
Problem with IT SSB after Drupal upgrade yesterday - now solved Fri, May 11, 06:00 Service Information Service Service is unavailable Friday, May 11, 2012 - 10:15 Fri, May 11, 10:20
Warning displayed after publications in the IT SSB Thu, May 10, 10:00 Service Information Service Service is degraded Thursday, May 10, 2012 - 18:00 Thu, May 10, 16:26
AFS scratch server overload Thu, May 10, 09:30 Storage Service for Projects & Experiments Service is degraded Friday, May 11, 2012 - 11:50 Thu, May 10, 11:52
EDH unavailable Thu, May 10, 07:35 DB & App Platform for AIS Service is unavailable Thursday, May 10, 2012 - 10:00 Thu, May 10, 07:37
Ais (edh, foundation) node loss Wed, May 9, 11:35 DB & App Platform for AIS Service is degraded Wednesday, May 9, 2012 - 11:50 Wed, May 9, 12:25
EOSATLAS namespace problem Mon, May 7, 11:50 Storage Service for Projects & Experiments Service is degraded Thursday, May 10, 2012 - 01:00 Mon, May 7, 12:48
General purpose database (PDBR) partially unavailable Fri, May 4, 14:35 DB & Application Platform Service for Projects & Experiments Service is degraded Friday, May 4, 2012 - 15:15 Fri, May 4, 15:53
Batch nodes failing, kill processes and resend notification emails to users Fri, Apr 27, 16:21 Batch Service Service is degraded Friday, April 27, 2012 - 19:00 Fri, Apr 27, 16:29
Traffic loss on the CERN-IN2P3 LHCOPN link Thu, Apr 26, 11:00 Multiple Services Service is degraded Friday, April 27, 2012 - 16:00 Thu, Apr 26, 13:55
Hardware issue in the Geant Geneva router Wed, Apr 25, 09:34 Network Infrastructure Service Service is degraded Friday, April 27, 2012 - 18:52 Wed, Apr 25, 09:55
Problems with Drupal and AIS applications Tue, Apr 24, 17:15 DB & App Platform for AIS, General Purpose DB & App Platform, Web Service Service is unavailable Tuesday, April 24, 2012 - 17:35 Wed, Apr 4, 16:47
Lxbatch still unstable (20 Apr 9:50) - now resolved Fri, Apr 20, 09:58 Batch Service Service is degraded Friday, April 20, 2012 - 12:30 Fri, Apr 20, 10:02
Lxbatch down on 19 Apr at 20:15 - now resolved Thu, Apr 19, 20:15 Batch Service Service is unavailable Friday, April 20, 2012 - 00:30 Thu, Apr 19, 21:28
CCC gpn router crash Wed, Apr 18, 13:50 Network Infrastructure Service Service is unavailable Wednesday, April 18, 2012 - 14:15 Wed, Apr 18, 14:17
Performance degraded on firewalled traffic to/from Internet Tue, Apr 17, 10:24 Firewall Configuration Service Service is degraded Tuesday, April 17, 2012 - 13:43 Tue, Apr 17, 11:34
Connectivity issues with Geant Thu, Apr 12, 18:38 Multiple Services Service is degraded Friday, April 13, 2012 - 19:10 Thu, Apr 12, 18:50
Sunrise network disruption (Geneva area) Thu, Apr 12, 09:00 Mobile Telephone Service Service is degraded Thursday, April 12, 2012 - 12:30 Thu, Apr 12, 10:28
sindes and swrep interruption due to server crash Wed, Apr 11, 09:45 IT Support Services for IT Operations Service is unavailable Wednesday, April 11, 2012 - 09:52 Wed, Apr 11, 09:54
CASTOR central nameserver errors Wed, Apr 4, 09:00 Storage Service for Projects & Experiments Service is degraded Wednesday, April 4, 2012 - 11:15 Thu, Apr 5, 09:32
cerndb1/edmsdb/accdb and EDMS services unavailability Wed, Apr 4, 06:53 DB & App Platform for AIS, DB & App Platform for EDMS Some applications linked to services are unavailable Wednesday, April 4, 2012 - 07:40 Wed, Apr 4, 07:10
Oracle Test and Dev database and application services not accessible Mon, Apr 2, 12:02 DB & App Platform for AIS Service is unavailable Wednesday, April 4, 2012 - 17:30 Mon, Apr 2, 12:06
Fixed Telephony Service disruption Mon, Apr 2, 08:32 Fixed Line Telephone Service Service is degraded Monday, April 2, 2012 - 14:45 Mon, Apr 2, 15:58
Drupal Mysql instance rebooted. Fri, Mar 30, 02:45 General Purpose DB & App Platform Service is degraded Friday, March 30, 2012 - 03:10 Fri, Mar 30, 04:03
SLS update problems Wed, Mar 28, 21:00 IT Support Services for IT Operations Service is degraded Thursday, March 29, 2012 - 15:50 Thu, Mar 29, 14:40
Service-now is down Mon, Mar 26, 11:59 Process Application Support Service is degraded Monday, March 26, 2012 - 12:56 Mon, Mar 26, 12:00
CVI Self Service Interruption Fri, Mar 23, 18:05 Server Hosting Service Service is degraded Friday, March 23, 2012 - 19:00 Fri, Mar 23, 18:09
Batch resources partially unavailable Fri, Mar 23, 08:00 Batch Service Service is degraded Monday, March 26, 2012 - 11:00 Mon, Mar 26, 11:44
QUALIAC PRODUCTION web service: Possible problems Thu, Mar 22, 08:57 DB & App Platform for AIS Service is degraded Friday, March 23, 2012 - 17:00 Thu, Mar 22, 09:05
Problem with HRT reports (both online and scheduled) Thu, Mar 22, 07:00 Personnel Reporting Application Support Service is degraded Thursday, March 22, 2012 - 14:30 Thu, Mar 22, 12:10
Database service degradation for COMPASS (COMPR) Fri, Mar 16, 12:30 DB & Application Platform Service for Projects & Experiments Service is degraded Friday, March 16, 2012 - 17:15 Fri, Mar 16, 14:14
CDBSQL synchronisation problems Wed, Mar 14, 11:00 IT Support Services for IT Operations Service is degraded Friday, March 16, 2012 - 16:00 Fri, Mar 16, 12:12
e-groups intermittent proxy errors Tue, Mar 13, 12:06 DB & App Platform for AIS Service is degraded Tuesday, March 13, 2012 - 12:20 Tue, Mar 13, 12:09
SINDES lookup to egroups broken Mon, Mar 12, 16:05 IT Support Services for IT Operations Service is degraded Tuesday, March 13, 2012 - 10:30 Mon, Mar 12, 16:07
EOSCMS unavailable Mon, Mar 12, 01:10 Storage Service for Projects & Experiments Service is unavailable Monday, March 12, 2012 - 06:50 Mon, Mar 12, 11:15
Problems found in CASTORALICE instance Wed, Mar 7, 16:01 Storage Service for Projects & Experiments Service is degraded Wednesday, March 7, 2012 - 16:20 Wed, Mar 7, 16:03
Qualiac Database unavaliable this morning Wed, Mar 7, 03:27 DB & App Platform for AIS Service is unavailable Wednesday, March 7, 2012 - 07:50 Wed, Mar 7, 10:06
Primary Geant access down (connectivity to European Research and Education Networks) Tue, Mar 6, 16:04 Network Infrastructure Service Service is degraded Tuesday, March 6, 2012 - 17:30 Tue, Mar 6, 16:12
Problem with Vidyo Booking through Indico Mon, Mar 5, 16:40 Remote Collaborative Service Service is degraded Monday, March 5, 2012 - 18:00 Mon, Mar 5, 16:43
EOSCMS files unavailable Thu, Mar 1, 10:45 Storage Service for Projects & Experiments Service is degraded Thursday, March 1, 2012 - 13:50 Thu, Mar 1, 13:21
Compass database hanging Tue, Feb 28, 04:15 Multiple Services Service is unavailable Tuesday, February 28, 2012 - 05:40 Tue, Feb 28, 04:54
CMSR database hanging Tue, Feb 28, 04:15 Multiple Services Service is unavailable Tuesday, February 28, 2012 - 05:40 Tue, Feb 28, 04:52
ATLAS atlasgrp SVN Repository corrupted Fri, Feb 17, 16:59 GRID Development Service Service is degraded Saturday, February 18, 2012 - 18:00 Tue, Feb 21, 08:29
SLS not responding Thu, Feb 16, 18:07 IT Support Services for IT Operations Service is unavailable Thursday, February 16, 2012 - 23:00 Thu, Feb 16, 18:08
CERN to IN2P3 network affecting VOMS, FTS, SAM, MyProxy, .... Thu, Feb 16, 14:35 Multiple Services Service is unavailable Thursday, February 16, 2012 - 17:00 Thu, Feb 16, 14:40
Sudden increase of HTAR traffic is overloading access to Geant (R&E network in Europe) Wed, Feb 15, 17:28 Multiple Services Service is degraded Thursday, February 16, 2012 - 07:00 Wed, Feb 15, 17:33
EOSCMS unavailable for 45min Wed, Feb 15, 12:20 Storage Service for Projects & Experiments Service is unavailable Wednesday, February 15, 2012 - 13:05 Wed, Feb 15, 13:11
Certain SVN Projects not available Wed, Feb 15, 10:00 Multiple Services Service is degraded Wednesday, February 15, 2012 - 10:30 Wed, Feb 15, 15:21
DFS home folder m,t intervention Tue, Feb 14, 09:30 Multiple Services Service is degraded Tuesday, February 14, 2012 - 10:00 Tue, Feb 14, 09:43
AFS154 unavailable Mon, Feb 13, 21:49 Storage Service for Projects & Experiments Service is degraded Monday, February 13, 2012 - 22:10 Mon, Feb 13, 22:42
EOSCMS in readonly mode Mon, Feb 13, 14:05 Storage Service for Projects & Experiments Service is in read-only mode Monday, February 13, 2012 - 14:35 Mon, Feb 13, 14:38
Batch service unavailable Mon, Feb 13, 06:30 Batch Service Service is unavailable Monday, February 13, 2012 - 09:00 Mon, Feb 13, 17:04
CERNDB1/ACCDB/EDMSDB perfomance degraded Fri, Feb 10, 14:30 DB & App Platform for EDMS Service is degraded Friday, February 10, 2012 - 18:40 Fri, Feb 10, 15:24
Network connectivity problems, including WIFI, linked to site power cuts Wed, Feb 8, 14:00 Multiple Services Service is degraded Thursday, February 9, 2012 - 11:00 Wed, Feb 8, 14:08
Intermittent Restarts on EOSCMS Mon, Feb 6, 06:00 Storage Service for Projects & Experiments Service is degraded Monday, February 6, 2012 - 14:55 Mon, Feb 6, 15:14
DFS services - web page "Manage Your DFS Folders and Quotas" unavailable Thu, Feb 2, 11:12 Web Service Service is unavailable Thursday, February 2, 2012 - 16:00 Thu, Feb 2, 11:14
LCGR database experiencing problems Wed, Feb 1, 10:00 DB & Application Platform Service for Projects & Experiments Service is degraded Wednesday, February 1, 2012 - 11:50 Wed, Feb 1, 10:29
General network connectivity problem affecting services including Service-now software Fri, Jan 27, 08:55 Service Management Service Service is unavailable Friday, January 27, 2012 - 09:20 Fri, Jan 27, 09:00
No internet connectivity Fri, Jan 27, 08:55 Multiple Services Service is degraded Friday, January 27, 2012 - 09:20 Fri, Jan 27, 09:04
Drupal Web sites not available Thu, Jan 26, 15:51 Web Service Service is unavailable Thursday, January 26, 2012 - 16:18 Thu, Jan 26, 16:30
Windows patching server DESKTOPWSUS.cern.ch is unavailable Thu, Jan 26, 14:24 Windows Desktop Service Service is degraded Friday, January 27, 2012 - 17:30 Thu, Jan 26, 14:34
Possible issue with SLC6 kernel upgrade Thu, Jan 26, 09:17 Linux Desktop Service Service is degraded Friday, January 27, 2012 - 08:00 Thu, Jan 26, 09:21
EOSATLAS instability, some files temporarily not available Wed, Jan 25, 10:40 Storage Service for Projects & Experiments Service is degraded Wednesday, January 25, 2012 - 12:00 Wed, Jan 25, 12:14
Messages to physics.utoronto.ca were delayed Tue, Jan 24, 17:00 E-Mail Service Service is degraded Wednesday, January 25, 2012 - 16:45 Wed, Jan 25, 18:00
Web Services site unavailable Fri, Jan 20, 07:59 Web Service Some applications linked to services are unavailable Friday, January 20, 2012 - 09:30 Fri, Jan 20, 08:04
Service 513-C-IP601 intermittent connectivity Thu, Jan 19, 18:37 Not specified Service is degraded Friday, January 20, 2012 - 16:53 Thu, Jan 19, 18:41
Service 513-C-IP601 flapping Thu, Jan 19, 16:40 Not specified Service is degraded Thursday, January 19, 2012 - 18:14 Thu, Jan 19, 16:43
Service 513-C-IP160 intermittent connectivity Thu, Jan 19, 13:05 Not specified Service is degraded Thursday, January 19, 2012 - 14:23 Thu, Jan 19, 14:09
GPN services at Safehost Thu, Jan 19, 09:00 Multiple Services Service is degraded Thursday, January 19, 2012 - 16:29 Thu, Jan 19, 10:05
Perturbation des services GSM et téléphonie fixe (numérotation courte 16xxxx et 7xxxx) Wed, Jan 18, 11:30 Multiple Services Service is degraded Wednesday, January 18, 2012 - 15:30 Wed, Jan 18, 12:50
Computer centre network problem 02:30 am today Wed, Jan 18, 02:33 Multiple Services Some applications linked to services are unavailable Wednesday, January 18, 2012 - 02:33 Wed, Jan 18, 02:45
Short interruption for a few LCG services (vault) on Friday 13/01 Fri, Jan 13, 17:18 Not specified Service is degraded Friday, January 13, 2012 - 17:33 Fri, Jan 13, 17:44
Ansys license server crashed Fri, Jan 13, 09:12 Mechanical Design Software Service is degraded Friday, January 13, 2012 - 09:30 Fri, Jan 13, 09:14
Disk problem with the CDB server Thu, Jan 12, 10:52 IT Support Services for IT Operations Service is unavailable Thursday, January 12, 2012 - 11:15 Thu, Jan 12, 10:53
Connect problems to CASTOR (after NameServer database upgrade) Wed, Jan 11, 12:10 Storage Service for Projects & Experiments Service is unavailable Wednesday, January 11, 2012 - 15:00 Wed, Jan 11, 12:27
Ansys license server problem Mon, Jan 9, 17:10 Mechanical Design Software Service is unavailable Monday, January 9, 2012 - 21:10 Mon, Jan 9, 17:27