CERN Accelerating science

Major incident with SVN repositories

 
Date & time of incident: 
Tuesday, August 21, 2012 - 09:06
Incident Description: 

A number of SVN repositories were accidently deleted on Tuesday morning (21/08).  Data recovery is in progress, meanwhile users of SVN, SVNweb and Trac are affected. Affected SVN repostories will generate an error.

Thanks to a lot of work from our AFS colleagues to restore a huge amount of data, nearly all SVN repositories are now restored to the state they were on Monday evening (20 Aug 2012). These are now available.

Please note: on some SVN repositories, the Trac cache may not be in sync with the SVN revisions. This will be fixed on a step by step basis, but the operation takes time. 

August 24, 2012 - 4:28pm

We have restored from AFS all the repositories which were known to be damaged by the deletions on Tuesday August 21. The backups were from Monday evening August 20, starting at 1800. See the corresponding "Update" at the bottom for the repositories which are still known to have problems and which will be followed up.

 

Workaround: 

Instead of using Trac to browse revision history, please use Websvn:

https://svnweb.cern.ch/websvn/websvn.php

Service Element Affected: 
Multiple Services
Specific Service detail: 
SVN
Any other affected service(s): 
SVNweb and Trac
Impact: 
Service is degraded
Status: 
Resolved
Resolution date: 
Tue, Aug 28, 09:05
Expected resolution or Next Update Time: 
Thursday, August 23, 2012 - 18:00
Posted by: 
IT-PES
Unit responsible for resolution: 
IT Department