CERN Accelerating science

20 SVN repositories awaiting recovery after AFS problem

 
Date & time of incident: 
Thursday, July 18, 2013 - 19:00
Post date: 
Friday, July 19, 2013 - 14:16
Incident Description: 

20 SVN repositories are waiting to be recovered from AFS backups after a problem with an AFS partition on Thursday July 18 at 1900. However these is a problem with the recovery from AFS backup which is delaying this.

The names of these repositories are:

acdb
adumitru
ais-hr
amore-modules
atlas-jalmond
atlas-kfiekas
atlas-wilsona
aunderevent
batchtok
camachophdthesis
cmsscquest
DstarN
emcal-beams
emisa2
heppdt
higgs
latechgroup
lemonipmicc
lheinric
mtaGen

Note these repositories will not be accessible until the recovery is complete.

We apologise for the inconvenience.

 

 

Workaround: 

All the SVN conf/authz files have now been restored, but the most recently restored files are from older backups. It was felt more useful to have some usable authz file which allows the SVN repository to be used than to wait for the TSM Backup robot to be fixed.

So some of the authz files will not be up to date versions, but having a usable version allow the librarians to update them now.

Service Element Affected: 
SVN Service
Impact: 
Service is degraded
Status: 
Resolved
Resolution date: 
Tue, Jul 30, 08:45
Expected resolution or Next Update Time: 
Monday, July 22, 2013 - 10:00
Posted by: 
IT-PES
Unit responsible for resolution: 
IT Department

Updates

Posted July 29, 2013 - 6:09pm

There are still problems recalling AFS backups from TSM but in order to have some usable conf/authz to work with SVN, we have been able to restore older authz versions for all except one SVN repository. So we have back put old conf/authz in place for the following repositories:

higgs
emisa2
emcal-beams
DstarN
cmsscquest
batchtok
adumitru

these should be usable now.

The only repository still awaiting a proper conf/authz file is heppdt - we will restore this as soon as it can be done.

We apologise for the long time to recover from this problem.

Posted July 24, 2013 - 9:22am

Some more conf/authz files have been restored and put in place. The list of SVN repositories awaiting recovery of their authz files is:

higgs
heppdt
emisa2
emcal-beams
DstarN
cmsscquest
batchtok
adumitru

We apologise for the inconvenience.

Posted July 20, 2013 - 12:21pm

Some SVN repository AFS volumes have been restored and the authz files recovered.

The list remaining to be done is:

adumitru
batchtok
cmsscquest
DstarN
emcal-beams
emisa2
heppdt
higgs
lemonipmicc
lheinric
mtaGen