Koozali.org: home of the SME Server

Obsolete Releases => SME Server 7.x => Topic started by: Marco Hess on February 01, 2011, 01:08:01 PM

Title: EXT3 corruption after power fail - Help!
Post by: Marco Hess on February 01, 2011, 01:08:01 PM
My system is a RAID6 with 6 1T SATA (barracuda) drives and has been running without a hitch for over a year now and is up to date to the latest 7.5.1 but last monday we had some major power fails in the area. I was not there when it happened so I am not sure if there were only one or multiple power interruptions but it lasted for several hours into the night.

Anyway, the next morning I found the server not responding and on the console I found it failed with a

 Kernel panic - not syncing: Attempted to kill init!

The other error messages on the console are:

 EXT3-fs error (device dm-0): ext_check_descriptors: Block bitmap for group 3840 is not in group (block 267435456)
 EXT3-fs: group descriptors corrupted!

Above that there were

 mdadm: /dev/md2 has been started with 4 drives (out of 6)

and above that a list of 5 drives 0, 1, 3, 4, 5 (/dev/sdc2 missing)

 2 logical volume(s) in volume group "main" now active

So it looks like the LVM is up and running but the problem is in the EXT3 fs.

I booted with sme rescue and tried to follow http://wiki.contribs.org/Recovering_SME_Server_with_lvm_drives. I got it to rebuild the array (without /dev/sdc2) but after rebooting it still fails with the same or very similar error.

There were other suggestions in the Recovering_SME_Server_with_lvm_drives and other forum message but a lot of those commands are not available on the sme rescue disk.

I really don't know anymore how I can recover from this situation bar reformatting but I really don't what to do that at this stage.

I was about to burn a Knoppix CD but have no idea how to use this to fix this EXT3-fs corruption problem.

What can I do?

Marco


Title: Re: EXT3 corruption after power fail - Help!
Post by: nickw on February 01, 2011, 02:37:01 PM
Have a look at what VIP-ire suggested in my thread..

http://forums.contribs.org/index.php/topic,47120.msg232085.html#msg232085