Koozali.org: home of the SME Server

broken system after power fail

Offline jameswilson

  • *
  • 794
  • +0/-0
    • Security Warehouse, trade security equipment
broken system after power fail
« on: October 25, 2013, 02:21:26 PM »
i have a 32 bit sme install running raid 6 with a spare on 7 drives.
We had a power fail this morning which was to long for the ups.
On power up the kernel panics, not syncing
using a rescue cd it says it cant find any linux partitions.

I have no clue what to do to rescue this thing, any help gratefully recieved.

Yours desperatly
James

Offline janet

  • *****
  • 4,812
  • +0/-0
Re: broken system after power fail
« Reply #1 on: October 25, 2013, 02:35:05 PM »
jameswilson

Quote
I have no clue what to do to rescue this thing, any help gratefully recieved.

One approach is to rebuild the system from CD (new install) & restore from backup
You should then configure the UPS/SME server/Nut to gracefully shut your SME server down so that a similar issue does not happen in the future.
Please search before asking, an answer may already exist.
The Search & other links to useful information are at top of Forum.

Offline jameswilson

  • *
  • 794
  • +0/-0
    • Security Warehouse, trade security equipment
Re: broken system after power fail
« Reply #2 on: October 25, 2013, 04:16:38 PM »
i can restore the sme bits but i was running various windows vm's in virtualbox. These were not backed up and i need a file from one of them.

Offline jameswilson

  • *
  • 794
  • +0/-0
    • Security Warehouse, trade security equipment
Re: broken system after power fail
« Reply #3 on: October 25, 2013, 04:29:13 PM »
on researching all of the 7 drives have no md superblock, on sda through to sdg

Offline jameswilson

  • *
  • 794
  • +0/-0
    • Security Warehouse, trade security equipment
Re: broken system after power fail
« Reply #4 on: October 25, 2013, 07:25:52 PM »
Quote
mdadm --assemble --force /dev/md0 /dev/sd[abcdefg]1

found in other places that this command worked but im not brave enough to use it?

Offline stephdl

  • *
  • 1,523
  • +0/-0
    • Linux et Geekeries
Re: broken system after power fail
« Reply #5 on: October 26, 2013, 01:17:58 AM »
using a rescue cd it says it cant find any linux partitions.

do you mean the system-rescue-cd

http://www.sysresccd.org/SystemRescueCd_Homepage

normally it can mount automatically the raid system, you can see https://wiki.contribs.org/Recovering_SME_Server_with_lvm_drives

i suppose that you have lvm activated.
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline jameswilson

  • *
  • 794
  • +0/-0
    • Security Warehouse, trade security equipment
Re: broken system after power fail
« Reply #6 on: October 26, 2013, 04:30:51 PM »
i was using the sme cd and selecting rescue mode.

Its a standard install i didnt select no spare etc at install time.

Offline stephdl

  • *
  • 1,523
  • +0/-0
    • Linux et Geekeries
Re: broken system after power fail
« Reply #7 on: October 26, 2013, 05:02:23 PM »
i was using the sme cd and selecting rescue mode.

Its a standard install i didnt select no spare etc at install time.

therefore you should try this howto https://wiki.contribs.org/Recovering_SME_Server_with_lvm_drives#Method_A_with_SystemRescueCd

after the boot you can check if the raid is activated by performing

cat /proc/mdstat
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline jameswilson

  • *
  • 794
  • +0/-0
    • Security Warehouse, trade security equipment
Re: broken system after power fail
« Reply #8 on: October 26, 2013, 05:15:11 PM »
it cant ensemble the array cos non of the drive have any superblock info.

on boot
cat /proc/mdstat gived

Quote
Personalities : [raid0] [raid1] [raid 10] [raid 6] [raid 5] [raid 4]
unused devices: <none>

Offline stephdl

  • *
  • 1,523
  • +0/-0
    • Linux et Geekeries
Re: broken system after power fail
« Reply #9 on: October 26, 2013, 06:00:03 PM »
well it is not good, really not
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline jameswilson

  • *
  • 794
  • +0/-0
    • Security Warehouse, trade security equipment
Re: broken system after power fail
« Reply #10 on: October 26, 2013, 07:49:47 PM »
its not a superblock problem i was using the wrong command

I have issued the command

mdadm --stop /dev/md2

then

mdadm --assemble --force /dev/md2 /dev/sd[abcdefg]2

mdadm then reports

mdadm: /dev/md2 has been started with 5 drives (out of 6).

cat proc/mdstat then reports the array as active.

But i cant mount it?

Any other ideas

Offline jameswilson

  • *
  • 794
  • +0/-0
    • Security Warehouse, trade security equipment
Re: broken system after power fail
« Reply #11 on: October 26, 2013, 08:01:01 PM »
so the array starts im hopefull i can save this.
when doing
mdadm --examine /dev/sda2 i get different output depending on the drive.
can i make sme do a force on boot or stop it thinking it has a problem to it doesnt need the force?

Offline stephdl

  • *
  • 1,523
  • +0/-0
    • Linux et Geekeries
Re: broken system after power fail
« Reply #12 on: October 26, 2013, 08:36:14 PM »
if the raid starts, now you have to start the lvm as described in the howto
See http://wiki.contribs.org/Koozali_Foundation
irc : Freenode #sme_server #sme-fr

!!! Please write your knowledge to the Wiki !!!

Offline jameswilson

  • *
  • 794
  • +0/-0
    • Security Warehouse, trade security equipment
Re: broken system after power fail
« Reply #13 on: October 26, 2013, 09:29:05 PM »
that has worked. i can now browse the array. How do i make sme boot so i can save the data i need or replace drives etc as needed?

Offline jameswilson

  • *
  • 794
  • +0/-0
    • Security Warehouse, trade security equipment
Re: broken system after power fail
« Reply #14 on: October 26, 2013, 09:33:42 PM »
ie the fact i can now access the data means i should be able to boot sme normally and replace the drive that appears to causing the problem