Topic: Help please... my Server has crashed and wont boot.

[groutley]

Hi all,
Merry Christmas,  and happy new year to you all...

I am in need of some assistance..   I don't know where to start..

My Server running SME7  latest level with all fixes (up until 3 days ago) crashed.
I think I have a hardware fault which had been causing a Kernel panic every few days.

Unfortunately, after this last Kernel panic, SME will no longer boot..

Says it can't open a filesystem 'main'  and kernel panics.
So it boots to a point of looking at the fstab and mounting filesystems. and then bombs.

How do I even start to fix this ?
Do I boot from the SME7.4 CD ?

I'm afraid I have no clue what to do ...
Please Help...

Currently loaded up a sme8b6 server to get connected to the internet,  but I have no phones / SAIL(asterisk)
let alone all the files and other contribs I had running.

Glen

[BarryO]

Whoa that is a pile of whoa's...

Now is a good time to restore from backup.  You do have a backup, right??  No backup, you could try booting the hard drive to a new machine and see what that gets you. 

Since you have a sme8b6 running I guess that means you have alternative hardware while you test your damaged goods.

After that you could try recovering data via a live cd.  That might be OK for data files, but could be real sketchy for database info.

This isn't anything you don't know already, so maybe the most this post does is let you know someone is listening.

BO

[groutley]

Thanks Barry,
 I appreciate your moral support
and what is this "Backup" thing you mention ?   Does that mean I should run over the Server with my car and Backup over it again to ensure I got it ?? 

I was thinking of connecting the SME7 drive into the SME8 machine and seeing what data I could recover.
My biggest issue is that SAIL3.1/asterisk on SME8 isnt doing what I need.  i.e. SPA3102 Trunk.
In fact as I have posted in the SAIL/asterisk forum, I have gone backwards there majorly too !

So I am really hoping there might be a way I can 'fix' the SME7 filesystem and get it running again.

I'm going to remove a memory DIMM one at a time and try to isolate what would appear a memory issue in it.
but getting it running again is my prime preference right now.

Here's hoping my current whoa's are not setting the standard for 2011 !!

[groutley]

Exact details of the Boot error:

Code: [Select]

Activating logical volumes
   2 logical volume(s) in volume group "main" now active
Creating root device
Mounting root filesystem
VFS: Can't find ext3 filesystem on dev dm-0.
mount: error 22 mounting ext3
mount: error 2 mounting none
Switching to new root
switchroot: mount failed: 22
umount /initrd/dev failed: 2
Kernel panic - not syncing: Attempted to kill init!


[groutley]

I have downloaded and burnt a SME.5.1 install CD with the aim of running 'sme rescue'

however I think I am missing something !  All my searches on here would suggest that sme rescue is simple and follow your nose stuff.
But after I boot the CD and pick my language and keyboard type, I am asked where is the Rescue media !
Well I thought it was the CD I just booted  !  but I pick Local CD and it tells me to select a driver !!

What am I missing?  it booted the CD so it must be able to read the CD  so why do I now have to specify a driver that I have no clue what it is !

Appreciate any help,  I dont know what I am doing wrong and the topic http://wiki.contribs.org/Booting#Rescue_Mode_Help is not exactly helpful..
Maybe I have had too much Christmas Cheer. topped up with New Years Eve lack of sleep !!

Glen

[groutley]

OK, still plodding along here..
  Think I got the sme rescue further..  but it then scans for the linux partition and I get..

Code: [Select]

You don't have any Linux partitions. Press resturn to get a shell. The system will reboot automatically when you exit from the shell.
So I can get a shell / command prompt ...  but what to do now ?

[CharlieBrady]

What am I missing?  it booted the CD so it must be able to read the CD  so why do I now have to specify a driver that I have no clue what it is !

When you boot the CD, it is BIOS code reading the CD. If the SME installer code asks you what driver to use, it usually means that the SME server linux has not detected the CD. Did you use this same CDROM drive to install SME server software?

[CharlieBrady]

Exact details of the Boot error:

Code: [Select]

Activating logical volumes
   2 logical volume(s) in volume group "main" now active
Creating root device
Mounting root filesystem
VFS: Can't find ext3 filesystem on dev dm-0.


I think this indicates that you have LVM set up on your hard drive, but the initrd is set up to expect the ext3 root file system to be on the RAID partition, not within LVM on that RAID partition. I don't know how you could have got into that state.

You may be able to boot this system by appending "root=/dev/main/root" to the grub command line.

[groutley]

When you boot the CD, it is BIOS code reading the CD. If the SME installer code asks you what driver to use, it usually means that the SME server linux has not detected the CD. Did you use this same CDROM drive to install SME server software?

Thanks Charlie, that makes a lot of sense,
It would be nice if details like this were in the sme rescue help section rather than just stating what you would obviously use sme rescue for.
Regarding the CD itself, no I installed this server back in 2008 so would have been a 7.0 or 7.1 CD back then, it's got to 7.5.1 via yum upgrades.
So I downloaded the 7.5.1 CD just for this recovery purpose believing the recovery image would be better matching the system.

[groutley]

I think this indicates that you have LVM set up on your hard drive, but the initrd is set up to expect the ext3 root file system to be on the RAID partition, not within LVM on that RAID partition. I don't know how you could have got into that state.

You may be able to boot this system by appending "root=/dev/main/root" to the grub command line.

I have come to that same belief with all my attempts to mount the filesystem to maybe copy off some data, although with no luck at all.
I will try your grub parameter suggestion, fingers crossed !

[groutley]

Current grub command is set to

Code: [Select]

root (hd0,0)
kernel /vmlinuz-2.6.9.89.31.1.ELsmp ro root=/dev/main/root
initrd /initrd-2.6.9-89.31.1.ELsmp.img
SO I assume this is how it should be and not need "root=/dev/main/root" appened again?

[CharlieBrady]

Current grub command is set to

Code: [Select]

root (hd0,0)
kernel /vmlinuz-2.6.9.89.31.1.ELsmp ro root=/dev/main/root
initrd /initrd-2.6.9-89.31.1.ELsmp.img
SO I assume this is how it should be and not need "root=/dev/main/root" appened again?

Yes, I agree.