Topic: Lightning Strike

[Ed Form]

I got a call to my one production SME server yesterday morning. Turns out that a building about 450 yards away took a roof-breaker lightning wack and the event knocked out the NTL cable service in the whole of Ipswich. My client's server, a Dell 1400 sits, and all of the network sits behind a big healthy UPS so the voltage surge in the area didn't get in that way. Instead it came in down the cable modem and knocked the server over.

On investgating the situation I found that there was no real damage to hardware but the Perc 2S card had dropped the two SCSI disks in the mirror offline so that there was no server available.

I stripped out the server to diagnose all of its bits, and first I hung the Perc card and disks on another machine. When I looked at the system it was reporting two disks present, but offline and after poking at everything I could the only recourse was to force the two disks back online. After I did this, the system booted and ran through the whole of the Linux startup sequence to the point where it starts the journals up again [commit interval 5 secs etc]. It then issued a 'kernel panic: No init found. Try passing init= option to kernel' message and stopped.

I've looked the situation up in the forums and there's only one 4 message exchange on this subject which didn't help at all.

Anyone got any advice?

I'm basically at the point where I'll have to reinstall and restore a backup from tape, but if there is a chance of not having to do this it would be preferable.

Machine is a Dell 1400 twin 850MHz Pentium III with 256MBytes of RAM, a Perc 2S card and two 36Gbyte Quantum 10,000rpm drives in miror.

Ed Form

[guestHH]

Hi,

Did you try the bootlable floppy? or any other bootable floppy with hd tools to check the harddisk?

SME version?

[Ed Form]

guestHH wrote:
>
> Hi,
>
> Did you try the bootlable floppy? or any other bootable
> floppy with hd tools to check the harddisk?
>
> SME version?

Things have come on a bit since the original message. I ran the Perc2 utilities and did a consistency check. with this completed the system runs further through the boot sequence, reaching the following conclusion...

your system appears to have shut down uncleanly
Press Y within 5 secs to force file system integrity check...
Checking root file system
/ contains a file system with errors, check forced.
/:
Inode 376129 has illegal block(s)

/: UNEXPECTED INCONSISTENCY; run fsck manually,
                   (ie. without the -a or -p options)

It then lets me enter the root password for maintenance and after that gives me the prompt...

(Repair filesystem) 1 #

What to do?

Ed Form

[Ed Form]

Sorry, forgot to add this is SME 5.6U6

Ed Form

[guestHH]

1. when the system boots and you see the MITEL screen, press CTRL-X
2. Login with 'MITEL-SME single'
3. run the checktools
4. after that reboot normally

[Terry Brummell]

From the (Repair filesystem) 1 # prompt, type fsck / and answer yes to all the questions.  Atleast this is what I need to do on 5.5 when losing power.

Terry

[Ed Form]

guestHH wrote:
>
> 1. when the system boots and you see the MITEL screen, press
> CTRL-X
> 2. Login with 'MITEL-SME single'
> 3. run the checktools
> 4. after that reboot normally

That comes back with 'No such Image [TAB] gives list' and [TAB] simply shows the SMP and UP alternatives.

The mode I get into when I give the root password after the disk checks fail is the repair mode so the fsck utility ought to work there

What was throwing me was the weird reply from fsck /dev/sda2 which looks as though its the swap partition.

I ran fsck /dev/sda1 and got a clean reply.
Then I ran fsck /dev/sda2 and got some strange stuff about fsck.swap.
Then I ran fsck /dev/sda3 and got yards of stuff about bad blocks and inodes followed by a second pass and dozens of duplicate/bad blocks. Since every single one of them was in teh system area and not in the data ibays I just said yes to everything it suggested.

Bingo, the server works again, all except the video chips on the board and the network card that had the cable modem attached. Junk S3 card from the parts bin and a £5 network card and the system is running perfectly.

Good old SME. I'm going for a beer.

Ed Form

[Ed Form]

Terry Brummell wrote:
>
> From the (Repair filesystem) 1 # prompt, type fsck / and
> answer yes to all the questions.  Atleast this is what I need
> to do on 5.5 when losing power.

Never occurred to me to run it from the root of the filetable!!!! Dummy.

Still going for a beer.

Ed Form

[wewa]

Our building had a power outage lasting few minutes today and then no internet, mail, no ipaddresses (DHCP), thus no printing on the LAN as well.

Reset network switch, Road Runner Cisco cable modem, etc. and then found that the bootup of SME 5.5 had red FAILED appearing on screen 'bad inode'. Didnt have server backup so freaked out a bit.

Couldn't access this discussion board of course, so had to setup a temporary netgear router we have around, thank goodness, then got online and was able to print on the network copier printers.

repair filesystem prompt instructions from Terry worked fine. so many 's to agree to, but turned out ok.

Tried the ctrl-x instructions from RequestedDeletion and that was a dead end for me as was for Ed Form in that it came back with 'no such image.'

Thus i now have a UPS on the server, and installed a VXA tape drive that i didnt get around to doing.

Now i just have to configure it, and make backups, but i'm getting closer to getting it done.

whew. close call.

Thanks guys.

[kreka]

Just a comment, use fsck -y to automatically answer "yes". It is a lot faster.

Regards