Topic: Story of failed hard drives, linux recovery software, and IMAP message stores

[foldershare]

Hello All.  I have a customer who had a SME 7 server with software RAID.  He managed his own server, and generally relied on ms backup to backup is samba files, and RAID to backup his IMAP mail.  Well, sometime last year the RAID mirroring failed, but only the server knew about it.  Last week, the "working" drive failed.  He gave me a call about the failure, so I had him switch the drives (IE boot of the alternative drive).  The system rebooted, and after a bit, he realized the data was one year old.  Now, he had a good backup of the Windows data in the ibay, but a year of IMAP Email was gone.  The first thing to walk away with is this, RAID is not a good backup medium, so don't rely on it.   I tried all the fsck tricks I knew but was unable to recover any of the data. 

So, ultimately I found a company in India that sold Linux recovery software for $50 US, so I bought it.  Well, the company had horrible tech support if you could get a reply at all, and the product WILL NOT recover data with the default settings, but finally I did get all the data backed up.  For $50, I guess I can't have it all.

Now, to my question/issue.  My IMAP message store recovery plan is the following.  I'll first convert all my users to POP mail (so the current mail they have will store locally on their own PC's).  Then I'll backup the IMAP directory tree structure on the server.  Then I'll copy old (restored) IMAP directory tree structure back in place, and set all the permissions and ownership correctly.  Then, I'll test, and if all goes well, copy the POP mail back to IMAP.

Can someone tell me if my restore plan has holes in it?  I've never had to restore IMAP before, and I think this will work, I'm just hoping I'm on the right track.


Chris Curtis

[dmay]

First things first. RAID was/is never intended for backup. Even a perfectly functional RAID array will not prevent corruption or loss of data. Delete a file and the file is gone. RAID won't assist in getting this file back. You need to implement a backup strategy. Many solutions exist, including SME default backup options plus numerous backup contribs.

Step one. Backup your server.

Darrell

[foldershare]

For those interested, I was able to restore the IMAP message store using the outline below.  The only problem I has was lost date attributes on the recovered Emails.   And as Darrell so redundantly reinforced, RAID is NOT a good solution for backup!

Chris Curtis

Hello All.  I have a customer who had a SME 7 server with software RAID.  He managed his own server, and generally relied on ms backup to backup is samba files, and RAID to backup his IMAP mail.  Well, sometime last year the RAID mirroring failed, but only the server knew about it.  Last week, the "working" drive failed.  He gave me a call about the failure, so I had him switch the drives (IE boot of the alternative drive).  The system rebooted, and after a bit, he realized the data was one year old.  Now, he had a good backup of the Windows data in the ibay, but a year of IMAP Email was gone.  The first thing to walk away with is this, RAID is not a good backup medium, so don't rely on it.   I tried all the fsck tricks I knew but was unable to recover any of the data. 

So, ultimately I found a company in India that sold Linux recovery software for $50 US, so I bought it.  Well, the company had horrible tech support if you could get a reply at all, and the product WILL NOT recover data with the default settings, but finally I did get all the data backed up.  For $50, I guess I can't have it all.

Now, to my question/issue.  My IMAP message store recovery plan is the following.  I'll first convert all my users to POP mail (so the current mail they have will store locally on their own PC's).  Then I'll backup the IMAP directory tree structure on the server.  Then I'll copy old (restored) IMAP directory tree structure back in place, and set all the permissions and ownership correctly.  Then, I'll test, and if all goes well, copy the POP mail back to IMAP.

Can someone tell me if my restore plan has holes in it?  I've never had to restore IMAP before, and I think this will work, I'm just hoping I'm on the right track.


Chris Curtis

[CharlieBrady]

Hello All.  I have a customer who had a SME 7 server with software RAID.  He managed his own server, and generally relied on ms backup to backup is samba files, and RAID to backup his IMAP mail.  Well, sometime last year the RAID mirroring failed, but only the server knew about it.

Between you and your customer, one of you should have been reading admin email, and would have been notified of the hard drive failure.

[I've changed the subject - software RAID did not fail, the hard drives failed.]

[foldershare]

Actually Charlie, the RAID did fail, and no email was sent to the admin account (I checked).  After the RAID failed, the system continued to run for many more months, until the remaining drive went down with bad clusters.  When I had the customer reboot with the "good" drive, it came right up, but with very old data.  I have installed dozens of SME systems, and never had software RAID skip a beat, until now.

Anyway, I reposted so others who may need to reload IMAP messages, would know at least how one other person did it.  Also, hopefully someone else will read about it, and choose to do something other than use RAID as a backup solution. 

In my business I find that the customers we have that either:
A:  Admin their own systems
B:  Shop around for the cheapest tech support or whoever can show up early.

Often have the worst problems.  They nickel and dime their networks to "save money" and end up waisting more money fixing things that are not managed correctly.

Thanks for the post, I'm closing this one down (at least for myself!).

Chris Curtis

Between you and your customer, one of you should have been reading admin email, and would have been notified of the hard drive failure.

[I've changed the subject - software RAID did not fail, the hard drives failed.]

[byte]

Actually Charlie, the RAID did fail, and no email was sent to the admin account (I checked).

If you didn't get an email then you should always report bugs and potential bugs in the bug tracker as SME Server should "just work". On a side note are you absolutely sure you didn't get an email?