Topic: I need a good backup strategy

[Jeff Fox]

A couple of days ago I started a thread asking about using firewire drives as a backup solution.
It was suggested that firewire support is not reliable under SME and that SATA drives would be a better solution.

Here's what I think would work. Please forgive me if I sound crazy. I don't really know anything about SATA drives/controllers/etc yet. My hardware vendor is looking into my suggestion. He's not familiar with SME so that's why I'm asking here.

2 250GB drives mirrored as RAID 1 on a SATA controller. One drive is mounted internally and the other in an external cartridge that can easily be removed.
When it's time to do a backup pull the external drive and replace it with an identical drive. This will break the mirror and provide a perfect backup that can be taken off site. When the new drive is inserted the mirror is automatically rebuilt.
The drives are hot-swappable so there is no downtime.
Will this work with SME out of the box?
If so then what hardware is supported that will work this way?
If SATA is not suitable for this then can someone suggest a SCSI solution, with supported drives/controllers.

I'm open to any other suggestions. What I really need is stable internal storage (RAID) and a way to do quick offsite backups. I don't want to use tapes optical media because they are a pain to handle and restore.

Also, can somebody point me to the RedHat 7.3 HCL? All I can find is the one for 9.0.

[%sig%]

[Boris]

You may want to take a look at accusys 7500 hardware RAID subsystem. It does mirror automatically and is hot swap. It is not SATA, but regular ATA (IDE) and newer versions support big drives. Using this solution you are running into the risky situation. Think what if primary drive dies while still syncing with the mirror? Another inconvenience is restore individual file or folder (accidentally removed).
This solution is only good for quick disaster recovery, but between disasters you may want some conventional backups.

[Jeff Fox]

Then what about a 3rd drive on board in a hot swapable (maybe it automount s and unmounts?) cartridige that will receive backups every night? After the backup that drive could be swapped for offsite storage. The swaps would not happen during the backup phase.

The production RAID and the backup drive might be a few hours out of sync, but no more than that.

The planned server will be a small office machine. It is unlikely that it will be doing disk writes 24/7 so a lag between backups and hot swaps would be acceptable.

[Boris]

I think it would be easier to put USB2 or Firewire drive to one of the workstations and use backup2ws contrib or Tapeware configured to backup to the folder (on the extarnal drive) instead of tape. That external drive can be taken offsite.

Now you have to many options to choose from and can feel for you

[Jeff Fox]

Google turned up some good hits for this product and it sounds like just what I've been looking for.

Review Here:
http://www.motherboards.org/articlesd/hardware-reviews/1041_1.html
Buy it Here:
http://www.aberdeeninc.com/abcatg/MI559.htm
or if you need the Western Digital version Here:
http://www.xopenonline.com/pdetails.asp?itemID=581

[Tom]

Jeff Fox wrote:

> Here's what I think would work.

> 2 250GB drives mirrored as RAID 1 on a SATA controller. ...
> When it's time to do a backup pull the external drive and
> replace it with an identical drive. This will break the mirror
> and provide a perfect backup that can be taken off site.

The technique you are proposing sounds fine in theory, but in practice is a really bad idea. Breaking a RAID is fraught with danger. There's the danger that your extracted mirror may not function at all as a backup when the time comes. There's the possibility that breaking the mirror can down the entire array for no apparent reason and thus the server is offline. (Don't laugh. I've seen it happen.)

There's also the SEVERE performance hit when the array is rebuilding the replacement drive. I've seen servers become almost unusable during a RAID rebuild; and that was with a hardware RAID SCSI subsystem. I'd hate to see a SATA subsystem in the same role.

A better choice has already been suggested and that is to rotate a drive as a standard backup target. Using good backup software and rotated nearline backup drives will give you much more security than the RAID breaking folly.

[Reinhold]

Jeff,

I agree with Tom and Boris... it's a fundamentally bad idea to do backup via raid resync ((
This is what linux command "dd" ... dupe disk was made for !

i) set up your raid1 system. On a fresh SME 6.0 this means "use 2 identical HDs as hda and hdc - then say YES to the raid question" - DONE!
ii) buy a good '3.5 HD - 5" frame' for your box and 2 cases with one HD each that fit. For some $$$ those will even be hotpluggable - but, unless you need to win the uptime-bragging-contest against a windows peer - there is no real harm in powering down "your 9-5 office system" once a week ...
-
iii) familiarize yourself with the "dd" command and run a backup on /dev/hda every friday morning, preferably early, during lunch time or after hours - since "dd" will take up to 30% of cpu cycles during the 1-2h it does run.
iv) either hot-unplug or powering-down ... take framed backup drive out&home ... insert the one from "an earlier week".
v) NOTE: this is NOT a really good backup strategy !!! You will NOT have data from a couple of weeks ago. Intentionally deleted data will be GONE - however you will be 99.999 % operational - YOU do the math.
BUT): this is simple - it is cheap - it will work with little additional time
...and you can still decide whether you want ATA or STAT drives (...or SCSI if you care).
...just my 2c

+1cent: resyncing on a raid1 takes: drivecapacity*writerate ->  1-2h or your 200G+ setup... the lowest 500MHz-software-raid may take 3hrs but you will still have 70% cpu power available ... more than enough for a small office with www/mail and stuff - just don't do multimedia .-)

hth

[Jeff Fox]

Thanks for all of the input guys.

Here is my revised plan.
1. Get a Accusys 7500 hardware RAID subsystem (or a similar product). This should make RAID failure recovery a no brainer. If the client needs an immediate offsite backup then it can be used that way but it will not be used as the primary means of system backup.

2. Get a HD frame and 2 or more cases and drives for backups. I'll let the client decide how many of these are necessary for historical backups and then do 'dd' backups as frequently as the customer desires.

3. The removeable backup drive does not need to be hot-swap. I'll let the client decide if hot-swap is necessary.

Advantages:
By adding cases and drives the backup frequency and historical accuracy can be easily increased.

Disadvantages:
Cost of additional backup media.

Have I missed anything?

[Jeff Fox]

Does 'dd' perform a total disk image copy?
Could the system be booted from one of the backup drives if necessary?

[Boris]

Sounds good except it is not usually necessary to have too many redundant “full disk copy” of the backup.
1.Raid subsystem as a hardware fault tolerance is good.
2.Removable HDs are good as backup media.
3.DD maybe a bit slow and requires exactly the same drive capacity or bigger (may be expensive) wherefore you may consider using backup “data only” to the removable drive.
You may fit few copies of data to the drive during the week (daily backups) and weekly replace the drive for offsite storage. Daily backups can be done with dungog.net’s backup package, tapeware, configured to backup to folder, or with script using tar, gzip, rar or any other compression utility and you can do differential, full backup, critical files only or any other type of backup you want.

[Jeff Fox]

Understood.
I'll explain the cost/benefit to the client of the different options and then choose appropiately.

[Arne]

Dont know if this tread will be of any help (About the dd command)
https://listman.redhat.com/archives/seawolf-list/2002-November/msg00201.html

[Kelvin]

Hi Jeff,

May I ask why you are not considering tape backups at all ?

Using Accusys 7500s for your live system for redundancy is a good thing.

Couple that with a tape drive (depending on your required backup capacity - tape media should be cheaper than keeping multiple hard drives with various backup sets and certainly less prone to damage from minor mishandling) and you should have a good system going.

Kelvin

[Arne]

I made a test based on the red hat instruction just now. Had a 15 GB HD on the test machine and installed one other. runned this command:

dd if=/dev/hda of=/dev/hdc bs=1024

The PC then started to work and it seemed to be copying datas for approx half an hour.

I then installed the backup disk in the possition of the orginal disk while the orginal disk was removed.

The very first part of the bootup workd like nomal with the blue Mitel screen, but then after short time: "Kernel panic: No init found" and so it stopped.

I dont know why the dd command did not make a bootable copy. Anybody who knows. (On the other hand I think it could have worked if I had transfered back the files from the backup to the orginal disk, because it has the boot sector/boot files but this would not help me much if I ha a dik crach.)

Anybody who knows how to make a bootable copy with the dd command ??

[Arne]

Just a rather stupid fault .. Had set the jumpers in different possitions - so now it works !

[Arne]

Discovered one rather not so pleasant side with the dd complete HD copy alternative, I think.

If you are using a blank HD this seems to work ok.

But on the other hand, when I had the orginal HD and the Copy in the PC at the same time when booting, the sme 6.0 got totally confuced over this set of two identical hd so it crahed the file system on the main disk.

But anyhow .. I had just made myself a copy ..

So it seems like you have to use a blank disk for the copy if it should be there during the boot up prosess, or possibly something like a hot plugin divice like USB 2 ??? (Don't know, have never tried.)

[ken]

Optical by itself could be a problem as files become read only but when coupled with RAR zipping it is no longer a problem as the files get the original permissions back when extracted from the archive. Optical provides 4.5 gig DVD RW capabilities now for about $150.00 with zipped files thats probably 9 gig for a DVDRW that costs about $3.00 . You can splurge and by 1 for every day of week plus mid month and end month.

We actually had our server stolen last summer though with password protection we assume they got the box but not the data.

We backed up nightly to even and odd backups on a workstation which created several 650 meg files all zipped in rar format. which we copied to CDR's next day.

The older workstation was not stolen so we only had to do a restore to another computer and it did not take long at all using winrar and it automatically put everything back to the original locations with correct permissions. I was on holiday 3000 miles away at the time so I was very relieved the restore went so smoothly.

BackuptoWS available in the contribs section can automate the copying to the workstation drive.  What more can you ask for?

[Del]

Hi All,
This what I do, I have a HDD caddy on a XP machine + 2 caddy trays both with 80gig HDD's, I set up backup2ws to back up the XP machine (sharing the HDD in the caddy tray) at 10.00pm every night. I then swap the caddies over each morning so i am only ever 1 day behind. I have installed a free program on XP called slawdog tht enables me to set a shutdown time for the XP machine. You just have to work out how long the backup takes (and add a bit) to arrive at a time for the shutdown to take place. Never had any problems yet. I only backup data though as I figure SME is so easy to install I just reinstall it fresh if I need to.
slawdog can be found here: http://www.slawdog.com/
Hope this helps.
Regards,
Del

[%sig%]

[Boris]

>We actually had our server stolen last summer though with password protection we assume they got the box but not the data.

Unless you use encryption for the filesystem, data is easy retrivable by booting this server in the single mode or from the bootable linux CD (like knoppix or any other)
I would assume that if they were interested in data, they've got it. The only hope is they were after the hardware itself.

[Arne]

I'm really not an expert on theese backup things, but from reading these treads and also some other places on the net I have found out a little.

I have found two ways of backing things on command line level that works. It's the dd command, but this is a rather low levlel tool that consumes quide a lot of resourses but that exept for that works ok.

The other way that seems to be working really good is the rsync command. This is really a fantastic thing that first just compares to file set or directories and the makes a backup based on the difference between the file sets. In this way you might have a full backup of the datas, while really only writing in the changes from last backup. When I tested rsync it was running just very gently in the backgrund while it did the job. (Prossessor load dd ca 20 % making a complete digital copy, prosessorload rsync 1-2 % while making a incremental update of a full copy.)

This howto mention only backup to disk but it can actually backut to different media, I think:

http://mirror.contribs.org/smeserver/contribs/dmay/mitel/howto/rsync-backup-howto.html

[M Alden]

We use a strategy very similar to the one suggested by Del, on three different sites, and have had great success.  As he said, it's a no-brainer.  Took a few minutes to figure out backup2ws, but that was about it.  Then, the drive can be taken off site and secured.

IMO,

M Alden

[Roger]

Install RAID 5 storage system.

Use 2 or more USB or Firewire external HDDS and rotate.

Use rsync http://rsync.samba.org/ to sychronise from RAID to USB/Firewire.

The Firewire/USB external HDD solution provides fast+cheap+reliable true "hot swap" backup with 24x7 server up-time and even "raw metal" recovery in event of catastrophic server failure if you get clever with the external HDDs

R.

[Greg Allt]

While this link doesn't take you to an e-smith specific article, the principles discussed are very good.

http://www.faqs.org/docs/Backup-INTRO/

IHMO It is well worth the read

Greg

remove the x in rogersx.com to reply directly to me