Topic: [Announce] backup2usb alpha-release

[dmay]

[Announce] backup2usb alpha-release

Will show up in my contrib area shortly:

http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/alpha/smeserver-backup2usb-0.0.1-a2dmay.noarch.rpm

This is an alpha release and is highly experimental. USE WITH CAUTION!! I expect this may not work for everyone. Please install on a test server only.

This contrib is identical in theory to backup2ws except instead of backing up to a workstation, it backs up to an attached USB hard drive. However, it is up to you to configure your own USB hardware and make sure your drive works.

For myself, here is what I had to do on my test server:

Add the USB drive to your server and see whether it gets recognized and on what device:

# tail -100 /var/log/messages

Dec  8 10:07:10 test kernel: Attached scsi disk sda at scsi1, channel 0, id 0, lun 0

# fdisk /dev/sda1

Press p
Press n
Press w

# mkfs.ext3 /dev/sda1

# reboot

# tail -100 /var/log/messages

Dec  8 10:49:07 test fstab-sync[3298]: added mount point /media/usbdisk for /dev/sda1

Test the USB drive

# mount /media/usbdisk
# ll /media/usbdisk
total 20
drwx------  2 root root 16384 Dec  8 10:42 lost+found

Finally once your USB drive is working from the command line, try the Backup to USB panel and report your results.

Darrell

[pickles]

Darrell,

I followed your instructions and everything has worked.  The only hiccup was the example text in the web panel states "/media/usbdisc".  It should read "/media/usbdisk".

I have so far performed a 911 backup with level 3 compression and a selective restore of one ibay.

I have disconnected/reconnected the USB drive and am now performing a full restore.

I'm running this on a pretty standard SME 7beta8.

Justin

[dmay]

Darrell,

I followed your instructions and everything has worked.  The only hiccup was the example text in the web panel states "/media/usbdisc".  It should read "/media/usbdisk".

Actually the panel reads:

Code: [Select]

Enter the mount point of your USB hard drive. Example: [/media/usbdrive]
I agree this should be changed to "/media/usbdisk". Will do so in my next build. Thanks for the testing and feedback :->

Also, I found a neat utility for mounting ext3 file system formated drives under Windows XP:

http://www.fs-driver.org/index.html

Tip. Make sure the drive is formated ext3 only. If it had a previous NTFS partition you have to zero the drive to remove all NTFS remnants, prior to partitioning and formating under SME. As my test server used /dev/sda for my USB drive, I had to do this (carefully entering the correct device!):

dd if=/dev/zero of=/dev/sda bs=512 count=63
dd if=/dev/zero of=/dev/sda1 bs=512 count=63

Then repeat the fdisk, mkfs.ext3 process mentioned above. After that the drive worked great under SME and XP with the above driver installed.

Darrell

[wjhobbs]

Darrell,

That's great!

I note that this targets SME7. Do you know if it will work on 6.0.1?

John

[dmay]

Do you know if it will work on 6.0.1?

Quick answer is Yes. I've confirmed it works on my test 6.0.1 server environment.

The harder part is getting your USB drive set up. It is a more complicated manual process in 6.x. You need to create your own mount point directory and add the correct entries to /etc/fstab. However, if you can get your hardware to work from the command line, the contrib will work.

[pickles]

Darrell,

I've just tried smeserver-backup2usb-0.0.1-a4dmay.noarch.rpm

Unfortunately I didn't get around to testing scheduled backups before upgrading from 0.0.1-a2.

So, after over-coming a small hiccup with the 'Save' button(I forgot to click it), everything is working fine.

BTW - Thanks for the tip on IFS Drive Control, that works great.

Justin

[dmay]

Thanks for the continued feedback.

I'd also like to hear from unique USB hardware users. Particularly hardware such as hot-swap usb drive bays, Iomega REV drives, Storcase DE50, just to name a few. Anyone with this type of hardware, pease test and report.

I'm also thinking of merging backup2usb/backup2ws together into one rpm. Of course the new challenge will be what to call the single rpm.

[HarryM]

Hay Darrell

I have test your backup2usb rpm. Works fine for 6.01 and 6.5.
Under 7beta8 i have a error in my messages log file:
Dec 17 16:29:46 test e-smith-bg: cannot handle file 'libc.so.6' with TLS data

I have no idea somewhat is wrong?
Did you have any idea. It was install on a clean install system.

Greet
Harry

[dmay]

Hay Darrell

I have test your backup2usb rpm. Works fine for 6.01 and 6.5.
Under 7beta8 i have a error in my messages log file:
Dec 17 16:29:46 test e-smith-bg: cannot handle file 'libc.so.6' with TLS data

I have no idea somewhat is wrong?
Did you have any idea. It was install on a clean install system.

Greet
Harry

7.x requires a different rar version:

http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/rar-3.5.1-7dmay.i386.rpm

Make sure you download then install with yum to get the required dependancy rpms:

yum install rar-3.5.1-7dmay.i386.rpm

[smitti]

Hi,

Thanx for this contrib.
I am testing it on 6.5 and works like a charm......

The drive i use a MobileDisk (thats the name on it)
It's a cheap usb 2.0 drive 3.5GB

Thanx again

Peter

[jester]

Hi Darrel,

Installation in SME7b8 went fine. Problem is i've got a second disk symlinked to an ibay as described by: http://no.longer.valid/phpwiki/index.php/extra%20hard%20disk

The files on this extra disk are NOT backed up! Hope this can be solved because i'd really love to use this contrib.


Kind regards,
jester.

[dmay]

Hi Darrel,

Installation in SME7b8 went fine. Problem is i've got a second disk symlinked to an ibay as described by: http://no.longer.valid/phpwiki/index.php/extra%20hard%20disk

The files on this extra disk are NOT backed up! Hope this can be solved because i'd really love to use this contrib.


Kind regards,
jester.

Hi Jester,

The default rar backup does correctly back up symlinked files & directories. If your symlink to a second disk is correct and the disk is mounted, it should back up just fine. Easy to test and view results via the command line. Simply initiate a small rar backup and watch the results:

# rar a /root/test.rar /your_symlink_here

As an example, lets test a standard symlink we backup during the 911 job:

[root@test ~]# ll /home/e-smith/files/primary
lrwxrwxrwx  1 root root 33 Dec 14 15:39 /home/e-smith/files/primary -> /home/e-smith/files/ibays/Primary

[root@test ~]# rar a /root/test.rar /home/e-smith/files/primary

Updating archive /root/test.rar

Adding /home/e-smith/files/primary/files OK
Adding /home/e-smith/files/primary/cgi-bin OK
Adding /home/e-smith/files/primary/html/index.htm OK
Adding /home/e-smith/files/primary/html OK
Adding /home/e-smith/files/primary OK
Done

Here is a test backing up a symlink going to a second mounted drive, in this case an external usb drive:

[root@test ~]# ll /home/e-smith/files/mp3
lrwxrwxrwx  1 root root 16 Dec 22 09:30 /home/e-smith/files/mp3 -> /mnt/usbdisk/mp3

[root@test ~]# rar a /root/test.rar /home/e-smith/files/mp3

Updating archive /root/test.rar

Adding    /home/e-smith/files/mp3/file1.mp3                           OK
Adding    /home/e-smith/files/mp3/file2.mp3                           OK
Adding    /home/e-smith/files/mp3/file3.mp3                           OK
Adding    /home/e-smith/files/mp3/file4.mp3                           OK
Adding    /home/e-smith/files/mp3/file5.mp3                           OK
Adding    /home/e-smith/files/mp3                                     OK
Done

[jester]

Hi Darrel,

Using your commandline method i works like it should, but if i use the server-manger panel it does not work... Below are the relevant lines of my backup2usb log:

Adding    /home/e-smith/files/users/admin/home/.AppleDB/db_errlog         OK
Adding    /home/e-smith/files/users/admin/home/.AppleDB/lock              OK
Adding    /home/e-smith/files/users/admin/home/.AppleDB/cnid2.db          OK
Adding    /home/e-smith/files/users/admin/home/Temporary Items/.AppleDouble/.Parent      OK
Adding    /home/e-smith/files/users/admin/home/.AppleDouble/.Parent       OK
Adding    /home/e-smith/files/users/admin/home/.AppleDesktop/.volinfo      OK
Adding    /home/e-smith/files/users/admin/home/Network Trash Folder/.AppleDouble/.Parent      OK
Adding    /home/e-smith/files/ibays/projects                              OK
Adding    /home/e-smith/files/ibays/Primary/.AppleDB/db_errlog            OK
Adding    /home/e-smith/files/ibays/Primary/.AppleDB/lock                 OK
Adding    /home/e-smith/files/ibays/Primary/.AppleDB/cnid2.db             OK
Adding    /home/e-smith/files/ibays/Primary/html/index.htm                OK
Adding    /home/e-smith/files/server-resources/regedit/win98pwdcache.reg      OK
Adding    /home/e-smith/files/server-resources/regedit/winxplogon.reg      OK

My problem is with the 'projects' ibay and as you can see it sais that it adds: /home/e-smith/files/ibays/projects but none of the 2Gb data in that ibay. The rights on this symlink look ok:

[root@tiger ~]# ll /home/e-smith/files/ibays/
total 4
drwxr-xr-x  6 root root 4096 Dec 17 13:24 Primary
lrwxrwxrwx  1 root root   26 Dec 19 13:27 projects -> /mnt/array2/ibays/projects
[root@tiger ~]#
[root@tiger ~]# mount
/dev/mapper/vg_primary-lv_root on / type ext3 (rw,usrquota,grpquota)
none on /proc type proc (rw)
none on /sys type sysfs (rw)
none on /dev/pts type devpts (rw,gid=5,mode=620)
usbfs on /proc/bus/usb type usbfs (rw)
/dev/md1 on /boot type ext3 (rw)
none on /dev/shm type tmpfs (rw)
/dev/ida/c0d1p1 on /mnt/array2 type ext3 (rw,usrquota,grpquota)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
[root@tiger ~]#

Or im i missing something?

Kind regards,
jester.

[dmay]

My mistake. I forget the rar options used in the action scripts and the reasons I used them. I had to take a look myself to remember.

In the script backup2usb-backup or backup2ws-backup I use the option -ol

# man rar
-ol    Save symbolic links as the link instead of the file.

One example of why this is used instead of following links is due to the primary ibay:

[root@test usbdisk]# ll /home/e-smith/files
lrwxrwxrwx  1 root root   33 Dec 14 15:39 primary -> /home/e-smith/files/ibays/Primary

Both the link and the source are under the directory /home/e-smith. If we have /home/e-smith as a source for backup, and we followed symlinks, we would back up the primary ibay twice:

/home/e-smith/files/primary -> /home/e-smith/files/ibays/Primary
/home/e-smith/files/ibays/Primary

I could remove the option and set an exclusion, however, I believe for the majority of SME Server users the current settings are the better default.

In your case you have added a custom link and mount point. The default options will back up and retain the link. All that is remaining is for you to add the custom mount point to the backup sources to have this custom store backed up. In your case you would append /mnt/array2/ibays/projects or to back up the entire array, /mnt/array2.

Regards,

Darrell

[wjhobbs]

Darrell,

First, the feedback....

I tried it on a fresh install of 6.0.1-01 using an ULTRA Mini Portable Disk case (purchased at TigerDirect) and a 300GB Maxtor drive installed and formatted as ext3. Tried on a test server and the 911 job worked well.

Update: On reboot e2fsck gets called and can't deal with the drive. The boot hangs. The message suggests I get a more recent version of e2fsck. I have to comment out the fstab entry to get boot to complete -- but the fstab entry is needed for the backup job. Fedora Legacy for 7.3 shows the same version as on this machine. Any suggestions for getting around this? The backup job has no problem with the drive.

Next the question(s)...

In setting up the system I had several false starts. For example, since I was just testing, I did a manual mount; but the job failed. It was only after I added the fstab entry that things went well. Then I went to look at the result and the drive appeared to be missing -- I presume you do an unmount at the end.

Also, for doing a 911 restore (I really want to test this before putting it on a production box) what is the restore process? Presumably fresh install, install backup2usb, and then restore -- but should default versions of the other contribs on the box be installed before the restore?

Is there a comprehensive set of documentation for setting up and using backup2ws/backup2usb? This would help a noob like myself to understand what is going on.

If there is a single place that has this kind of guidance, could you please point me to it?

This is a great piece of work!

Thanks, Darrell.

John

[dmay]

Is there a comprehensive set of documentation for setting up and using backup2ws/backup2usb? This would help a noob like myself to understand what is going on.

The panels are very informative on their own. I've documented the 911 restore procedure here.

[jfentum]

Just thought I would be a little more explicit about getting this great contrib working on 6.x system.

Darrell wrote:

"The harder part is getting your USB drive set up. It is a more complicated manual process in 6.x. You need to create your own mount point directory and add the correct entries to /etc/fstab."

 However, if you can get your hardware to work from the command line, the contrib will work."

Here's what I did:

Followed the instructions posted by Darrell up to the part where it says to reboot (but don't reboot!).

#cp /etc/fstab/ /etc/fstab.copy

(This was done as a precaution, just in case.)

#pico /etc/fstab

Add the following line at the end of the file:

/dev/sda1               /media/usbdisk           ext3    defaults        2 1

Control-X and "Y" to save the file.

Now test...

#mount /media/usbdisk

and (if there are no error messages)...

#ll /media/usbdisk

Should give you a return that looks something like this...

drwx------    2 root     root        12288 Jan 26 18:17 lost+found

I was then able to create a 911 job with "/media/usbdisk" as the Destination.

Hope this helps.

[idyll]

Hello.

I have been using Backup to USB on my 6.01 system, backing up to a 250GB Maxtor residing inside a "SATA Vault"  USB enclosure. This is one of those with a removable slider tray.

Link to product blurb for reference

http://www.cooldrives.com/esata-usb-enclosure-removable.html

Your link to the .pdf which describes the recovery process is broken. Is there anything more to it than just clicking upon "restore" and then performing the post event routine?


thanks in advance.

patrick

[idyll]

I attach the emailed output of a manual 911 job. it looks fine, as have the previous automatic backups. The machine is a 6.0.1-01 version.

The problem is there is nothing on the external usb drive when I cd and look at it? Is there some syntax I need to learn to "show" the .rar files when using a shell? I normally use "ls -alg" but obviously this is not working. See below. It says "total 8" but is listing nothing???

Your help is appreciated.

patrick

-- snip --

# ----------------------------------------------------------------
# backup2usb report for Disaster Recovery, job number 911:
# ----------------------------------------------------------------

Please review the dates, time and sizes of the following:

Actual backup destination = [ /media/usbdisk/911 ]

-rw-rw-rw-    1 root     admin    681574400 Feb  2 08:01 /media/usbdisk/911/911.02.02.06.part1.rar
-rw-rw-rw-    1 root     admin    681574400 Feb  2 08:02 /media/usbdisk/911/911.02.02.06.part2.rar
-rw-rw-rw-    1 root     admin    428375217 Feb  2 08:03 /media/usbdisk/911/911.02.02.06.part3.rar

Log file:

-rw-rw-rw-    1 root     admin     2004036 Feb  2 08:03 /var/log/backup2usb/911.log

End of log file reports:

Adding    /root/plus/dhcpmanager                                          OK
Adding    /root/plus/sacolprng                                            OK
Adding    /root/plus/sacoisoqlog                                          OK
Adding    /root/plus/rkhunter                                             OK
Adding    /root/plus/twig                                                 OK
Adding    /root/plus/raid                                                 OK
Adding    /root/raidmonitor                                               OK
Adding    /root/.razor                                                    OK
Adding    /root/.spamassassin                                             OK
Done

--------snip----------

[root@galadriel usbdrive]# ls -alg
total 8
drwxr-xr-x    2 root     admin        4096 Jan 31 17:06 .
drwxr-xr-x    4 root     admin        4096 Jan 31 17:06 ..
[root@galadriel usbdrive]#

[wjhobbs]

Darrell,

This is a feature request.

Backup2usb seems to work really well. One of my reasons for using a usb drive for backup is to be able to take the backup media off-site. However, while the disk is in transit it represents a security risk in itself.

Would it be possible to add a capability to encrypt the backup archive? In that way, if the backup media fell into the wrong hands, it would be very difficult (nothing is impossible) for my data to be compromised.

Thanks for considering the idea.

John

[warren]

Something that others might over look .

I attach the emailed output of a manual 911 job. it looks fine, as have the previous automatic backups. The machine is a 6.0.1-01 version.

The problem is there is nothing on the external usb drive when I cd and look at it? Is there some syntax I need to learn to "show" the .rar files when using a shell? I normally use "ls -alg" but obviously this is not working. See below. It says "total 8" but is listing nothing???


[root@galadriel usbdrive]# ls -alg
total 8
drwxr-xr-x    2 root     admin        4096 Jan 31 17:06 .
drwxr-xr-x    4 root     admin        4096 Jan 31 17:06 ..
[root@galadriel usbdrive]#

The drive is unmounted after the backup , to seethe files you must mount the drive ( mount /mdeia/usbdisk  ) and then  ls -l /media/usbdisk
which will show a 911 directory.

Secondly, when setting up the usb disk in /etc/fstab , one could come across a issue that the usb drive is on a (remote or i suppose local accessable) server, the usb drive gets removed for one or other reason say after a backup and whilst removed, you need to re-boot the server.
with the following settings in /etc/fstab

Add the following line at the end of the file:

/dev/sda1 /media/usbdisk ext3 defaults 2 1

The server will not boot as it cant find the filesystem referenced by the fstab, and you will be dropped out to a shell; ( problematic if you are accessing via Putty ?   )

Rather put the following into fstab :

/dev/sda1 /media/usbdisk ext3 noauto 0 0

The backup2usb contrib mounts the usb at start of the backup job and unmounts it on completion.

I have this set up with a WD Passport 80GB Drive on SME 6.0.1-01.
Darrell, great contrib, thanks.  


Warren