Koozali.org: home of the SME Server
Obsolete Releases => SME 7.x Contribs => Topic started by: dmay on October 25, 2006, 07:01:42 PM
-
Gordon Rowell has an excellent contrib and build of rdiff-backup available here:
http://www.gormand.com.au/smeserver/WIP/smeserver-rdiff-backup
rdiff-backup-1.1.5-1sme01.i386.rpm
smeserver-rdiff-backup-0.0.1-18.noarch.rpm
I spoke with Gordon about adding a server-manager panel to his contrib in support of backup to local disk. I added a panel and then dived in and made some significant code changes as well. My revised build will be available as soon as mirrors sync here:
http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/testing
smeserver-rdiff-backup-0.0.1-18dmay.noarch.rpm
If you wish to try and comment on my highly experimental changes, please do so at your own risk!
To explain what my build is all about the panel states:
This backup method uses a software package called rdiff-backup to create a copy of your server configuration and user data files, and copies it to an attached local hard drive every night. The initial backup will match your current data size. Subsequent backups will synchronize your data and only copy changes which will make your actual backup time very efficient. In addition incremental changes are stored so you can recover old data overwritten some time ago. The idea is to combine the best features of a mirror and an incremental backup. This method allows you to restore your configuration and user data files easily using command-line tools.
Darrell
-
Hi Darrel,
This looks interesting.
I am for the moment using rsync on a USB drive and a script that excludes some (big) files that don't need to be backed up.
Are you planing on making a panel or so to be able to exclude files, or can it be done in command line?
Per
-
Are you planing on making a panel or so to be able to exclude files, or can it be done in command line?
No plans, however I'm listening...
Darrell
-
No plans, however I'm listening...
Darrell
OK, I really like dar2 because it's easy to backup exactly what you want.
I really like rdiff because you can go back in time and it's incremental, it's also easy to just pick one file if needed.
A combination of the two must be ideal, (in my needs...), but i also see the advance in having dar2 as a replacement for the SME backup system as it is. Great work... I wish i could do stuff like that.
Per
-
Give release 19 a try. Same highly experimental, do so at your own risk warning!
# rpm -qp --changelog smeserver-rdiff-backup-0.0.1-19dmay.noarch.rpm
* Fri Oct 27 2006 Darrell May <dmay@myezserver.com> 0.0.1-19dmay
- added Include/Exclude db entries
- updated panel, templates, do_rdiff-backup to support above
- smeserver-rdiff-backup-0.0.1-19dmay.patch
Darrell
-
Darrell,
what's advantage of rdiff over dar for incremental backups ?
-
Give release 19 a try. Same highly experimental, do so at your own risk warning!
Darrel,
It's not in your download area.
Per
-
Darrell, what's advantage of rdiff over dar for incremental backups ?
rdiff-backup creates a current at time of backup snapshot in standard file system format. Restores are as easy as copying back whatever you want to restore using standard command line tools. Also due to the standard file system format the available restore from disk action works to perform a full restore. Incrementals allow you to go backward from the current snapshot. There are no backup sets to create or administer. You always have a current full backup in standard format and incrementals going back as long as you have set the duration for.
With dar it works in the opposite direction. You first create a full backup and then create incrementals that go forward from your last full backup snapshot. You build up a backup set then save the entire set and repeat the process to build a new backup set. This is a tedious process to administer and support. In addition the process provides a high potential for failure. Any corruption in the full backup file in a backup set may render that set broken.
Having said this rdiff-backup does not work going to a Windows workstation as all ownership and file rights would be lost. This is why I see a fit for both dar and rdiff-backup.
Darrell
-
Darrell;
Thanks again for your work, this contrib and your dar contrib address one of the biggest issues I have with SME, backup and easy restore.
I see a combination of your dar contrib and this rdiff contrib as the all around solution to backup, is that your vision?
-P
-
It's not in your download area.
I uploaded rpms into my contribs ftp area yesterday. They are definitely there however it looks like contribs.org syncing to mirrors isn't working. :(
Darrell
-
I see a combination of your dar contrib and this rdiff contrib as the all around solution to backup, is that your vision?
I expect most people will use one or the other based on which one meets their needs and hardware environment best. However having said this I'm sure someone will have their own personal reasons for using both at the same time. Which is entirely possible. 8)
Darrell
-
With dar it works in the opposite direction. You first create a full backup and then create incrementals that go forward from your last full backup snapshot. ...
In addition the process provides a high potential for failure.
Could you explain what you mean ?
Any corruption in the full backup file in a backup set may render that set broken.
Not so trivial. There are two possibilities for a full backup to be corrupt :
1. failure during backup : dar informs you about it if you want. So there is a simple way - like in e-smith-backup with dar - to do a full backup in place of incremental backup on the next day. And you have at one's disposal preceding good full or/and incremental backups (I speek about e-smith-backup with dar), whereas rdiff seems to leave you without full backup until next day.
2. failure after good backup : it occurs if you have a problem on the backup support. I think there is no real difference between rdiff and dar, although dar has a possibility to use parchive to recover errors and dar-cp (copy a file with io errors) to read corrupt backup file.
-
It's not in your download area.
I uploaded rpms into my contribs ftp area yesterday. They are definitely there however it looks like contribs.org syncing to mirrors isn't working. :(
Darrell
Darrel,
It's still not there... Could i ask you to mail it to me?
per at per-s dot dk
Per
-
It's not in your download area.
I uploaded rpms into my contribs ftp area yesterday. They are definitely there however it looks like contribs.org syncing to mirrors isn't working. :(
Darrell
Darrel,
It's still not there... Could i ask you to mail it to me?
per at per-s dot dk
Per
I did notify staff@contribs.org. However until syncing is fixed I've put copies here:
http://myezserver.com/smeserver/7.x/testing/
Darrell
-
Darrel,
My USB drive does not auto mount, but i can mount i manually with:
# mount -t ext3 /dev/sda1 /media/usbdisk
How ever, when rdiff starts, it somehow unmounts it again and the backup never starts.
It would also be nice to have a "backup now" option in the panel.
Have you maybe thought about backup over internet with rdiff. That is where it gets VERY interesting since it's incremental.
Per
-
My USB drive does not auto mount,...
Sounds like your missing /etc/fstab details for your hardware. You might try manually adding something like this:
/dev/sda1 /media/usbdisk ext3 noauto 0 0
Darrell
-
Ok i sorted out my USB problem, it was actually mounting on /dev/usbdisk1.
Now if i try to make a backup with the default settings i get this:
Executing pre-backup event
Connecting to /media/usbdisk1
[mntent]: line 11 in /etc/fstab is bad
Performing rdiff-backup to /media/usbdisk1/rdiff-backup/9BD66936-9BE9-11DA-AE64-8E5A3C8248E7
Fatal Error: Fatal Error: The file specification
'#------------------------------------------------------------'
cannot match any files in the base directory
'/'
Useful file specifications begin with the base directory or some
pattern (such as '**') which matches the base directory.
And nothing happens on the USB disk.
If i the delete the "/" in front of all the lines i get this:
Executing pre-backup event
Connecting to /media/usbdisk1
[mntent]: line 11 in /etc/fstab is bad
Performing rdiff-backup to /media/usbdisk1/rdiff-backup/9BD66936-9BE9-11DA-AE64-8E5A3C8248E7
Fatal Error: Bad rdiff-backup-data dir on destination side
The rdiff-backup data directory
/media/usbdisk1/rdiff-backup/9BD66936-9BE9-11DA-AE64-8E5A3C8248E7/rdiff-backup-data
exists, but we cannot find a valid current_mirror marker. You can
avoid this message by removing the rdiff-backup-data directory;
however any data in it will be lost.
Probably this error was caused because the first rdiff-backup session
into a new directory failed. If this is the case it is safe to delete
the rdiff-backup-data directory because there is no important
information in it.
And the backup directory on the USB disk actually gets written but nothing more.
Per
-
[mntent]: line 11 in /etc/fstab is bad
You need to fix this reported error.
Fatal Error: Fatal Error: The file specification...
This was fixed in release 20. What are you using? Finally manually mount and delete /media/usbdisk1/rdiff-backup and start over.
Darrell
-
[mntent]: line 11 in /etc/fstab is bad
You need to fix this reported error.
Yes, thanks for pointing that out. I got that fixed now. "expand-template /etc/fstab" does not overwrite/renew fstab fully. Is that a normal behaviour?
This was fixed in release 20.
You didn't tell me there was a release 20 :o
Anyway i got that installed now and the backup is running at the moment. I can't do more testing for now, i have to go away for a couple off weeks, but it looks good.
Per
-
release 21 will appears as soon as mirrors sync. Same highly experimental, do so at your own risk warning!
# rpm -qp --changelog smeserver-rdiff-backup-0.0.1-21dmay.noarch.rpm
* Tue Oct 31 2006 Darrell May <dmay@myezserver.com> 0.0.1-19dmay
- rename rdiff-backup-restore rdiff-backup-dr-restore
- added rdiff-backup-restore
- smeserver-rdiff-backup-0.0.1-21dmay.patch
FYI. Restoring data from your last backup is easy. Just use standard command line tools to mount, access and copy data from your backup drive.
However when restoring incremental data using the rdiff-backup command this easily turns very dangerous if you enter the wrong commands. AFAICT when you restore data it syncs your restore request to current data, essentially removing anything that doesn't match. So restores have to be handled VERY carefully.
In release 21 the rdiff-backup-restore action script recovers data from your backup, either latest (now) or to a defined past time (1D, 2D etc) and places the recovered data on your backup drive. Backups are found under $MOUNT/rdiff-backup and restores are placed under $MOUNT/rdiff-restore. It then launches Midnight Commander which you can use to complete the restore process to your system drive. This will make more sense when you run it and see it in action. The syntax is:
rdiff-backup-restore restore-as-of path-to-restore
Darrell
-
Hi,
It will not exclude the directories i specify. The rdiff-backub.exclude file looks like this:
/home/e-smith/files/ibays/bjarne_bu
/home/e-smith/files/ibays/covers
/home/e-smith/files/ibays/film
/home/e-smith/files/ibays/fop
/home/e-smith/files/ibays/images
/home/e-smith/files/ibays/musik
/
Is it the right format and is the last slash supposed to be on the last line?
I am on release 21.
Per
-
As you know, Gordon and I have been taking a look at rdiff-backup for some time. Gordon has been working with it a lot longer then I and pointed out to me an e-mail he received from the rdiff-backup-users mailing list. (see below) It appears that rdiff-backup has a few known bugs and is not being actively maintained. Due to this we've concluded that rdiff-backup is not worthy of further attention. I have therefore removed the rpms from my contrib area and ask that anyone testing please do the same and remove the rpm from your test server.
I have returned to working on my original contrib which was based on rsync and will release an updated rpm shortly, announced on another thread.
Darrell
-----Original Message-----
From: rdiff-backup-users-bounces On Behalf Of dean gaudet
Sent: Wednesday, November 01, 2006 2:55 PM
To: Keith Edmunds
Cc: rdiff-backup-users@nongnu.org
Subject: Re: [rdiff-backup-users] rdiff-backup being maintained?
On Tue, 31 Oct 2006, Keith Edmunds wrote:
> Ben, are you still reading this? Dean?
i'm here...
rdiff-backup just hasn't been at the top of my list for a while. mostly
because i have a setup which works and there's a zillion other things with
higher priority right now.
i'm still using rdiff-backup 1.0.x cvs branch (there's a patch or two in
there beyond 1.0.4 release). i only use linux -> linux, and only ext3/xfs
filesystems.
most of the stuff i've seen going across the list are 1.1.5 bugs, osx
bugs, windoze boxes, linux samba bugs, etc. just not anything i'm excited
about working on, and i frequently can't even review such patches because
i don't have such a test setup.
there are a few bugs which could affect me -- such as the disk full issues
-- but my backup target disk has a heck of a lot of free space.
sorry :(
but this shouldn't stop anyone else from contributing patches.
-dean
_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki