Koozali.org: home of the SME Server
Obsolete Releases => SME 7.x Contribs => Topic started by: jpl on July 25, 2007, 03:56:38 PM
-
What is e-smith-backup with dar ?
e-smith-backup with dar is an upgrade of current e-smith-backup sme core rpm which provides panels for daily full or incremental backup on LAN workstation (via nfs, smbfs or cifs) or local usb disk, and full or selective restore with use of dar program.
What provides e-smith-backup with dar ?
e-smith-backup with dar completes core e-smith-backup with four panels :
- configure daily full or incremental backup to smbfs/cifs/nfs share on a LAN workstation or on local usb disk
- verify backup on LAN workstation or local usb disk
- restore full backup from LAN workstation or local usb disk
- selective restore of files or directories from LAN workstation or local usb disk
New backup options are provided with additional perl functions included in backup panel script, a new perl backup script /etc/e-smith/events/actions/workstation-backup-dar, and a new backupwk record in configuration db. All are written with the same logic as present backup scripts.
What are specific problems of SME Server LAN backups managed by e-smith with dar ?
With experience of more than four years backuping sme servers with dar on LAN workstations (in production environments) the most important of encountered problems is that, when the backup session is started, you cannot rely upon a perfect working of the LAN or the distant workstation.
Inevitably, one day the LAN is too busy and/or the connection to the distant share hacks. After the failure sme may keep zombies backup or mount processes, and with some contribs you can be left with incomplete or buggy backup until hopped next good daily backup. These problems are problems of real world with LAN, that backup scripts must take into account.
Another common problem with backup through a LAN is that your lan disponibility for backup flow is not 24/24, but more probably something like "only from 20h to 6h, except on sunday where nobody works on the LAN..." So in case of large volume of data to backup, beeing able to backup all the datas during the week-end and doing only incremental backup on the week days really makes sense.
All these problems are considered with e-smith-backup with dar.
Scripts verify the absence of old share mounts, and alert admin about such situation. Test is made that the distant backup share is available and writable. Of course with daily backup a maximum backup session time of only 24 h is available, but if this limit occurs e-smith-backup with dar cleanly stops the backup session (maybe all is ok and the backup has not enough time to complete), the lan share is unmounted, and daily process can restart anyway. If you have enabled incremental backup and the last backup (full or incremental) was not completed because of a timeout, the backup can then safely restart and complete in an incremental way.
The admin can also set a timeout limit which keeps the session inside a specified number of hours, excluding traffic of LAN backup sessions when users work on the LAN.
Admin user can also set backup configuration so that this time limit occurs only on incremental backup sets. Then if you have a lot of data and your full backup time exceeds time of LAN inactivity, you can always backup all your data each end of week and preserve LAN bandwith for users. If you program a full backup on saturday night, you get 24 hours to do full backup and even if the full backup time exceeds 24 hours, the first incremental backup on sunday night will backup remaining data. You begin your week with full data saved, and incremental backup session time is limited to 6 or 8 hours during week days.
Full backup sessions can also be allowed on every day, or limited to a specific day of week.
Are other new functions provided ?
Yes. Aside use of session timeout, the first is incremental backup.
This means that you can backup and restore data for the period of time you want : one day, three days, one week, one month, 100 days... and restore your system at any state it was during this period of time. This probably has no utility to do full restore of the system as it was one month ago, but restoring a file lost by a user two or three weeks ago can be useful. And restoring a safe system more than one day old can be needed.
The second function is keeping more than one set of backup (a set is full backup data and all datas of next daily incremental backups) with automatic rotation. e.g. you can do only nightly full backups but keep three sets of backup for security reason (as being able to restore the system as it was 72h ago).
The third function is selective restore of any backuped file or directory, exactly as it was for any of the backuped days in your sets. Not only you can restore a lost file at his last state, but also say : make restore of the most recent version of the file before this given date... Selective restore is not an easy thing to manage by hand, and providing this in a simple way with panels is useful. Dar permits to manage selective restore and e-smith-backup with dar panels tries to keep this function as simple as possible to use.
Where to find e-smith-backup with dar ?
Presently beta version of e-smith with dar can be uploaded at http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-16eos1.el4.sme.noarch.rpm
SRPMS available at http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-16eos1.el4.sme.src.rpm
Dar rpm is available at http://beta.free-eos.org/sme/RPMS/dar-2.3.2-1.i386.rpm or mirrored at http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/dar-2.3.2-1.i386.rpm
I read about problems about smbfs or cifs support with sme, and nfs packages are not on my sme...
Before sme 7.2, smb and kernel versions of sme have bugs in smb mount and cifs support.
For smb mount bug, e-smith-backup with dar includes necessary work-around.
For cifs support, corrected 2.6.9-55 kernel is packaged with sme 7.2.
For nfs support, e-smith-backup doesn't need install of any nfs package on the sme server (not even portmap).
Preventing automatic upgrades from sme yum updates...
You can prevent sme yum upgrade system from upgrading e-smith-backup with dar with new e-smith-backup core version.
Just enter as root :
# echo e-smith-backup > /etc/e-smith/db/yum_repositories/defaults/smeupdates/Exclude
# /etc/e-smith/events/actions/initialize-default-databases
# /sbin/e-smith/expand-template /etc/yum.conf
Manual yum upgrade is always available.
Comments are welcomed.
/edit/
sme 7.2 version
Jul 26 : corrected rpms release and link
-
I was reluctant to update my server because prior breakages for this great contrib ... Now I can update all servers.
Great Job jpl ... I love your contrib.
Regards,
Tib
-
Seems there where a problem while generating rpm.
Updated version available in a few moments...
-
Corrected rpms are now available for download (release 16eos1).
Some scripts were no more included in first 16eos release, so only upgrades from e-smith-backup with dar older releases where OK, not upgrades from core e-smith-backup.
Should be now corrected. Apologies...
-
I think you need to update your dar links... I found one at http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/dar-2.3.2-1.i386.rpm
Also, are there any docs pages or pages more clearly explaining the options on the workstation backup page. I'm not sure I understand exactly what the # of sets and dailys in a set mean.
I *think* that by setting it to 4 sets with 7 dailys per set with Friday set as the only full day should give me 4 sets of 7 files with one large file per set bing the Friday full and the rest being daily incrementals.
-
Thanks for the dar link.
In fact, Darrel originally made a copy of my rpm, and I have not seen that he provides now an updated version :)
You are right about sets.
Sorry for the doc, it has yet to be made on the wiki. But panels are almost (almost only I know) self explainatory.
-
Thanks for the dar link.
In fact, Darrel originally made a copy of my rpm, and I have not seen that he provides now an updated version :)
Sorry again, it's not a new rpm, it's the one I provided to Darrel.
Just my links that are wrong, as you say. I correct them.
-
Ok... so I got this:
==================================
DAILY BACKUP TO WORKSTATION REPORT
==================================
Backup started at Thu Jul 26 20:15:02 2007
Backup of mysql databases has been done.
Mounting backup shared directory netshare/backup
*** No backup allowed or error during backup ***
Error while mounting netshare:backup :
mount error: could not find target server. TCP name netshare/backup not found rc = 134526947
No ip address specified and hostname not found
My settings are:
Backup is enabled.
Backup is made on LAN workstation netshare via cifs
Destination backup share folder is backup
Login is backup
Password is ********
Number of rotating backup sets is 4
Number of daily backups contained in each set is 7
Compression level (0-7) of backup is 4
Daily backup occurs at 20:15
Each daily backup session is cleanly timed out after 10h except full backups which are cleanly timed out after 24 hours
Full backup session (new backup sets) is allowed only on Friday
This SME Server is 10.11.12.14 on my LAN and "netshare" is 10.11.12.3 on the same segment. I changed the workstation name to 10.11.12.3 for tonights attempt at a backup. Can anyone suggest anything else or possibly a way to test it now without actually starting a backup since I've got users hitting it?
Thanks,
E
-
Not at home for eigth days, and don't have acess to server.
You can look at /etc/e-smith/events/actions/workstation-backup-dar script, find the command which connect your cifs target, and try to debug it directly as a shell command.
I think it is :
/bin/mount -t cifs $smbhost:$smbshare $mntdir -o user=$login,pass=$password
in your case :
/bin/mount -t cifs netshare:backup /mnt/smb -o user=backup,pass=backup
You can also look at your /var/log/messages log for some more information about your cifs mount.
-
..I must be blind today...
I have made a fresh install of the SME7.2 and installed the "dar-2.3.2-1.i386.rpm" contrib.
But is my understanding correct that for this contrib there is no user-interface displayed in the web-panel??
-
But is my understanding correct that for this contrib there is no user-interface displayed in the web-panel??
It adds more options to the standard backup panel.
-
Elliot,
a stupid question: which contribs have to installed in order to see
the see the additional these options??? For me here nothing has changed, strange ?? (& I used signal-event post-upgrade // signal-event reboot)
thanx
gerd
-
Elliot,
a stupid question: which contribs have to installed in order to see
the see the additional these options??? For me here nothing has changed, strange ?? (& I used signal-event post-upgrade // signal-event reboot)
thanx
gerd
-
Follow the directions at the start of this post. That's it.
Then go to the "Backup or restore" tab on the SME Server Console. That's it. The new options are in the drop down list on the bottom.
-
...yep, indeed - I was blind....it works perfect.
thanx
gerd
-
...finally, it doesn't work "that perfect" for me:
- the usb drive is ext3 formatted.
- the usb drive is automatically recognized by the SME7.2 server (media/usbdisk1)
- when I launch the workstation-backup_setconf I select local usb drive
- then I start the backup, but the files are not written on the local usb drive as expected - they are written into the directory /mnt/smb/; and that means in fact the files are written on the internal harddisk array. And this is basically not what I intended to do....
Can anyone enlighten me about this behaviour? As I am a newbie in Linux
my guess is that I have made somewhere a mistake - but no idea where to start with my investigations??
Just to check I installed the contrib "smeserver-dar2-0.0.1-3dmay.noarch.rpm, for the local usb drive I entered media/usbdisk1 and the backup worked perfect.
thanx
gerd
-
The first goal of e-smith-backup with dar is to backup the server on an Lan smffs, cifs or nfs workstation share. A bug is always possible, specialy on usb backup which is not frequently used.
then I start the backup, but the files are not written on the local usb drive as expected - they are written into the directory /mnt/smb/
e-smith-backup with dar always do backup to /mnt/smb.
The question is : is the usb drive mounted on /mnt/smb, and if not why ?
Can you make some tests about it
-
JPL,
If i would want to remove/undo this contrib what would need to be done and would it revert back to the default SME Server installation so that upgrading could cause no problems with future releases of SME Server?
jester.
-
Nothing special. e-smith-backup with dar does not modify configuration needed for core e-smith-backup.
If you upgrade with new core e-smith-backup version, all should be OK... Just verify no cron job left in /etc/cron.d.backupwk
If you want to restore actual core e-smith-backup, rpm -e e-smith-backup ,reinstall core e-smith-backup rpm, and post-upgrade event should do the trick.
-
@jpl
The question is : is the usb drive mounted on /mnt/smb, and if not why ?
When I connect the USB drive to the SME7.2 server, the USB drived is automatically mounted under /media/usbdisk. I made the test with three
different SME7.2 servers - with the same result. :( :(
-
Hello !
I have update my SME to 7.2 and installed the rpm for DAR backup below but after restart, it stills doesnt appear in server-manager.
Any suggests ?
When I try to reinstall, it says "already installed"
-
syscom
Answered earlier in this thread
Q. But is my understanding correct that for this contrib there is no user-interface displayed in the web-panel??
A. It adds more options to the standard backup panel.
-
syscom
Answered earlier in this thread
Q. But is my understanding correct that for this contrib there is no user-interface displayed in the web-panel??
A. It adds more options to the standard backup panel.
Sorry but it DOES NOT add more options to the standard backup panel.
:P
-
gerd, syscom & anyone else
...I installed the contrib smeserver-dar2-0.0.1-3dmay.noarch.rpm...
To avoid confusion, that's a totally different contrib than the e-smith-backup that this thread is talking about. It's a standalone backup system that adds a seperate server manager panel.
This thread refers to e-smith-backup (with dar) rpm which is a replacement rpm for the standard e-smith-backup rpm, and it adds extra settings to the standard server manager backup panel.
Note that as this contrib uses dar to perform backups, then you need to install both
e-smith-backup-1.14.0-16eos1.el4.sme.noarch.rpm
and
dar-2.3.2-1.i386.rpm
Note also that doing a normal upgrade using the software installer panel or yum update, will most likely overwrite the special version of e-smith-backup that needs to be installed in order to have the extra functionality.
There are commands given earlier in this thread that you need to enter to prevent yum from updating the e-smith-backup rpm automatically (which would be undesirable as you will lose the added functionality). ie
echo e-smith-backup > /etc/e-smith/db/yum_repositories/defaults/smeupdates/Exclude
/etc/e-smith/events/actions/initialize-default-databases
/sbin/e-smith/expand-template /etc/yum.conf
The author of the e-smith-backup (with dar) rpm, will I assume continue to update the rpm and release a new version as and when any changes in the base sme server rpm occur. In that case users of e-smith-backup (with dar) rpm will need to manually update that particular rpm.
Note the rpm version (without dar support) that is installed on a standard sme7.2 server is
e-smith-backup-1.14.0-16.el4.sme.noarch.rpm
If you are having problems then check the rpm version actually installed
rpm -qa e-smith-backup
rpm -qa dar
To install e-smith-backup with dar
download the correct version rpms to a clean folder on your sme server and do
rpm -Uvh dar*.rpm
rpm -Uvh e-smith-backup*.rpm
Open server manager & select the Backup or restore panel
Extra choices should have been added to the pull down menu for Select an action
ie
Configure workstation backup
Verify workstation backup
Restore from workstation
Selective file restore from workstation
Select the required backup type and configure it etc.
-
jpl
I added this to the Wiki
http://wiki.contribs.org/Backup_with_dar
It probably needs some further clarification re configuring backup sets and the number of daily backups in the set & associated practical usage concepts.
-
If you are having problems then check the rpm version actually installed
rpm -qa e-smith-backup
rpm -qa dar
After checking I got this reply:
e-smith-backup-1.14.0-16.el4.sme and dar-2.3.2-1
And I did follow the wiki instructions.
The problem seems to be the order of installation. Yum install of dar reverts e-smith-backup to original specs :?
I only had to run ยป rpm -Uvh e-smith-backup*.rpm and it was up and running.
Thanks Ray.
-
Karsa
Perhaps yum localinstall does not overwrite the existing rpm of the same name.
I'll check that. For now I changed the install order in the Wiki, but that may not be sufficient.
I appreciate you said you followed the wiki, but please confirm that you did actually download the rpms with wget and do a yum localinstall. Thanks
-
I made a directory (backupwdar), downloaded the files (wget) and made a localinstall with yum.
Followed with signal-event post-upgrade // signal-event reboot and after reboot...
echo e-smith-backup > /etc/e-smith/db/yum_repositories/defaults/smeupdates/Exclude
/etc/e-smith/events/actions/initialize-default-databases
expand-template /etc/yum.conf
I'm now trying to setup a lan backup...
-
Karsa
This is required, I fixed the Wiki
wget http://mirror.contribs.org/smeserver/contribs/dmay/smeserver/7.x/dar-2.3.2-1.i386.rpm
wget http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-16eos1.el4.sme.noarch.rpm
yum localinstall dar*.rpm
yum localinstall e-smith-backup*.rpm
Thanks
-
Still stuck at "Configure workstation backup" :-(
I'm using SME Server 7.2 and the clients are using XP Pro SP2.
Any help would be welcome... I'm lost looking at the connection types (cifs, nfs,...yes total nood). :?
-
I'm using SME Server 7.2 and the clients are using XP Pro SP2.
Any help would be welcome... I'm lost looking at the connection types (cifs, nfs,...yes total nood). :?
smbfs or cifs should be OK.
-
jpl
e-smith-backup with dar is an upgrade of current e-smith-backup sme core rpm which provides panels for daily full or incremental backup on LAN workstation (via nfs, smbfs or cifs) or local usb disk, and full or selective restore with use of dar program.
Good job, an impressive add on.
What is your longer term goal with this rpm ie is it still in beta stage, are you going to continue to support it with each sme update/release etc ?
Have you or do you plan to submit it to the sme developers for possible inclusion in the base code ?
Regards
-
jpl
What is your longer term goal with this rpm ie is it still in beta stage, are you going to continue to support it with each sme update/release etc ?
Have you or do you plan to submit it to the sme developers for possible inclusion in the base code ?
I have done the first release of this package about one year ago, following creation of the very lively bug 1880 on the bugtrack. Code is stable for about nine months. I had in mind inclusion in the base code and tried to develop dar integration in respect of core e-smith-backup code. It seems to me that I have asked about the question of submiting it for inclusion (and how to submit it) but had no response.
I think the question of inclusion is still very topical, but I don't know the correct way to ask about it.
I know that releasing a modified core package is not satisfactory... but doing a contrib from my work before getting sme team point of view is not a better choice and I have not given up inclusion.
But if things don't go further (or if core team doesn't want an inclusion) I think the viable solution is making a contrib permitting use and update of unmodified core e-smith-backup. But it leads to add another specific panel to existing backup panel, and I don't like this idea. I think at least core team should modify backup panel to provide (and specify) the way to plug other backup solutions, but it is not so trivial.
Opinions and suggestions are welcomed...
-
smbfs or cifs should be OK.
Thanks, but something isn't working, I don't get any backup file...
Probably my shared directory because I get "ERR_NO_HOST_DIRteste.teste.local" when trying to verify the backup (that doesn't exist but I should be able to log to workstation.
My settings:
Backup is enabled.
Backup is made on LAN workstation 10.1.1.7 via cifs
Destination backup share folder is backup
Login is admin
Password is *******
Number of rotating backup sets is 2
Number of daily backups contained in each set is 7
Compression level (0-7) of backup is 5
Daily backup occurs at 22:30
Each daily backup session is cleanly timed out after 10h
Full backup session (new backup sets) is allowed only on Saturdays
My SME server IP's 10.1.1.6. Any ideas?
-
"teste.teste.local" seems a strange UNC name. Are you sure it is resolved by your SME WINS ?
You should try to use the IP of your backup workstation in place of his workstation name.
Before, you could try to mount your cifs share directly from the shell as a debugging way.
Verify you have no mount left on /mnt/smb and enter :
# /bin/mount -t cifs smbhost/backup /mnt/smb -o username=admin,pass=*****
Use first IP of your workstation as smbhost, and if succeeds, then try UNC host name (it is possible than only ip is ok).
Same tests could be done with smbfs and command :
# /bin/mount -t smbfs //smbhost/backup /mnt/smb -o username=admin,password=*****,fmask=777,ip=ip_of_smbhost
Don't forget to umount /mnt/smb when mount succeeds.
-
Tried smbfs command and it didn't work, with cifs I managed to made the mount. But I still fail to get any backup file and I get the same error message.
"teste.teste.local" is the SME server name. :-(
-
Are you able to write and create directories on /mnt/smb after the cifs mount ?
If not perhaps admin has not appropriate rights on the share.
-
No problem with that, I made directories both in the client and the server :???:
I must be missing some step... probably a obvious one :roll:
-
JPL,
I'm working with an Iomega REV drive (USB) that uses cartridges, that need to get mounted before a backup and unmounted afterwards preferably even ejected after an backup if possible. Is there a possibility for this?
-
I must be missing some step... probably a obvious one :roll:
Maybe...
I think the best way should be to trace and debug the perl script /etc/e-smith/events/actions/workstation-backup-dar whick does the effective backup. You could get some useful information about what is wrong.
-
JPL,
I'm working with an Iomega REV drive (USB) that uses cartridges, that need to get mounted before a backup and unmounted afterwards preferably even ejected after an backup if possible. Is there a possibility for this?
Two events, launched by daily SME backup, are available to customize such things : pre-backup and post-backup events. I think you could add actions scripts to these events for your specific needs.
Look for example at /etc/e-smith/events/actions/S50rewind-tape, lauched by pre-backup event, for such a script which rewind tape if the backup is a tape backup.
-
Hi jpl
You wrote to another poster:
The first goal of e-smith-backup with dar is to backup the server on an Lan smffs, cifs or nfs workstation share. A bug is always possible, specialy on usb backup which is not frequently used.
e-smith-backup with dar always do backup to /mnt/smb.
The question is : is the usb drive mounted on /mnt/smb, and if not why ?
Can you make some tests about it
I appreciate your contribution as it is exactly what I am looking for. However I also find that it is not quite ready for use for backups to USB. My understanding of SME Server is that when you connect a USB drive, the HAL daemon will automatically create the directory usbdisk in /media, but will not actually mount the drive in that directory (you have to do that manually). The directory is removed when the USB drive is disconnected. When configuring e-smith-backup with dar to backup to USB (using "Configure workstation backup") the software seems to get confused if the USB drive is connected but not mounted: it sees the usbdisk directory and allows you to select it, but when the backup is run it creates the backup in "/media/usbdisk " where a trailing space is added to the target directory. That directory is of course on the local server and not on the USB drive.
If the USB drive is connected and manually mounted when configuring the backup then the first backup is successful, and is created in /media/usbdisk/hostname on the USB drive. However the next backup attempt then fails, with error:
FATAL error, aborting operation
Badly formatted terminator, cannot extract catalogue location
Dar error during backup : 2
Backup terminated: backup failed - status: 7424
If the /media/usbdisk/hostname directory on the USB drive is renamed then successive backups work OK.
My conclusion is that there are two issues: the software needs to understand how to mount/unmount/test for mounting of the USB drive correctly, and it also seems to have a minor bug involving adding a space to the directory name in some situations. The "Backup share" selection in the configuration screen is also a little misleading as it is a misnomer - when backup is to USB then it is a mount point. However as the user does not have any options to change the default of usbdisk, it would be preferable to change the wording slightly and make this read-only, eg: "Mount point usbdisk".
I think that the /mnt/smb directory is used for the temporary files. That means that the full capacity of the USB drive is available for storing the backup.
On my wish list would be a request for an option to restore files to a new location, eg in an i-bay - ideally with a picklist for the final directory. This would give the user the option to check the files before doing a restore to original location.
Thanks again for this great contribution.
-
Jim Holland
FATAL error, aborting operation
Badly formatted terminator, cannot extract catalogue location
Dar error during backup : 2
Backup terminated: backup failed - status: 7424
The Dar exit code is:
2 - Error due to a hardware problem or a lack of memory.
-
Follow the directions at the start of this post. That's it.
Then go to the "Backup or restore" tab on the SME Server Console. That's it. The new options are in the drop down list on the bottom.
I did all of that, installed the rpm's and confirmed the installation and did the signal-event post-upgrade // signal-event reboot but still no extra options in the server management panel and I am running SME server 7.2..
Any ideas?
-
Ptah
...no extra options in the server management panel...
No extra panels are aded to server manager.
Extra options are added to the existing Backup or restore panel.
Read Reply #23 and
http://wiki.contribs.org/Backup_with_dar
-
jpl
I wonder if you could advise on the following situation please, ie how best to resolve it.
I successfully did a full backup of approx 95GB of data (87 file parts) to a removable 120Gb drive on a workstation. I then did two more daily incremental backups OK (4 parts & 1 part).
For the next three nights the backup has terminated with exit code 11, and messages saying that user mail folders were changed during reading.
WARNING! File modified while reading it for backup
The incomplete backup files (4 parts) are left in the tmp folder on the backup drive each night, and these get overwritten the next night, only to be left there again when the backup terminates again the following night.
Reading dar literature this appears to be correct behaviour, as the file being read is in an unknown state, but it certainly does not achieve the objective of completing the backup.
There are users email accounts that are automatically receiving & sending messages, so it's possible that email folders could change at any time.
At this rate I don't know when the system will complete an incremental backup again.
Any advice/comments/suggestion, thanks.
Emailed reports below:
No terminal found for user interaction. All questions will be assumed a negative answer (less destructive choice), which most of the time will abort the program.
WARNING! File modified while reading it for backup: /home/e-smith/files/users/xxxxxx/home/email/xxx yyy - Inbox.dbx
--------------------------------------------
9422 inode(s) saved
with 75 hard link(s) recorded
1 inode(s) changed at the moment of the backup
628563 inode(s) not saved (no inode/file change)
0 inode(s) failed to save (filesystem error)
254 inode(s) ignored (excluded by filters)
2445 inode(s) recorded as deleted from reference backup
--------------------------------------------
Total number of inode considered: 640685
--------------------------------------------
Dar error during backup : 11
Backup terminated: backup failed - status: 7424
==================================
DAILY BACKUP TO WORKSTATION REPORT
==================================
Backup started at Thu Sep 6 23:55:14 2007
Backup of mysql databases has been done.
Mounting backup shared directory stationxx/backup
Backup temp directory /mnt/smb/tmp_dir/server.xxxxxxxx.xxx.xx is mounted and is writable
Using set number 0 of 1
Attempt to incremental backup number 3 of 7
Backup base file name is inc-003-20070906
Making backup on temporary dir...
using a backup session timeout of : 17970 seconds
*** No backup allowed or error during backup ***
Dar error during backup : 11
Thanks
-
Geez, excuse me as I am very new to SME server.
I get what you saying about it not being in the server manager. Without having someone laugh at me now but how do you access the existing Backup or restore panel. As far as my knowledge stretch(erm..) there is two management gui's. One from the ssh-client with login as admin and the other is the webfront server management page. Am I missing something here??
:?
-
Ptah
I think you really need to read all of the manuals from start to finish, linked at the top of the forum as your knowledge and understanding seems weak.
I'm referring to what you call the "webfront server management page", which is commonly (and always as far as I know) called server manager, with a URL https://www.yourdomain.com/server-manager
and which by default is only accessible from your local network unless you enable remote access & enter your remote host IP.
Open server manager and click on the heading for Backup or restore, then select one of the options from the pull down menu (which should now have extra options if you have installed the correct e-smith-backup rpm).
With that knowledge please go and read the earlier posts and the wiki entry again, perhaps they will make more sense to you now.
-
JPL,
an actionscript to mount and unmount alone will not do the trick. If there is no tape in the drive (or in case of an usbdisk not being present) the mount will fail, so then the backup has to be canceled and a warning mail be sent that there was no tape in the drive. I don't think this can and i wouldn't know how this can be accomplished by an action-script.
I know the Backup2 contrib had a mount/unmount option (and if i remember correctly even an eject option) and it worked for my REV-drive. If it still works on SME7.2 this might be the better option for usb-REV-drives/usb-disks (although it is not maintained anymore) for now.
-
Hi jpl
You wrote to another poster:
I appreciate your contribution as it is exactly what I am looking for. However I also find that it is not quite ready for use for backups to USB. ...
Hi Jim,
Thanks for your comments. I agree my job needs improvement for usb disk backup. Here are some further details.
When usb type backup is used, no mount or umount is made. Dar backup files are written on the directory specified in the server-manager panel.
So if you specify /media/usbdisk in your panel and you connect an usk disk whitout mounting it, backup is made on your server's local /media/usbdisk directory.
Present script cares about a real mount for smbfs, cifs, and nfs, but not for usb.
I think that the /mnt/smb directory is used for the temporary files
No. Even temporary files are on the backup system.
So, what could be done ?
We cannot trust about the fact that the usb mount point is named /media/usbdisk. It can be another name. The admin must know this name and specify it in the server-manager.
But I think the problem of the real mount can be solved by adding a test of /proc/mounts, to verify ther is a real mount (not only for usb) in the backup script.
About the possibilty to do the mount, I think we should take in account the two situations : admin keeps his usb disk always mounted, or admin wants the usb disk is mounted before each backup, and unmounted after each backup.
If we want to provide the two possibilities, we have two solutions :
- keep things unchanged and resquire the admin to add an action which mounts the usb disk in the pre-backup event, and unmount it in the post-backup event (it could permit to handle specific situations)
- let the admin specify his choice with an additional option in the usb backup setting panel, and modify the scripts consequently.
The second way needs a little modifications of the backup script.
Comments and opinions ?
N.B. I see no reason about the failure of the second backup. After the mount, script is the same for usb and other types of backup support.
-
jpl
I wonder if you could advise on the following situation please, ie how best to resolve it.
...
Ray,
I think there is nothing else to do... than keeping an eye on the backups, to see if this error happens again for several days and should prevent the backup to succeed.
I think there is no way to accept in a valuable backup a file which changes during the backup. dar choice to cleanly stop the backup and preserve only safely backuped files is the way to go. But the documentation is not clear about what can cause an error 11 and the way dar detects a file change.
If dar detects a change between the moment it reads the beginning of file, and the moment it reads it's end, there is little chance this happens in two consecutive backups... But if dar consider files which are changed beetween the beginning and the end of the backup job, things are more problematic. In such case, perhaps should we consider the possibility to relaunch immediatly a backup ended with an error 11. But is it a safe way ?
-
JPL,
An option in the server-manager panel to mount/unmount would be fine for me but maintaining the 'Keep it simple' credo of SME Server in mind the most elegant solution would be to have some logic in the background doing this automagically.
Besides the mount/unmount issue i would still be interested in being able to eject (in my usb REV-drive case) after the unmount/backup to not by accident overwrite a (full) backup. An option for this would be very much appreciated.
Regards.
-
Hi,
I seem to have a severe problem with this.
My old server has given up the ghost, no problem I thought since I have taken daily backups on to my FreeNAS box, I can just restore it !
So I have built a new server, installed a fresh version of 7.2, configured the remote access and local networks to allow server manager, and installed the two required rpm via yum localinstall.
I then configured the network backup to point to the share on the FreeNAS box and started the restore on the latest (27mb) backup.
Lots of lovely messages telling me that files are restored, followed by
--------------------------------------------
9497 inode(s) restored
0 inode(s) not restored (not saved in archive)
0 inode(s) ignored (excluded by filters)
0 inode(s) less recent than the one on filesystem
0 inode(s) failed to restore (filesystem error)
0 inode(s) deleted
--------------------------------------------
Total number of inode considered: 9497
--------------------------------------------
Restore complete
You must reboot the server to activate any configuration changes that were made as a result of this restore.
Rebooted, went back to look at the server and NOTHING has been restored - no ibays and no files in them.
Thinking I must have done something wrong, I have repeated the install but gone to look at the file structure using WinSCP before the reboot. Same result, no restored files, NADA !
Tried to look at var/log/messages - not initialised.
Please, Please, what do I do next - I really do not fancy rebuilding 5 sites from scratch.
NWJ
-
Dar saying "restore complete" and doing no restore sounds strange to me.
Strange to me also is the last message about the reboot. I didn't remember about such a message from dar, and the restore action beeing made through the server-manager I thought that the reboot action was requested by the sever-manager panel. But perhaps I forgot this message.
I think you should first verify your backup.
Two actions are available to verify your backup :
- the first one is not yet possible in your situation : use workstation backup verify option from theserver-manager panel,
- the second is always possible : you can use selective restore panel to verify if the backup really contains the files you need (it must show them), and test if restore of one of them via selective restore is possible.
Please report the result, you must first be sure of the content of your backup.
-
Hi, JPL,
Thanks for your response.
I have done as you suggested. The files in the backup appear to be OK. I have restored small sections with no problems, but if I select a large number it just says "Restore Complete" without listing or indeed restoring any files.
There seems to be an issue with the number of files to be restored at one time, so I gradually increased the number that I was trying to restore until I got the message "Error while calling execvp:Argument list too long".
I hope this gives you a handle on the problem.
NWJ
-
Hi jpl
You wrote:
No. Even temporary files are on the backup system.
I have 140 GB to backup, which takes around 18 hours for a full backup. I see the temporary files appearing in /mnt/smb, which is on the source disk. Only when complete are they moved to the target USB drive directory of /media/usbdisk. That is ideal behaviour for me as it means that the maximum utilisation of the USB drive can be achieved.
About the possibilty to do the mount, I think we should take in account the two situations : admin keeps his usb disk always mounted, or admin wants the usb disk is mounted before each backup, and unmounted after each backup.
If we want to provide the two possibilities, we have two solutions :
- keep things unchanged and resquire the admin to add an action which mounts the usb disk in the pre-backup event, and unmount it in the post-backup event (it could permit to handle specific situations)
- let the admin specify his choice with an additional option in the usb backup setting panel, and modify the scripts consequently.
The second way needs a little modifications of the backup script.
Comments and opinions ?
I would be happy to have only the option of automatic mounting followed by auto unmounting at the end of the backup, although at present I leave it mounted all the time. We don't need too many options.
I am glad to report that subsequent attempts have achieved one successful full backup and one incremental backup. Now for a test restore on a spare machine!
Regards
Jim
-
Hi jpl
I see the temporary files appearing in /mnt/smb, which is on the source disk.
It's an error in my scripts, which I will look for. The temp backup should be on the backup host, not on the local server, even with usb disk. This is a security reason : if you program a daily full backup (probably the most frequent use of backup) and the transfer to the backup workstation or usb disk fails, you have no more available backup. But when the tmp backup is made on the backup workstation, if it fails the preceding backup is always available.
Yes, you need more space on the backup disk, but in the other solution you need more space on the server disk AND the backup process is far less reliable.
I would be happy to have only the option of automatic mounting followed by auto unmounting at the end of the backup, although at present I leave it mounted all the time. We don't need too many options.
I also think it's the way to go, and I will do more tests in this direction.
-
Hi JPL,
On a Restore from Workstation:
I also have "nowhinjing" 's problem, and this message in server-manager :-
After the restore completes you must reboot the server. Your restore is complete when /etc/smbpasswd has been restored and the words "Restore complete" appear at the bottom of your screen.
The backup files are being restored to eg.
Restoring file: /etc/e-smith/web/panels/manager/cgi-bin/sys
At the end of the restore, the message:-
Restore complete
You must reboot the server to activate any configuration changes that were made as a result of this restore.
The directory structure is in:- /etc/e-smith/web/panels/manager/cgi-bin/
But, only the /home and /root directory's have any files in them.
These files are not being moved to their correct locations after the reboot
With a selective restore:-
Files are being restored, I have phpmyadmin and eventum in /opt/
these are not being backed-up, they are not in the "40go-into" template.
Any info on how to get a full backup and restore, would be welcome.
TIA
-
The directory structure is in:- /etc/e-smith/web/panels/manager/cgi-bin/
But, only the /home and /root directory's have any files in them.
These files are not being moved to their correct locations after the reboot
Default backuped files and directories are the same as with tape or desktop restore.
/opt directory is not saved by default.
But hopefully e-smith-backup wit dar is templated, so you can add directories and files you want to backup/restore.
Copy /etc/e-smith/templates/etc/dar/Dailybackup.dcf/40go-into to /etc/e-smith/templates-custom/etc/dar/Dailybackup.dcf/40go-into and modify /etc/e-smith/templates-custom/etc/dar/Dailybackup.dcf/40go-into to add needed files and directories.
Then expand the template with :
/sbin/e-smith/expand-template etc/dar/Dailybackup.dcf
-
jpl
Just a minor correction, in case it was not obvious to readers
Dailybackup.dcf should read DailyBackup.dcf
Here's a corrected & slightly modified procedure:
mkdir -p /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf
cp /etc/e-smith/templates/etc/dar/DailyBackup.dcf/40go-into /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf
(the above is all on one line)
to add needed files and directories modify
/etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf/40go-into
Then expand the template with :
expand-template /etc/dar/DailyBackup.dcf
-
jpl
Just a minor correction, in case it was not obvious to readers
Dailybackup.dcf should read DailyBackup.dcf
Great thanks for the correction :???:
-
JPL,
I was very excited to discover this contrib as I've been looking for this functionality to roll out to my customers.
Unfortunately....
I tested this on my home/test server with a scheduled backup to a local usb disk (which had been manually mounted) and the backup process created the temporary files in /mnt/smb on the server itself which completely filled the server disks and brought the server to its knees.
I managed to delete the files using an SCP connection but obviously this is not a great solution if so much free space is required on the server for a successful backup.
I have gathered from your previous posts that the temporary files 'should' be placed on the USB disk itself which would be a much better solution.
Alternatively, could the temporary files not be transferred after the creation of each file instead of building a complete backup set before moving them all?
-
To all
Here's the procedure to set the backup file size.
The default slice or part setting is 700Mb (to suit CD's).
I wanted to change it to suit 4.7Gb DVD's, by making 3 parts fit to a DVD, so the slice size is 1529Mb (allowing for overheads & real data size).
mkdir -p /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf
cp /etc/e-smith/templates/etc/dar/DailyBackup.dcf/90slice /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf
(the above is all on one line)
to change the file size modify (& save)
/etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf/90slice
to read like this
{
($backupwk{Slice} || '1529M') eq '1529M' ? "--slice 1529M\n" : "--slice $backup
}
Then expand the template with
expand-template /etc/dar/DailyBackup.dcf
-
JPL,
I was very excited to discover this contrib as I've been looking for this functionality to roll out to my customers.
Unfortunately....
After my time was taken to release a Jabber server contrib for SME, I am about to modify e-smith-backup and test a release which mounts/unmounts usb devices, like other mounts, and I hope resolve these problems (but I not before next week).
But I think there is a difficult choice when we consider usb devices : usb keys have limited space and storing temporary backup files on usb can be difficult, usb disks have a lot of space for backuping large volume of server data and for such usb disks temporary backup files need to be stored on the usb disk. Maybe we should consider two usb logical devices for backup : usb key and usb disk...
-
Hi,
I successfully installed this release of e-smith-backup. However, I've got some problems with the display in the server-manager.
The new list items seems to be displayed as WORKSTN_CONFIGURE, WORKSTN_VERIFY and so on...same problem when I choose one of these new items.
For information, my SME is set up in french language.
Is there someone able to help me ?
Thanks in advance.
Bye.
-
This is because e-smith-backup is a modified core package, and this core package has a french translation without addition of new features.
Due to smeserver choices for locale packaging in SME 7 (for core packages), translations other than en-us are not in the package itself, but put together in a package for each language.
Nothing to do before updating smeserver french locale rpm, but I have an updated french localized file for backup panel, and I'll put it in my contrib area at the end of week (not at home before).
New post when done.
-
Right.
Thanx for your reply.
I'll wait till week end. :smile:
-
Here (http://beta.free-eos.org/sme/backup) is an updated french localized backup file to replace /etc/e-smith/locale/fr/etc/e-smith/web/functions/backup.
I am sorry to tell you that it is only partially translated, but it should not be difficult to complete the translation. However you'll not get messages like WORKSTN_CONFIGURE or WORKSTN_VERIFY :o
-
jpl,
I believe most of your work has been integrated into the e-smith-backup that can be found in the smetest repo. Would you be willing to work with byte (Gavin) to ensure all necessary changes are made to get this package ready for release. He is a great tester and will help verify that all functionality that you have worked on is there.
-
jpl & others
I ran into issues with the size of removable backup disks (120Gb IDE on a workstation) and the resultant backup data size (120Gb of data compressing to approx 50Gb).
I was able to get one full backup onto a disk, and a weeks worth of daily incrementals, but the next weekly full backup failed, due to the need to save the new backup in the tmp folder on the removable disk, before deleting the old backup files. There was not enough space on the disk (size) I was using.
I created a cron job and put it in /etc/cron.d with a script in /opt/scripts/..., which mounts the drive and deletes the old backup just before the new full backup commences. It works quite OK.
I realise this deletes the old backup before the new one has completed, but I'm happy to forgo that in order to use the available hardware.
In /etc/cron.d/deletebkp1
30 1 * * 3 root . /opt/scripts/deletebkp1
In /opt/scripts/deletebkp1
umount /mnt/smb
/bin/mount -t smbfs //stationXX/bkp1 /mnt/smb -o username=XXXXXX,password=XXXXXXXXXXXXX
rm -f -R /mnt/smb/servername.yourdomain.com/
rm -f -R /mnt/smb/tmp_dir/
-
SRPMS available at http://beta.free-eos.org/sme/RPMS/e-smith-backup-1.14.0-16eos1.el4.sme.src.rpm
This link doesn't seem to work at the moment and when I trace down throught he server at EOS the branch isn't there either.
Has it gone somewhere else to live?
Ed Form
-
Has it gone somewhere else to live?
We have incorporated JPL's excellent work in the e-smith-backup base version which you can find in smetest repo:
http://distro.ibiblio.org/pub/linux/distributions/smeserver/releases/7.2/smetest/i386/repodata/repoview/e-smith-backup-0-1.15.0-05.el4.sme.html
The more people who test and use this package means it can get in to base and be a standard package, to install you can do by typing:
yum upgrade --enable smetest e-smith-backup
Remember this is in smetest so there will/may be bugs so don't forget to open a bug and open a bug in the SME Server 7.x release as the package in smetest is not a contrib. Thanks.
PS. JPL would you be interested in working together to get this polished and in to the base ? Thanks.
-
PS. JPL would you be interested in working together to get this polished and in to the base ? Thanks.
Sure, I was about to post an update to Gavin (byte). But if this is not the prefered way to do it, please send me the right way by mail.
-
Dear All
The How to has been updated
http://wiki.contribs.org/Backup_with_dar
-
I've had a rather disturbing incident with this backup procedure today.
I had to replace a disk in my server because of persistent dropping offline so I decided to rely on my backups and remake the disk mirror from scratch.
Loaded e-smith, did basic setup in the server manager and then loaded the e-smith-backup with dar system using yum --enable smetest upgrade e-smith-backup. After the reboot at the end of the install, I also ran the software installer and accepted the full set of updates available followed by another reboot from the reconfigure button at the bottom of the yum results.
Then did a full restore from the latest backup - it's on a NAS device - and it ran right through and stopped but didn't put up the reboot notice at the end which the instructions indicated I should get. After a reboot there were apparently no files in the data store ibay!!!
Ran the full restore again and this time I noticed that it was putting the restored files in /etc/e-smith/web/panels/manager/cgi-bin. I've looked again at the configuration of the backup and I can't see anything I could have done to cause this so why isn't the program restoring to the original locations?
Any ideas anyone!
Ed Form
-
I've updated e-smith-backup from the test repo and trying to backup to an Iomega usb REV (rrd) drive. The rev-drive is normally in /media/usbdisk and through the backup admin-panel i can select it when configuring a 'backup to workstation'. I've created an action for pre-backup to mount the REV-cartridge with (otherwise i presume the backup will end up in the /media/usbdisk/ directory in stead of on the rev-cartridge):
#!/bin/sh
exec /bin/mount /media/usbdisk
and two actions for post-backup to umount and eject the REV-cartridge:
#!/bin/sh
exec /bin/umount /media/usbdisk
and
#!/bin/sh
exec /usr/bin/eject /media/usbdisk
But i must be doing something wrong 'cause nothing appears on the rev-cartridge.... suggestions anyone?!
Thanx!
jester.
-
If you use last available version of e-smith-backup (e-smith-backup-1.15.0-7, or even -6)), the mount is made by the backup script.
So if you mount /media/usbdisk through a pre-backup script, the backup script should normally fail.
-
Thanks jpl, got things working now.... on with the testing.
-
Hello!
I have an annoying problem on my server with this contrib.
When I try a sel_restore from the web interface of server-manager, response is pretty bad:
"Corrupted database :Not a zero terminated string in file"
What's exactly going on? i didn't read/find anything about this problem...
Thanks!
-
I didn't see any mention of this, but would I be able to use e-smith backup to automatically backup selected folders on each one of my Windows XP machines?
Let's say I have an sme server, running e-smith-backup, and 3 Windows XP (pro or home) machines, with some shared folders:
XP 1 has "My Documents", "Games" & "Client invoices"
XP 2 has "My Documents", "random stuff", "quotes", "client invoices", "client work"
XP 3 has "My Documents", "client invoices"
How would I use e-smith-backup to automatically backup all of these to say /home/e-smith/files/ibays/backups/{machine name}?
Cause then I can automatically backup the backups ibays to my USB backup drive
-
SoftDux
e-smith-backup does not support that funtionality.
If you read earlier parts of this thread, you will see commands that allow you to mount a windows share, so if you created shares for those locations you could then mount them to /home/e-smith/files/ibays/backups/....
and they would be included in normal daily backups.
You would have to create seperate cron jobs/scripts to handle the mounting and unmounting.
There is the BackupPC contrib which is designed to do this.
Why not just point those locations on your windows PC's to use the users home drive (H:), or enable roamimg profiles, then that data is automtaically included in backups.
-
ok, thanx. I'll check out BackupPC
-
SoftDux
Assuming you are using sme as a domain controller, and as you only have 3 workstations, it would be very easy to just click on My Documents/Properties and change the path to H:, and then that stuff will automatically be saved on the server in the users home drive, and will therefore be included in normal backups, nothing extra to install or configure.
Alternatively if you don''t use sme as a domain controller and don't have home drives, change My Documents to another shared drive eg P:\username which happens to be an ibay, that will also be included in normal backups without any extra effort.
-
I don't use sme as a domain controller, and I don't want to move the Documents folder to a share on the server. I'm looking for something to implement on all the networks that I administrate, some have Windows XP, some Windows 2000, Some MAC, some Linux - all use samba though, so the principle will stay the same.
Ideally, I'd prefer to "pull" the files from the sme server, so the mount option will work fine :)
-
I have a small problem with this contrib, which otherwise is really helpful. I choose to backup to a USB-disk attached to the server, and the script automatically (?) selects "/media/usbdisk" as the backup-device, but this disk is non-existent. I want it to use "/media/usbdisk1" which is where the external disk is.
I found out that I can change the name of the disk with
/sbin/e-smith/db configuration setprop backupwk SmbShare media/usbdisk1
But this is annoying to do every time I change the backup settings.
Any ideas / help?
-
holck
http://wiki.contribs.org/USBDisks
-
Thanks for the link :-)
I have now labeled my USB backup disk:
# /sbin/e2label /dev/sda3 USB-backup
The two relevant lines in /etc/fstab:
/dev/sda3 /media/usbdisk1 ext3 defaults
/dev/sda1 /media/usbdisk ext3 pamconsole,noatime,sync,exec,noauto,managed 0 0
I changed to:
#/dev/sda3 /media/usbdisk1 ext3 defaults
LABEL=USB-backup /media/usbdisk ext3 defaults
/dev/sda1 /media/usbdisk ext3 pamconsole,noatime,sync,exec,noauto,managed 0 0
I hope it works, but I am little concerned whether these changes to /etc/fstab will "survive" various events and system changes.
-
holck
Not everything in sme is templated.
I think fstab changes will survive reconfiguration.
You can easily prove this by doing
signal-event post-upgrade
signal-event reboot
and then see if your fstab changes are still there.