Koozali.org: home of the SME Server

how to succesfully backup installed contribs?

Offline SoftDux

  • *****
  • 240
  • +0/-0
    • http://www.SoftDux.com
how to succesfully backup installed contribs?
« on: March 27, 2008, 09:28:03 AM »
Hi all

I know this has been discussed before, but searching for the words "backup contribs" returns thousands of threads, so I'm going to ask it straight.

I had to reinstall my SME 7.3 again last week after power failure, but I have a backup :) Thought the backup was a week old, I was on holiday so it's not so old then. The backup was done with Affa backup, and thought I could copy backup the users, group, shadow, passwd & samba stuff to /etc from the USB HDD, I couldn't see any of my many installed contribs on it?

Quote
# EDIT THIS:
my %job=(
        'remoteHostName'=>'localhost', # FQHN or IP address
        'TimeSchedule'=>'2230', # HHMM,HHMM,...
        'Description'=>'Backup server to HDD', # text string.
        'scheduledKeep'=>1, # integer >= 1
        'dailyKeep'=>7, # integer >= 1
        'weeklyKeep'=>5,# integer >= 1
        'monthlyKeep'=>12,# integer >= 1
        'yearlyKeep'=>2,# integer >= 1
        'SMEServer'=>'yes', # yes | no
        'Include[0]'=>'', # additional files or directories to include
        #'Include[1]'=>'',
        #'Include[2]'=>'',
        'Exclude[0]'=>'', # files or directories to exclude from backup
        #'Exclude[1]'=>'',
        #'Exclude[2]'=>'',
        'RPMCheck'=>'no', # yes | no
        'DiskSpaceWarn'=>'strict', # strict | normal | risky | none
        'localNice'=>0, # -19...+19
        'remoteNice'=>0, # -19...+19
        'Watchdog'=>'yes', # yes | no
        'sshPort'=>22, # default ssh port is 22
        'ConnectionCheckTimeout'=>120, # seconds
        'rsyncTimeout'=>900, # seconds
        'rsyncCompress'=>'yes', # yes | no
        'EmailAddresses'=>'', # name@domain.com,name@domain.com,...
        'chattyOnSuccess'=>0, # send N success notifications
        'postJobCommand'=>'', # full path to local program/script
        'preJobCommand'=>'', # full path to local program/script
        'AutomountDevice'=>'', # Device to auto mount (e.g. USB drive)
        'AutomountPoint'=>'', # the mountpoint for AutomountDevice
        'AutomountOptions'=>'', # Optionstring passed to mount command e.g. '-t cifs'
        'RootDir'=>'/usb_hdd/files', # where to save the archives. Don't use /root or /home/e-smith
        'Debug'=>'no', # yes | no
        'status'=>'enabled', # enabled | disabled
        'rsync--inplace'=>'yes', # yes | no : rsync on source supports '--inplace' option
        'rsync--modify-window'=>0, # integer >= 0, timestamp window
        );

### nothing to edit below this line ####


So, I obviously didn't configure it properly, or it cannot backup installed contribs. So, how do I backup additional contribs that I install after SME has been setup? I have been thinking about using DAR / DAR2 as well, so this question isn't limited to Affa only. Any other backup solution that can backup my users, ibays, MySQL DB's and contribs on a USB HDD will do :)

Offline raem

  • *
  • 3,972
  • +4/-0
Re: how to succesfully backup installed contribs?
« Reply #1 on: March 27, 2008, 12:21:45 PM »
SoftDux

You still have a lot to learn about sme !

Quote
So, I obviously didn't configure it properly, or it cannot backup installed contribs.
So, how do I backup additional contribs that I install after SME has been setup?
I have been thinking about using DAR / DAR2 as well, so this question isn't limited to Affa only. Any other backup solution that can backup my users, ibays, MySQL DB's and contribs on a USB HDD will do :)

The sme backup & restore concept is that only data (users & ibays) & configuration (including all mysql dbs) is backed up & restored.
All contribs (ie installed as rpms) are not backed up.

A restore is done to a fresh installation of sme server, and then contribs are reinstalled.
As configuration data is restored, you should not need to setup those reinstalled contribs again.

It's good admin practise to create an ibay specifically to house a copy of every rpm you install, so you can easily reinstall those from your backed up & restored copies, although with yum repos being available now it's not so necessary to do this. It does provide a reminder of everything you have installed though, in case you forget.

Affa provides you with a report of missing rpms (if you enable that setting), which is the difference between the source server and the backup server, so you can easily see what needs reinstalling after a restore or Affa "rise", or install those on the Affa backup server in advance (if using a separate backup server).

Note that a tape backup backs up all file systems on the server, but only restores the "standard restore" files. See this list http://wiki.contribs.org/Backup_server_config#Standard_backup_.26_restore_inclusions

If you wish to backup "everything" (including installed rpms), then the standard sme backup methods, which includes most of the available backup contribs eg DAR2, Backup with dar, etc, do not backup "everything".
You need to perform a bare metal or disk clone backup to achieve that.
The contrib that does that is Mondo, but I'm not sure if it was updated to be compatible with sme7.x. Another way of easily doing that is to swap out a RAID1 disk, and plug in a clean (empty) disk.
Disk cloning software can also be used, but it must support RAID partitions and lvm volumes, and some software does not.
The Linux dd command will also copy every bit on a disk, to another mounted disk, but it will be slow and may take a long time for a disk with a lot of data.

...

Offline SoftDux

  • *****
  • 240
  • +0/-0
    • http://www.SoftDux.com
Re: how to succesfully backup installed contribs?
« Reply #2 on: March 27, 2008, 04:26:54 PM »
I knew my request could get this kind of attention...... :)


SoftDux

The sme backup & restore concept is that only data (users & ibays) & configuration (including all mysql dbs) is backed up & restored.
All contribs (ie installed as rpms) are not backed up.

A restore is done to a fresh installation of sme server, and then contribs are reinstalled.
As configuration data is restored, you should not need to setup those reinstalled contribs again.

It's good admin practise to create an ibay specifically to house a copy of every rpm you install, so you can easily reinstall those from your backed up & restored copies, although with yum repos being available now it's not so necessary to do this. It does provide a reminder of everything you have installed though, in case you forget.

I have already done that to save some bandwidth and to make it easier to see what I have installed, but it's still a pain in the neck!

Affa provides you with a report of missing rpms (if you enable that setting), which is the difference between the source server and the backup server, so you can easily see what needs reinstalling after a restore or Affa "rise", or install those on the Affa backup server in advance (if using a separate backup server).
Well, since Affa isn't installed (or restored for that matter), it's useless to me right now.

If you wish to backup "everything" (including installed rpms), then the standard sme backup methods, which includes most of the available backup contribs eg DAR2, Backup with dar, etc, do not backup "everything".
You need to perform a bare metal or disk clone backup to achieve that.
The contrib that does that is Mondo, but I'm not sure if it was updated to be compatible with sme7.x. Another way of easily doing that is to swap out a RAID1 disk, and plug in a clean (empty) disk.
Disk cloning software can also be used, but it must support RAID partitions and lvm volumes, and some software does not.
The Linux dd command will also copy every bit on a disk, to another mounted disk, but it will be slow and may take a long time for a disk with a lot of data.
[/quote]
None of these options are really a recommendation :(

I have 4x 250GB HDD's in my machine, so it's not going to work for me. Where do contribs normally get installed? In various places? which folders to I need to backup to backup the contribs?

I know one particular script I had for my UPS, installed /etc/upsmon which wasn't even backed up, so the /etc folder doesn't get backed up either, only a few selected files.

Does Dar / Dar2 have a web interface where one could include / exclude files & folders to be backed up? Right now I don't have any backups as I just want to research a bit more first

Offline raem

  • *
  • 3,972
  • +4/-0
Re: how to succesfully backup installed contribs?
« Reply #3 on: March 27, 2008, 08:49:23 PM »
SoftDux

Quote
Well, since Affa isn't installed (or restored for that matter), it's useless to me right now.

If you use ANY add on contrib for backup purposes eg Affa or Dar2 or whatever, then after doing a fresh reinstall of the sme OS, the first thing you do is obviously reinstall that backup contrib and do some basic configuration manually. Then you would restore using that contrib, then you can reinstall all other contribs etc.

From what I have read earlier you will get into trouble trying to restore rpms which have executables installed in the normal Linux file system locations, that's why they should be reinstalled. Also some contribs will need to be updated if you restore to a newer version of the sme OS, so if you automatically restore old contribs, then you will break your server, in some cases.


Quote
None of these options are really a recommendation

They are recommendations, I have used them all except Mondo, and they all work fine, but they all have their own practical issues.
Those methods were a summation of some existing possibilities, you chose what you want to use or not use, depending on your situation.

Quote
I know one particular script I had for my UPS, installed /etc/upsmon which wasn't even backed up, so the /etc folder doesn't get backed up either, only a few selected files.

That's why scripts etc are best implemented via custom templates (which do get backed up), and any scripts can be put into /opt/xxx, and you then add /opt to your list of backup inclusions, or put into an ibay if preferred.


Dar2 has a web interface that is quite configurable.
If you exclude files then you don't really have a full backup do you !!!

I have found that DAR2 is useful if you do both full backups (ie the disaster recovery job) & other additional backup jobs (of selected data sets), again this depends on your particular needs.


...

Offline SoftDux

  • *****
  • 240
  • +0/-0
    • http://www.SoftDux.com
Re: how to succesfully backup installed contribs?
« Reply #4 on: March 27, 2008, 09:23:49 PM »
If you use ANY add on contrib for backup purposes eg Affa or Dar2 or whatever, then after doing a fresh reinstall of the sme OS, the first thing you do is obviously reinstall that backup contrib and do some basic configuration manually. Then you would restore using that contrib, then you can reinstall all other contribs etc.

true, I'd need to install Dar2 / Affa / etc to restore the backups, but to reinstall all those contribs is a pain in the neck. I have however thought of maybe writing a script that could install them for me, but since some of them need the server to be rebooted after installation, it could mean I need to write a few scripts. What does one need todo to nominate a certain contribs to be part of the base SME installation? Say like SME Admin, or Dar2 for example?

From what I have read earlier you will get into trouble trying to restore rpms which have executables installed in the normal Linux file system locations, that's why they should be reinstalled. Also some contribs will need to be updated if you restore to a newer version of the sme OS, so if you automatically restore old contribs, then you will break your server, in some cases.
I hear what you're saying, but that's only going to be the case if one installs a newew version of SME than what was installed before the problem arose. I for one update SME via yum update, which effectively should update the contribs (thos that use rpm's at least?) installed, right? Thus, if the backup was done a day or two before disaster struck, then the restore process will restore the system as it was when it was backed up


That's why scripts etc are best implemented via custom templates (which do get backed up), and any scripts can be put into /opt/xxx, and you then add /opt to your list of backup inclusions, or put into an ibay if preferred.

But what about scripts that expect config files in /etc? What will happen if I backup the whole /etc folder? Could it cause problems upon restoration?

Are you saying with "custom templates" that contrib writers should actually be using that custom templates? How does this work? How do I add upsmon or mrepo for example to a custom script?

Dar2 has a web interface that is quite configurable.
If you exclude files then you don't really have a full backup do you !!!
Yes, and no. What I meant with "include & exclude", is say I want to include the whole /etc folder in the backup, but exclude a few files folders from /etc, instead of choosing each & every file in /etc to include...
« Last Edit: March 27, 2008, 09:27:17 PM by SoftDux »

Offline SoftDux

  • *****
  • 240
  • +0/-0
    • http://www.SoftDux.com
Re: how to succesfully backup installed contribs?
« Reply #5 on: March 27, 2008, 09:27:46 PM »
P.S. Is it possible to increase the size of this reply box, at the bottom of the forum, when I need to reply to posts?

Offline raem

  • *
  • 3,972
  • +4/-0
Re: how to succesfully backup installed contribs?
« Reply #6 on: March 27, 2008, 10:32:17 PM »
SoftDux

Quote
to reinstall all those contribs is a pain in the neck

Agreed, but that's how the sme server backup and restore concept works. It's a development design decision.
If you want to change it or create another way of doing it, then take your requests and/or code to the bugtracker please.

See the link to dmay's howto in another post re building a sme6 ISO, and many other posts on the subject from a couple of years ago, I don't know how much of it will still apply to sme7.
Keep in mind that a lot of people got badly stung using a unofficial highly customised release of sme6.01.xx when upgrade time came, as there were many incompatibilities (rpms etc) that broke the standard upgrade process.
At that time some blame was cast at developers for upgrades that went wrong, but that release was not created by the main developers. In actual fact the developers did a lot of extra work in future releases to try and overcome the issues caused by the unofficial "gung ho" ISO release.


Quote
I hear what you're saying, but that's only going to be the case if one installs a new version of SME than what was installed before the problem arose.

There were two parts to my answer, one is that restoring "executables/binaries" is fraught with problems, the other related to incompatible rpms versions. They are separate issues and you ignored the first one, which I gather is the bigger problem. Charlie Brady advised/informed of the first issue some time ago in these forums.


Quote
what about scripts that expect config files in /etc? What will happen if I backup the whole /etc folder? Could it cause problems upon restoration?

Yes you will have problems, the issues are similar to that mentioned above.
Only some of /etc is backed up, here is the standard inclusions
/etc/e-smith/templates-custom
/etc/e-smith/templates-user-custom
/etc/group
/etc/gshadow
/etc/passwd
/etc/samba/secrets.tdb
/etc/samba/smbpasswd
/etc/shadow
/etc/smbpasswd
/etc/ssh
/etc/sudoers
/home/e-smith
/root

Note that all /etc/config files are a result of system installed code or template code or custom template code.
A new install of the OS installs standard /etc/config files, rpm contribs install additional templates that create additional or add to /etc/config files and custom template changes also add to /etc/config files. Only custom templates are included in back ups, as all the rest are easily recreatable.


Quote
..with "custom templates" .... contrib writers should actually be using that custom templates?

No, they should use the templates folder. It all ties in with the concepts we have been talking about throughout this thread ie a reinstall of contrib rpms will add new templates to the templates folder.


Quote
How does this work? How do I add upsmon or mrepo for example to a custom script?

It's time for me to point you to the Developers Guide, which you should read extremely slowly and work through the examples given on a test server. Then read it again another two or three times.
If you want to do development work, use the devinfo list (although I think that is not being used so much these days), or open specific bugs to track development of a particular rpm/function/code (which seems to be the preferred approach these days).

http://wiki.contribs.org/SME_Server:Documentation:Developers_Manual


Quote
What I meant with "include & exclude", is say I want to include the whole /etc folder in the backup, but exclude a few files folders from /etc, instead of choosing each & every file in /etc to include...

The DAR2 contrib has sections for Backup selections, Prune directories, and Exclude files, so you tailor it as you wish.

The general approach you are referring to (backup & restore of contribs) will no doubt cause you problems later, so I seriously suggest you do not try to change the underlying concept.

Search these forums a lot more ie particularly on posts by Charlie Brady as he has given us all some real "gems" of information, not to overlook many others who have told us a lot of useful info as well eg Gordon Rowell and Shad Lords etc.

Some advice.
A lot of development work has gone into the sme server over many years and it is an accumulation of much knowledge and experience, and long ago determined design criteria. Of course change is implemented as practically required and as developer time input allows. This is free software to most of us, but many have paid the price somewhere along the way, usually in contributed time or paid for specific sponsored development effort.

You seem to want sme server to be something else or perform differently accordingly to your desires or the way you see it. I suggest you use the software as it is, keeping in mind the considerable degree of effort that has got it to where it is today, far more effort than you could ever do.

...

Offline raem

  • *
  • 3,972
  • +4/-0
Re: how to succesfully backup installed contribs?
« Reply #7 on: March 30, 2008, 08:52:13 PM »
SoftDux

Quote
....restoring "executables/binaries" is fraught with problems....
Charlie Brady advised/informed of the ... issue some time ago in these forums

Here is that thread
http://forums.contribs.org/index.php?topic=27097.msg111256#msg111256
...

Offline dadoudidon

  • *
  • 84
  • +0/-0
Re: how to succesfully backup installed contribs?
« Reply #8 on: April 01, 2008, 06:14:11 PM »

The contrib that does that is Mondo, but I'm not sure if it was updated to be compatible with sme7.x.

Mondo was pretty nice to use on sme 6. It was including a panel in the server-manager.
I dream that it will run one day on sme 8 beacuse it was for me the best and simple way to backup and restore a full server including contribs.

david
« Last Edit: April 01, 2008, 06:15:59 PM by dadoudidon »

Offline SoftDux

  • *****
  • 240
  • +0/-0
    • http://www.SoftDux.com
Re: how to succesfully backup installed contribs?
« Reply #9 on: April 01, 2008, 08:36:47 PM »
ok, I have decided to include all the config files for the contribs I use in the Dar2 backup script, but would like to know something about the eGroupware contrib, if anyone is using it. How do you backup this contrib?

Offline raem

  • *
  • 3,972
  • +4/-0
Re: how to succesfully backup installed contribs?
« Reply #10 on: April 02, 2008, 06:02:07 AM »
SoftDux

Quote
I have decided to include all the config files for the contribs I use in the Dar2 backup script, but would like to know something about the eGroupware contrib, if anyone is using it. How do you backup this contrib?

It depends how and where you installed it.
If by tar.gz file to an ibay and manual configuration of custom templates (to templates-custom), then it is already fully backed up.
mysql dbs are included in most standard backups.

If by rpm to /opt then you would need to include /opt in your backup inclusion list. The rpm will have installed templates to the templates folder rather than the templates-custom folder.
You can see what scripts and files are installed by the rpm with
rpm -ql rpmname
but it gets messy now if you want to selectively include "stuff" from different locations.
It really depends on what the contrib rpm has done as to how practical or feasible it is, and whether doing so will mess up your server for restores and upgrades.
...

Offline SoftDux

  • *****
  • 240
  • +0/-0
    • http://www.SoftDux.com
Re: how to succesfully backup installed contribs?
« Reply #11 on: April 02, 2008, 02:26:54 PM »
well, that's the problem. The eGroupware contrib is an rpm, so it doesn't own thing according to the developer's preference. Are there things in /opt that shouldn't be backed up?

Offline aussierob

  • ***
  • 55
  • +0/-0
MONDO on SME 7.4
« Reply #12 on: September 21, 2009, 04:23:27 PM »
Mondo was pretty nice to use on sme 6. It was including a panel in the server-manager.
I dream that it will run one day on sme 8 beacuse it was for me the best and simple way to backup and restore a full server including contribs.

david

Hi David,
I too loved mondo - mondo rocks!

One CD / DVD of everything (on a small server) to restore from,
so easy recover or disk upgrade (sme, contribs, data - the lot).
One DVD to take sensitive data off site with you.
Did you get mondo running on sme7 (7.4)?
 
Maybe you and I are missing something, and maybe a tape is rhe only way to back everything up, but for some sites a DVD holds enough and with blueray the capacity jumps again, so it seems to have a place to me.
SME rocks too, but this does not seem considered a core requirement (no criticism team - just curious)
regards from Down Under

Offline Stefano

  • *
  • 10,894
  • +3/-0
Re: how to succesfully backup installed contribs?
« Reply #13 on: September 21, 2009, 04:30:30 PM »
aussierob:

AFAIR mondo doesn't work very well with lvm and raid..

I will add that if you have to do a restore on a bare metal different from the original, you could have some problems..

finally, SME installs on modern hw in.... 5 minutes? if you have the list of the contribs to install, you are ready in.. half an hour (data restoration time excluded)..?

so, IMHO, cloning a linux server to an image is not a wise idea..

my 2 c

Stefano

Offline aussierob

  • ***
  • 55
  • +0/-0
Re: how to succesfully backup installed contribs?
« Reply #14 on: September 21, 2009, 06:34:31 PM »
aussierob:

AFAIR mondo doesn't work very well with lvm and raid..

I will add that if you have to do a restore on a bare metal different from the original, you could have some problems..

finally, SME installs on modern hw in.... 5 minutes? if you have the list of the contribs to install, you are ready in.. half an hour (data restoration time excluded)..?

so, IMHO, cloning a linux server to an image is not a wise idea..

my 2 c

Stefano
Fair points Stefano, but
1. I still want to put data on a removable media each night so it can be removed next day and taken off site (and THAT needs recovery beyond sme).
2. Also I have found the install with RAID slow (45mins after an hour or two to format 2x1TB), and the initial raid sync takes ages too (maybe some raids are not equal - I was warned to set the drives down from 3Gb to 1.5Gb)
Regardless of the above, one of the great SME features is the user community - thanks for comments
Robert
ps I must hustle up a box and give sme8 a test fly.