Topic: USB Backup Rotation

[Bud]

hope you guys can help

i have a client that requires to make certain backups to a 500Gig External WD USB Drive.

Questions:
 1. I need to Backup the Following Data from the SME Server;
      a. 5 x Users Email Data
      b. 4 x MySQL Databases
      c. 2 x Webshare Folders
      d. 4 x ibays
      e. All Users Documentation (Word, Excel etc.) from user Directories including Outlook PST Files.
      f. Quickbooks Shared Database Folder (ibay)
What Application would you Recommend I use on the SME Server for the Backup eg: AFFA?

2. I need to Backup ALL this Data onto a 500 Gig External WD USB Drive, would i use the  http://wiki.contribs.org/USBDisks Contrib to Auto Mount the External USB Drive?

3. How do I know that the Data Backup went Ok and that all Data Integrity is Fine?

4. I have 2 x 500Gig WD External USB Drives and need to Rotate them on a Monday and Thursday. Can I just unplug the First 500Gig External WD USB Drive on Thursday Afternoon and Plugin the Next 500Gig External WD USB (Without Mounting the Drive) Drive for the Thursday to Monday Backup?

5. When and If I need to Restore Data what is the Procedure?

6. Are there any Other Contribs I need to Install to make my Life Easier for Data Backups/Restore?

Any Help Greatly Appreciated 

[janet]

Bud

I suggest you use the built in backup function.
See the server manager Backup or Restore panel, select Configure workstation backup & select Local USB disk.

If you have data in /opt to backup eg webshare folders, then add /opt to the dar includes. See the Backup with dar Howto. All other data plus the server configuration will be included in a default backup. See the Backup server config Howto for details of what is included.

You do not need to use Affa although it can be configured to backup to local USB. Affa is very good for offsite backups via Internet.

You should format your USB disk as ext3 for best performance, see http://wiki.contribs.org/USBDisks
The drive will be mounted before the backup starts and unmounted after it finishes, so no need to do any automount configuration. The drive should normally be left unmounted.

You will receive a comprehensive backup report by email each day when the backup is done.

I think you will have to do a full backup on each disk before you rotate them, and then do daily incrementals. Take a look at the scheduling possibilities when first configuring the backup ie number of sets and number of dailies in a set etc.

To restore data use the same server manager Backup or restore panel, and select the workstation restore options. You do a full restore to a clean install of SME server, or you can do selective file restores to an existing SME server.

There are other Backup applications, see the Backup server config Howto for names of a few, but you do not need to use those as the built in backup to workstation (USB) will do what you want. Don't complicate things by adding applications if not necessary.

The best learning is by doing, so setup a workstation backup to local USB, and see what happens.

[Bud]

mary

thank you very much for your help. it all makes sense and is much more easier to use and manage backups this way

my only question about the external usb drives are, must and can i also use the SME http://wiki.contribs.org/Disk_Manager Contrib or do i not need it ?

thanks again 

[janet]

Bud

must and can i also use the SME http://wiki.contribs.org/Disk_Manager Contrib or do i not need it ?

I do not personally see the need for using that contrib.
There is some useful info re formatting your drive to ext3, which is also in the USB Howto.
Generally speaking the least number of contribs you install, the easier it will be to manage your server under all eventualities.
Only install contribs where the functionality is absolutely needed for your business or personal reasons.

[Bud]

thanks mary i am using dar2 but i need some help

i have an External 500Gig USB Drive Connected to the SME 7.51 Server

i did the following

1. fdisk /dev/sdc1
2. mkfs.ext3 -L MyData /dev/sdc1
3. mkdir -p /media/MyData
4. mount LABEL=MyData

the drive does mount and is available

in the Server Manager Menu Option " Disk Archive " i have created a Backup Job with the Following

a. Job Name: myjob
b. Description: My Backup
c. Backup Slections: home/e-smith/files/ibays/mydata/files/eBooks
d. Target: media/MyData
e. File system: ext3
f. Slice size: 650
g. Compression: 0
h. Expiry: 0
j. Verbose: off
k. Backup day: manual
l. Backup time: 23:30

now whan i do a " Backup selections " to manually do the backup all seems fine
when i look at " media/MyData/server.name/myjob " i can see the backup folder name " myjob" and a file called " db.zip " inside the myjob folder

my problem is that the backup has not been done as there is NO " backupname.dar " File inside " media/MyData/server.name/myjob " just the  " db.zip " file

any ideas what i'm doing wrong ? 

please help i need to get this going asap 

[janet]

Bud

You should look at the log files for clues eg the messages log file, and then the dar backup log file(s)
As per the server manager panel instructions, the Target should be specified in the format of
/media/usbdisk
in your case
/media/MyData

Also you should search these forums for similar usage issues eg search on dar2

[Bud]

Mary

i have looked in " /var/log/dar2 " and there are no files
when i go to server manager and look for dar or dar2 log files there is no dar options to choose

what to do ? 

[janet]

Bud

Please READ  my suggestions again, ie look in the messages log file.

[Bud]

mary

i have uninstalled and reinstalled the DAR2 Contrib including a " signal-event post-upgrade; signal-event reboot "

checked the backup job in " Disk ARchive " then did a Manual Backup

thanks i did what you said and this is the output

===================================================

Sep 23 08:44:47 bear kernel: scsi3 : SCSI emulation for USB Mass Storage devices
Sep 23 08:44:47 bear kernel:   Vendor: WD        Model: My Passport 071A  Rev: 2011
Sep 23 08:44:47 bear kernel:   Type:   Direct-Access                      ANSI SCSI revision: 02
Sep 23 08:44:47 bear kernel: SCSI device sdc: 976717824 512-byte hdwr sectors (500080 MB)
Sep 23 08:44:47 bear kernel: sdc: assuming drive cache: write through
Sep 23 08:44:47 bear kernel: SCSI device sdc: 976717824 512-byte hdwr sectors (500080 MB)
Sep 23 08:44:47 bear kernel: sdc: assuming drive cache: write through
Sep 23 08:44:47 bear kernel:  sdc: sdc1
Sep 23 08:44:47 bear kernel: Attached scsi disk sdc at scsi3, channel 0, id 0, lun 0
Sep 23 08:44:47 bear kernel: USB Mass Storage device found at 3
Sep 23 08:44:47 bear scsi.agent[8493]: disk at /devices/pci0000:00/0000:00:1d.7/usb1/1-3/1-3:1.0/host3/target3:0:0/3:0:0:0
Sep 23 08:44:47 bear last message repeated 2 times
Sep 23 08:44:47 bear fstab-sync[8576]: added mount point /media/MyData for /dev/sdc1
Sep 23 08:45:01 bear su(pam_unix)[8599]: session opened for user qmailr by (uid=0)
Sep 23 08:45:07 bear su(pam_unix)[8599]: session closed for user qmailr
Sep 23 08:47:03 bear sshd(pam_unix)[8667]: session opened for user root by (uid=0)
Sep 23 08:48:12 bear kernel: kjournald starting.  Commit interval 5 seconds
Sep 23 08:48:12 bear kernel: EXT3 FS on sdc1, internal journal
Sep 23 08:48:12 bear kernel: EXT3-fs: mounted filesystem with ordered data mode.
Sep 23 08:48:36 bear e-smith-bg: Connecting to media/MyData
Sep 23 08:48:37 bear kernel: kjournald starting.  Commit interval 5 seconds
Sep 23 08:48:37 bear kernel: EXT3 FS on sdc1, internal journal
Sep 23 08:48:37 bear kernel: EXT3-fs: mounted filesystem with ordered data mode.
Sep 23 08:48:37 bear e-smith-bg: Expanding /etc/dar2/tester template
Sep 23 08:48:37 bear e-smith-bg: Creating media/MyData/smeserver.myoffice.co.au/myjob/db.zip
Sep 23 08:48:37 bear e-smith-bg: Expiring old backups
Sep 23 08:48:37 bear e-smith-bg: Executing pre-backup event
Sep 23 08:48:37 bear esmith::event[8791]: Processing event: pre-backup 
Sep 23 08:48:37 bear esmith::event[8791]: Running event handler: /etc/e-smith/events/actions/generic_template_expand
Sep 23 08:48:38 bear esmith::event[8791]: expanding /etc/dar/DailyBackup.dcf 
Sep 23 08:48:38 bear esmith::event[8791]: generic_template_expand=action|Event|pre-backup|Action|generic_template_expand|Start|1285224517 942500|End|1285224518 208322|Elapsed|0.265822
Sep 23 08:48:38 bear esmith::event[8791]: Running event handler: /etc/e-smith/events/pre-backup/S10mysql-delete-dumped-tables
Sep 23 08:48:38 bear esmith::event[8791]: S10mysql-delete-dumped-tables=action|Event|pre-backup|Action|S10mysql-delete-dumped-tables|Start|1285224518 208572|End|1285224518 239542|Elapsed|0.03097
Sep 23 08:48:38 bear esmith::event[8791]: Running event handler: /etc/e-smith/events/pre-backup/S20mysql-dump-tables
Sep 23 08:48:38 bear esmith::event[8791]: S20mysql-dump-tables=action|Event|pre-backup|Action|S20mysql-dump-tables|Start|1285224518 239876|End|1285224518 651182|Elapsed|0.411306
Sep 23 08:48:38 bear esmith::event[8791]: Running event handler: /etc/e-smith/events/pre-backup/S30freepbx-dump-astdb
Sep 23 08:48:38 bear esmith::event[8791]: Status: 400
 
Sep 23 08:48:38 bear esmith::event[8791]: Content-type: text/html
 
Sep 23 08:48:38 bear esmith::event[8791]:
 
Sep 23 08:48:38 bear esmith::event[8791]: <b>Security Alert!</b> The PHP CGI cannot be accessed directly. 
Sep 23 08:48:38 bear esmith::event[8791]:   
Sep 23 08:48:38 bear esmith::event[8791]: <p>This PHP CGI binary was compiled with force-cgi-redirect enabled.  This 
Sep 23 08:48:38 bear esmith::event[8791]: means that a page will only be served up if the REDIRECT_STATUS CGI variable is 
Sep 23 08:48:38 bear esmith::event[8791]: set, e.g. via an Apache Action directive.</p> 
Sep 23 08:48:38 bear esmith::event[8791]: <p>For more information as to <i>why</i> this behaviour exists, see the <a href="http://php.net/security.cgi-bin">manual page for CGI security</a>.</p> 
Sep 23 08:48:38 bear esmith::event[8791]: <p>For more information about changing this behaviour or re-enabling this webserver, 
Sep 23 08:48:38 bear esmith::event[8791]: consult the installation file that came with this distribution, or visit   
Sep 23 08:48:38 bear esmith::event[8791]: <a href="http://php.net/install.windows">the manual page</a>.</p> 
Sep 23 08:48:38 bear esmith::event[8791]: S30freepbx-dump-astdb=action|Event|pre-backup|Action|S30freepbx-dump-astdb|Start|1285224518 651553|End|1285224518 687708|Elapsed|0.036155|Status|65280
Sep 23 08:48:38 bear esmith::event[8791]: Running event handler: /etc/e-smith/events/pre-backup/S50rewind-tape
Sep 23 08:48:38 bear esmith::event[8791]: S50rewind-tape=action|Event|pre-backup|Action|S50rewind-tape|Start|1285224518 687978|End|1285224518 792196|Elapsed|0.104218
===============================================================

any ideas ?

[janet]

Bud

In the server manager Disk Archiver panel job setup, the Target should read
/media/MyData
not
media/MyData

[Bud]

mary

thanks for pointing that out

i have changed the backup output to media/MyData

it creates the db.zip file but still NO BACKUP.DAR file
i have use another external USB Drive 8Gig with a new backup job

this is my log file
=============================================

Sep 23 15:44:01 bear kernel: Initializing USB Mass Storage driver...
Sep 23 15:44:01 bear kernel: scsi2 : SCSI emulation for USB Mass Storage devices
Sep 23 15:44:01 bear kernel:   Vendor: Kingston  Model: DataTraveler G2   Rev: 1.00
Sep 23 15:44:01 bear kernel:   Type:   Direct-Access                      ANSI SCSI revision: 02
Sep 23 15:44:01 bear kernel: SCSI device sdc: 15654848 512-byte hdwr sectors (8015 MB)
Sep 23 15:44:01 bear kernel: sdc: Write Protect is off
Sep 23 15:44:01 bear kernel: sdc: Mode Sense: 03 00 00 00
Sep 23 15:44:01 bear kernel: sdc: assuming drive cache: write through
Sep 23 15:44:01 bear kernel: SCSI device sdc: 15654848 512-byte hdwr sectors (8015 MB)
Sep 23 15:44:01 bear kernel: sdc: Write Protect is off
Sep 23 15:44:01 bear kernel: sdc: Mode Sense: 03 00 00 00
Sep 23 15:44:01 bear kernel: sdc: assuming drive cache: write through
Sep 23 15:44:01 bear kernel:  sdc: sdc1
Sep 23 15:44:01 bear scsi.agent[7340]: disk at /devices/pci0000:00/0000:00:1d.7/usb1/1-3/1-3:1.0/host2/target2:0:0/2:0:0:0
Sep 23 15:44:01 bear kernel: Attached scsi removable disk sdc at scsi2, channel 0, id 0, lun 0
Sep 23 15:44:01 bear kernel: USB Mass Storage device found at 2
Sep 23 15:44:01 bear kernel: usbcore: registered new driver usb-storage
Sep 23 15:44:01 bear kernel: USB Mass Storage support registered.
Sep 23 15:44:04 bear fstab-sync[7417]: added mount point /media/MyData for /dev/sdc1
Sep 23 15:44:47 bear e-smith-bg: Connecting to media/MyData
Sep 23 15:44:47 bear kernel: kjournald starting.  Commit interval 5 seconds
Sep 23 15:44:47 bear kernel: EXT3 FS on sdc1, internal journal
Sep 23 15:44:47 bear kernel: EXT3-fs: recovery complete.
Sep 23 15:44:47 bear kernel: EXT3-fs: mounted filesystem with ordered data mode.
Sep 23 15:44:47 bear e-smith-bg: Expanding /etc/dar2/testing template
Sep 23 15:44:47 bear e-smith-bg: Creating media/MyData/smeserver.myoffice.co.au/testing/db.zip
Sep 23 15:44:47 bear e-smith-bg: Expiring old backups
Sep 23 15:44:47 bear e-smith-bg: Executing pre-backup event
Sep 23 15:44:48 bear esmith::event[7469]: Processing event: pre-backup 
Sep 23 15:44:48 bear esmith::event[7469]: Running event handler: /etc/e-smith/events/actions/generic_template_expand
Sep 23 15:44:48 bear esmith::event[7469]: expanding /etc/dar/DailyBackup.dcf 
Sep 23 15:44:48 bear esmith::event[7469]: generic_template_expand=action|Event|pre-backup|Action|generic_template_expand|Start|1285249488 120924|End|1285249488 469177|Elapsed|0.348253
Sep 23 15:44:48 bear esmith::event[7469]: Running event handler: /etc/e-smith/events/pre-backup/S10mysql-delete-dumped-tables
Sep 23 15:44:48 bear esmith::event[7469]: S10mysql-delete-dumped-tables=action|Event|pre-backup|Action|S10mysql-delete-dumped-tables|Start|1285249488 469452|End|1285249488 507567|Elapsed|0.038115
Sep 23 15:44:48 bear esmith::event[7469]: Running event handler: /etc/e-smith/events/pre-backup/S20mysql-dump-tables
Sep 23 15:44:48 bear esmith::event[7469]: S20mysql-dump-tables=action|Event|pre-backup|Action|S20mysql-dump-tables|Start|1285249488 507890|End|1285249488 939928|Elapsed|0.432038
Sep 23 15:44:48 bear esmith::event[7469]: Running event handler: /etc/e-smith/events/pre-backup/S30freepbx-dump-astdb
Sep 23 15:44:49 bear esmith::event[7469]: Status: 400
 
Sep 23 15:44:49 bear esmith::event[7469]: Content-type: text/html
 
Sep 23 15:44:49 bear esmith::event[7469]:
 
Sep 23 15:44:49 bear esmith::event[7469]: <b>Security Alert!</b> The PHP CGI cannot be accessed directly. 
Sep 23 15:44:49 bear esmith::event[7469]:   
Sep 23 15:44:49 bear esmith::event[7469]: <p>This PHP CGI binary was compiled with force-cgi-redirect enabled.  This 
Sep 23 15:44:49 bear esmith::event[7469]: means that a page will only be served up if the REDIRECT_STATUS CGI variable is 
Sep 23 15:44:49 bear esmith::event[7469]: set, e.g. via an Apache Action directive.</p> 
Sep 23 15:44:49 bear esmith::event[7469]: <p>For more information as to <i>why</i> this behaviour exists, see the <a href="http://php.net/security.cgi-bin">manual page for CGI security</a>.</p> 
Sep 23 15:44:49 bear esmith::event[7469]: <p>For more information about changing this behaviour or re-enabling this webserver, 
Sep 23 15:44:49 bear esmith::event[7469]: consult the installation file that came with this distribution, or visit   
Sep 23 15:44:49 bear esmith::event[7469]: <a href="http://php.net/install.windows">the manual page</a>.</p> 
Sep 23 15:44:49 bear esmith::event[7469]: S30freepbx-dump-astdb=action|Event|pre-backup|Action|S30freepbx-dump-astdb|Start|1285249488 940197|End|1285249489 44962|Elapsed|0.104765|Status|65280
Sep 23 15:44:49 bear esmith::event[7469]: Running event handler: /etc/e-smith/events/pre-backup/S50rewind-tape
Sep 23 15:44:49 bear esmith::event[7469]: S50rewind-tape=action|Event|pre-backup|Action|S50rewind-tape|Start|1285249489 45235|End|1285249489 150031|Elapsed|0.104796
Sep 23 15:45:01 bear su(pam_unix)[7510]: session opened for user qmailr by (uid=0)
Sep 23 15:45:07 bear su(pam_unix)[7510]: session closed for user qmailr
=========================================================

i even tried this on another SME Server and All Works (backup and restore) OK, just this server seems to be giving me a problem

i'm at a loss 

[janet]

Bud

Your testing on another server is highly suggestive the problem is NOT  with the DAR2 backup contrib.

Looking at the log file there are some specific messages being displayed that seem to indicate the source of your issue. I am guessing/surmising that the smeserver-php-cgi & freepbx contribs are installed, and at least some directive in php is causing the backup to fail.

I use neither of those so cannot comment more accurately.

If possible try using the default backup tool (e-smith backup with dar) that is in the server manager Backup or Restore panel & select Backup to wokstation, local USB. You can modify the inclusions & exclusions, see the wiki Howto article.

Check to see if those contribs are installed, and I suggest you then remove the php5 cgi contrib to see if the DAR2 backup then runs OK.
The other alternative may be to use SME8beta6 which has php5 in the base install.
It may be beta, but it is matching or bettering the functionality of SME7.5.1.

The other suggestion I have is that you lodge a bug report, I'm not sure what resolution you will obtain in bugzilla, nonetheless it is worth lodging as a stepping stone to fully identifying & resolving this apparent incompatibility issue.

I refer to these log entries in particular:
 
Sep 23 15:44:48 bear esmith::event[7469]: Running event handler: /etc/e-smith/events/pre-backup/S30freepbx-dump-astdb 
........ 
Sep 23 15:44:49 bear esmith::event[7469]: <b>Security Alert!</b> The PHP CGI cannot be accessed directly. 
Sep 23 15:44:49 bear esmith::event[7469]:   
Sep 23 15:44:49 bear esmith::event[7469]: <p>This PHP CGI binary was compiled with force-cgi-redirect enabled.  This 
Sep 23 15:44:49 bear esmith::event[7469]: means that a page will only be served up if the REDIRECT_STATUS CGI variable is 
Sep 23 15:44:49 bear esmith::event[7469]: set, e.g. via an Apache Action directive.</p> 
Sep 23 15:44:49 bear esmith::event[7469]: <p>For more information as to <i>why</i> this behaviour exists, see the <a href="http://php.net/security.cgi-bin">manual page for CGI security</a>.</p> 
Sep 23 15:44:49 bear esmith::event[7469]: <p>For more information about changing this behaviour or re-enabling this webserver, 
Sep 23 15:44:49 bear esmith::event[7469]: consult the installation file that came with this distribution, or visit   
Sep 23 15:44:49 bear esmith::event[7469]: <a href="http://php.net/install.windows">the manual page</a>.</p> 
Sep 23 15:44:49 bear esmith::event[7469]: S30freepbx-dump-astdb=action|Event|pre-backup|Action|S30freepbx-dump-astdb|Start|1285249488 940197|End|1285249489 44962|Elapsed|0.104765|Status|65280

[Bud]

mary

thank you so much for helping, much appreciated

this is what i did

1. yum remove smeserver-freepbx freepbx-src   (Removed FreePBX from the SME Server)
2. did a cleanup of freepbx via following script

==================================================
     #!/bin/bash

clear
echo "----------------------------"
echo "!!!!!     WARNING     !!!!!"
echo "----------------------------"
echo ""
echo "This script will remove from your server:"
echo "     - freepbx and asterisk cdr MySQL databases"
echo "     - freepbx MySQL User"
echo "     - freepbx DB entries (freepbx, httpd-fpbx and dahdi)"
echo "     - /opt/freepbx"
echo ""
echo -n "Are you sure you want to remove FreePBX permanently ? (y/n) [n] "
read confirm
if [ "$confirm" = "y" -o "$confirm" = "Y" ]; then
       echo "Droping MySQL databases..."
       DBNAME=$(/sbin/e-smith/db configuration getprop freepbx DbName)
       CDRDBNAME=$(/sbin/e-smith/db configuration getprop freepbx CdrDbName)
       mysql -e "DROP DATABASE $DBNAME"
       mysql -e "DROP DATABASE $CDRDBNAME"
       echo "Deleting MySQL User..."
       DBUSER=$(/sbin/e-smith/db configuration getprop freepbx DbUser)
       mysql -u root -e "REVOKE ALL PRIVILEGES ON *.* FROM '$DBUSER'@'localhost';"
       mysql -u root -e "DROP USER '$DBUSER'@'localhost';" > /dev/null 2>&1
       echo "Removing SME DB entries..."
       /sbin/e-smith/db configuration delete freepbx
       /sbin/e-smith/db configuration delete httpd-fpbx
       /sbin/e-smith/db configuration delete dahdi
       echo "Removing /opt/freepbx ..."
       rm -rf /opt/freepbx
       echo "Done!"
fi
=========================================================

3. signal-event post-upgrade; signal-event reboot

After the SME Server Reboot I checked my USB Backup Drive, made sure that it was mounted and then Manually did a Backup

Backup WORKED and went Fine without any Problems 

Now I need to get FreePBX to work

will keep you posted

mary thanks again for showing me how to do fault finding regarding the " Disk ARchive " Contrib 

[Bud]

sme team

Questions abour Disk ARchive

1. i need to be able to send the Disk ARchive backup status report via email to two users (manager and director) to show what backups have been done and details regarding the backups. how do i go about doing this?
2. i also need to include the MySQL Databases of the sme server. what folder(s) do i include for database backup in the Disk ARchive contrib backup options?
3. i need to backup folders of 5 x windowz xp Computers (my documents folder, desktop etc.). these computers have NO shares assigned, but i assume you could use the \\xp_ip\c$ option. how do i include this in the Disk ARchive backup option to do the remote windows xp backup?

any help greatly appreciated 

[janet]

Bud

1. i need to be able to send the Disk ARchive backup status report via email to two users (manager and director) to show what backups have been done and details regarding the backups. how do i go about doing this?

Send admin mail to both users and set procmail or maildrop filters to only receive/allow mail from dar2 reports. See the dungog.net site for details at http://www.dungog.net/wiki/Usermanager#Mailsorting

2. i also need to include the MySQL Databases of the sme server. what folder(s) do i include for database backup in the Disk ARchive contrib backup options?

A dump of msyql dbs is included in all DAR2 backup jobs. It is done at the start of the backup.
Look in your backup in /home/e-smith/db/mysql/ IIRC.

3. i need to backup folders of 5 x windowz xp Computers (my documents folder, desktop etc.). these computers have NO shares assigned, but i assume you could use the \\xp_ip\c$ option. how do i include this in the Disk ARchive backup option to do the remote windows xp backup?

There are probably a few ways to do this.
1) make a share of the required folder(s) to backup, then on the sme server mount the windows share to a location that is included in the backup job

2) use roaming profiles and point all user data to the users home folder, and include /home/e-smith/files/users/* and/or /home/e-smith/files/users/username in the backup job either globally or on a per user basis.

3) automate backups from each workstation to the server using Windows backup or other tools, to a location on the server that is included in the backup job

4) use backuppc contrib http://wiki.contribs.org/BackupPC

5) Search the forums as most if not all of this has been answered before.