Koozali.org: home of the SME Server

Offsite Backup of 2 million files using Affa

Offline jankowskid

  • *
  • 7
  • +0/-0
Offsite Backup of 2 million files using Affa
« on: November 23, 2009, 05:37:54 PM »
Background:
Total Data: 150GB
Average data change per day: @2 GB
Total number of files: @2 million
Keep:  Only Keeping One scheduled

Setup:
SME Source Server <--> FW <==> Internet <==> FW <--> SME Backup Server <--> NAS Storage

Problem:  The affa job spends all of it's time simply recreating the hardlinks to the last scheduled job's files.
Possible Solution:  Pre-hard-link scheduled.running to scheduled.0 using a preJobCommand (cp -al)

Questions:
1. At what point in affa's process does the prejobCommand execute?
2. Does affa connect the destination nfs share prior to the preJobCommand execution? 
3. Does the preJobCommand have to be a perl script, or can it be bash?
4. Are there any other ways that i might speed the processes up?

Notes:
I've done the process manually and it seems faster.  I had to remove the --inplace option from the job as well or rsync wouldn't remove the initial hard-link, it would just update the file (in both backup sets because of the hardlink) "in-place" as the option states.  Any suggestions are welcome.