Topic: Backup takes more than 24 hours

[michelandre]

Hi All,

I have more than 1,2 TB to backup on a SME-8. I used server-manager to backup to local workstation. After 24 hours the backup stops.

I looked all around to find a solution but no luck.

I saw:
http://wiki.contribs.org/Backup_with_dar
which says:
"To workaround set timeout to 23 hrs, this allows the backup to timeout cleanly, it will restart and attempt to complete the first full backup on the next run."

I don't quite understand "on the next run".
Does it mean schedule the first backup with a timeout of 23 hours and then schedule another one for 23:10 hours later?
Also, if I use a few 12 hours time-out periods, will it include the changed files?

I also saw: http://wiki.contribs.org/Backup_SME_Server_Remotely_Using_cURL
This uses "backup" and I do not know if it generates a .dar file as it says the backup file is tar.gz so maybe I will not be able to use restore from the server-manager??? Is restore able to take a tar.gz???

I do not want to try anything until certain as this server is very critical.
I can not add another disk on the server for backing up to local and then scp to workstation.

Any help will be appreciated,

Michel-Andre

[TerryF]

What are your backup settings..ie when have you configured a full backup, how often is it done and are you also setting incrementals for that backup set ?

[janet]

michelandre

"To workaround set timeout to 23 hrs, this allows the backup to timeout cleanly, it will restart and attempt to complete the first full backup on the next run."
I don't quite understand "on the next run".

It means on the next automatic scheduled run, as set in server manager panel.
ie if backup is set to "run" (or perform the backup) at 2 am, then it will time out at approx 1 am the next day. Then at 2am a scheduled backup will commence. The system sees that an interrupted backup exists & will recommence the original backup, from the point it was up to, until it finishes.
It may take 2 or 3 days even until it is finished.

With that amount of data, you really should backup to a USB drive attached to the server.
Backing up across a network to a workstation has many "bottlenecks" which can collectively cause a backup to take an excessively long time to complete.

Also for practical reasons you should do one full backup & then daily incrementals. The first backup may take a while but subsequent backups should be much faster depending how much data changes on the server each day.

I do not want to try anything until certain as this server is very critical.

If it is very critical, then for sure I would stop playing around with backup to workstation (which can be very unreliable due to other network issues etc), and connect a 2Tb USB drive to the server & start backing up to that. Swap the disk nightly too, if it is that critical.
There is more to say on this if you are interested.

I can not add another disk on the server for backing up to local.....

Why not ?

.... and then scp to workstation.

Why would you even think about scp'ing 1.2Tb of data across the network, it will take a long time.

[Stefano]

in my experience you shoul use NFS and not samba/cifs.. NFS is about 3 times faster, YMMV

I mean.. on the same hw, same network, ALL the same, backup to a NFS share is (I repeat, my experience) 3 times faster than samba..
FYI, the backup target was a freenas NAS

[michelandre]

Thank you all.

My friend and I are testing all your advices.
We will tell you the outcomes of our tests.

As always this forum has all the answers,

Michel-André

[michelandre]

Hi All,

Here are my observations running "top -d 1": YMMV

2 virtual SME-8, fully updated, in a VMware Workstartion 7.1.6 running on a SLES-10.4
Backup size: 5.2 GB of SME-8 server.

***** NFS
- about 9.6 MB/sec
- DAR: average: 18-42% // max peak around 78% (one time only)
- rpciod/0: max peak around 21% ("top looked like frozen for 2-3 sec)
- 716 MB/file with date and time in name

***** CIFS
- about 7.2 MB/sec
- DAR: average 19-59%  // lot of peaks at 65%
- pdflush: around 14%
- cifsd: 21%
- 716 MB/file with date and time in name

***** USB
- On a physical SME-8  //  120GB drive in  usb-2 casing
- about 6.1 MB/sec
- lot of peaks at 100%
- One file only named "smeserver.tgz". No date or time appended.

Michel-André

[lloydh]

I do my backups through Server-Manager to a USB disk and it takes 2.5 hours to backup 140GB, my MB is a 3 year old Gigabyte GA880GM-USB3 and the USB disk is a WD Elements 2TB.     You should be able to backup your 1.2TB in less than 23 hours to USB.

Hope that helps.