Topic: backups

[scree74]

Hi

Can some one help me we have over 500gb of data. when I a run a workstation backup it timing out,and there for I never get a complete full backup. I recently read a blog that if you set the time out for 23 hours then it will start the next cron job and complete the full backup, I tried this with little success.

can anyone suggest a different approach to this issue.

Also. How can I kill a backup job that’s already started?

[janet]

scree74

If your system cannot complete a full backup in 24 hours, then clearly you cannot expect to run/do a full backup every day.
The only option then is to run daily incremental backups, which will add only the changes in data & configuration etc to the original first full backup.

To get the first full backup to complete, then you need to disable the option for backup timeout, and run the backup on say Friday night & then allow it to run all day Saturday & Sunday (when the system is ideally not being used) if necessary until it does complete. Unless you have a really slow system then 2 days should be more than sufficient time.
You may need to disable the scheduled backup so it does not try to run again on Sat & Sun, and instigate the first backup manually using the command line.
Appropriate commands were posted in these forums not so long ago, so search & do a bit of back reading, or simply look in the server manager messages log file to see what happens when a backup is automatically started (ie indentify the command being used).

Are you backing up to a locally (server) connected USB drive or to a network workstation or drive ?
You may have slow network issues.

[scree74]

Hi Janet

Thanks for taking the time to reply its much appreciated. I am using the backup and restore option from the admin panel. Reading about this, the Full backup has a time limit of 24 built in so even if I remove the the tick  the backup times oat 24 hours. I read another blog that said to set it to time out a 23 hours and the next backup should complete the full backup job but i found that it doesn't move the dar files from the tmp folder to the set0 folder and when the next backup starts it clears the tmp folder and Ive lost all the original dar files.

I think I am going to have a play around with dar2 as you can set the slices to 2gig and see if that works.

Cheers
Scott

[mmccarn]

You may want to look into Affa.

It should still work, even though the developer has moved on from SME to Centos, with the current version of Affa maintained at Sourceforge.

Affa uses rsync and hardlinks to maintain local or remote full backups while minimizing network traffic.

I used to use Affa to maintain remote backups of hundreds of gigabytes of data using a backup window of an hour or less.

[janet]

scree74

Are you using sme 8 ?
Have you run all updates, ie yum update ?

I am using the backup and restore option from the admin panel.

Please be more specific.
Are you backing up to desktop, or a workstation or to a locally connected USB drive or some other system or device  ?

What settings are you using eg particularly for compression, but please give all settings.
I would have expected a backup of 500Gb to complete in 24 hours.
Please give details of your server hardware including processor & RAM.
Something does not seem right (to me), so I wonder about your device or system or network speed.

Please describe your network & what/where you are backing up to.

Let's try to resolve & fix why the backup is taking so long, before you go off using other (unsupported) backup solutions.
Using the DAR2 contrib you will probably have the same timing issues, the built in backup uses dar also.
The 2gig issue you refer to is an old issue, no longer applicable on most systems & browsers etc, so be careful when reading old posts.
There is a possibility that the DAR2 contrib does not function correctly with sme 8 (the issue may be related to doing a backup & restore) refer to recent forum post, & is also no longer maintained, so I would advise you not to use that contrib.
It has really been replaced by the built in e-smith dar backup for routine daily backups, which is actively maintained being part of the base release of sme server 8.

[janet]

scree74

.....so even if I remove the the tick  the backup times out 24 hours

Removing the tick is incorrect.
You need to tick the box that says
Don't timeout full backup sessions    

I think the trick is to allow a scheduled backup to start (say on Friday night at 8pm) and then after the backup has started, go into server manager (at 8:10pm) & disable the backup.
That way the backup program will NOT try to run another backup in 24 hours time (at 8pm on Saturday) which would as you say cancel the running backup & delete all the files on the backup disk tmp location.

This will allow the full backup job to complete, however long it takes whether 30 hrs, 36 hrs, 48 hrs or even 72 hrs (because you have told the system to NOT time out full backups).

When, in a day or two or three, you have a first full backup complete and you receive the email message to admin advising of a successful completion, you can re-enable the backup job.
Then I suggest you change the settings to daily incrementals for whatever time period you feel comfortable eg you can have 1 backup job with 365 daily incrementals, that way you only do a full backup once a year. In your case the cycle you chose will be governed by how often you want to perform this manual procedure to get a full backup completed.
 
If the daily incrementals do not complete in 24 hrs then they will be automatically postponed, & should recommence from the same point on the next scheduled backup.

I would imagine though that most incrementals will only need to backup a small amount of changed data, so the backup time, should be relatively quick eg a few minutes or an hour or 2. It really depends on how much data changes on your server every day & how big the changed files are.
Also if you use a high(est) compression rate, then your processor may be taking a long time to compress the data, & that could be the reason your full backups are taking so long (over 24 hours). In that case use a medium or lower level of compression setting.

That's why in my earlier post I suggested you should find out what is causing these long backup times & fix it, as it does not sound right to me.


PS I just thought of another reason why the files in your backup tmp location are being deleted.
It may be that the backup disk does not have enough space on it to save the tmp files and copy them to the main backup folder.
In that case the backup will fail, leaving the stranded files in the tmp location.
The next time the backup runs, the first thing it will do is delete the tmp files & then start again with a new backup.
The backup location needs to have more than twice the resultant backup data file size available.

[janet]

scree74

How can I kill a backup job that’s already started?

Please learn to use google for these type of "generic" requests eg
Google the phrase
kill a linux process

http://www.cyberciti.biz/faq/kill-process-in-linux-or-terminate-a-process-in-unix-or-linux-systems/