Topic: smeserver-dar2 dmay - old backups isn't expiring

[greatty]

Hi i've just installed smeserver 7.1 on a new machine with smeserver-dar2 contrib by dmay. Everything works well, but i have a problem about expiring old backups. I have set 5 backup sets in the web interface, daily backup on 1 am and that's what i received today from cron-daemon:

Listing backup archives on target:
total 134690744
-rw-r--r--  1 root root 2097152000 Feb 23 18:25 2007.02.23.1.dar
-rw-r--r--  1 root root  893840935 Feb 23 21:12 2007.02.23.10.dar
-rw-r--r--  1 root root 2097152000 Feb 23 18:28 2007.02.23.2.dar
-rw-r--r--  1 root root 2097152000 Feb 23 18:41 2007.02.23.3.dar
-rw-r--r--  1 root root 2097152000 Feb 23 18:43 2007.02.23.4.dar
-rw-r--r--  1 root root 2097152000 Feb 23 18:45 2007.02.23.5.dar
-rw-r--r--  1 root root 2097152000 Feb 23 18:46 2007.02.23.6.dar
-rw-r--r--  1 root root 2097152000 Feb 23 19:26 2007.02.23.7.dar
-rw-r--r--  1 root root 2097152000 Feb 23 20:17 2007.02.23.8.dar
-rw-r--r--  1 root root 2097152000 Feb 23 20:55 2007.02.23.9.dar
-rw-r--r--  1 root root 2097152000 Feb 24 01:04 2007.02.24.1.dar
-rw-r--r--  1 root root  893833470 Feb 24 03:51 2007.02.24.10.dar
-rw-r--r--  1 root root 2097152000 Feb 24 01:07 2007.02.24.2.dar
-rw-r--r--  1 root root 2097152000 Feb 24 01:20 2007.02.24.3.dar
-rw-r--r--  1 root root 2097152000 Feb 24 01:22 2007.02.24.4.dar
-rw-r--r--  1 root root 2097152000 Feb 24 01:23 2007.02.24.5.dar
-rw-r--r--  1 root root 2097152000 Feb 24 01:25 2007.02.24.6.dar
-rw-r--r--  1 root root 2097152000 Feb 24 02:05 2007.02.24.7.dar
-rw-r--r--  1 root root 2097152000 Feb 24 02:56 2007.02.24.8.dar
-rw-r--r--  1 root root 2097152000 Feb 24 03:34 2007.02.24.9.dar
-rw-r--r--  1 root root 2097152000 Feb 25 01:04 2007.02.25.1.dar
-rw-r--r--  1 root root  893832745 Feb 25 03:51 2007.02.25.10.dar
-rw-r--r--  1 root root 2097152000 Feb 25 01:07 2007.02.25.2.dar
-rw-r--r--  1 root root 2097152000 Feb 25 01:20 2007.02.25.3.dar
-rw-r--r--  1 root root 2097152000 Feb 25 01:22 2007.02.25.4.dar
-rw-r--r--  1 root root 2097152000 Feb 25 01:23 2007.02.25.5.dar
-rw-r--r--  1 root root 2097152000 Feb 25 01:25 2007.02.25.6.dar
-rw-r--r--  1 root root 2097152000 Feb 25 02:04 2007.02.25.7.dar
-rw-r--r--  1 root root 2097152000 Feb 25 02:56 2007.02.25.8.dar
-rw-r--r--  1 root root 2097152000 Feb 25 03:34 2007.02.25.9.dar
-rw-r--r--  1 root root 2097152000 Feb 26 01:04 2007.02.26.1.dar
-rw-r--r--  1 root root  893833467 Feb 26 03:50 2007.02.26.10.dar
-rw-r--r--  1 root root 2097152000 Feb 26 01:07 2007.02.26.2.dar
-rw-r--r--  1 root root 2097152000 Feb 26 01:20 2007.02.26.3.dar
-rw-r--r--  1 root root 2097152000 Feb 26 01:22 2007.02.26.4.dar
-rw-r--r--  1 root root 2097152000 Feb 26 01:23 2007.02.26.5.dar
-rw-r--r--  1 root root 2097152000 Feb 26 01:24 2007.02.26.6.dar
-rw-r--r--  1 root root 2097152000 Feb 26 02:04 2007.02.26.7.dar
-rw-r--r--  1 root root 2097152000 Feb 26 02:55 2007.02.26.8.dar
-rw-r--r--  1 root root 2097152000 Feb 26 03:34 2007.02.26.9.dar
-rw-r--r--  1 root root 2097152000 Feb 27 01:04 2007.02.27.1.dar
-rw-r--r--  1 root root  692871608 Feb 27 03:48 2007.02.27.10.dar
-rw-r--r--  1 root root 2097152000 Feb 27 01:07 2007.02.27.2.dar
-rw-r--r--  1 root root 2097152000 Feb 27 01:20 2007.02.27.3.dar
-rw-r--r--  1 root root 2097152000 Feb 27 01:22 2007.02.27.4.dar
-rw-r--r--  1 root root 2097152000 Feb 27 01:23 2007.02.27.5.dar
-rw-r--r--  1 root root 2097152000 Feb 27 01:25 2007.02.27.6.dar
-rw-r--r--  1 root root 2097152000 Feb 27 02:04 2007.02.27.7.dar
-rw-r--r--  1 root root 2097152000 Feb 27 02:56 2007.02.27.8.dar
-rw-r--r--  1 root root 2097152000 Feb 27 03:34 2007.02.27.9.dar
-rw-r--r--  1 root root 2097152000 Feb 28 01:04 2007.02.28.1.dar
-rw-r--r--  1 root root  696712498 Feb 28 03:48 2007.02.28.10.dar
-rw-r--r--  1 root root 2097152000 Feb 28 01:07 2007.02.28.2.dar
-rw-r--r--  1 root root 2097152000 Feb 28 01:20 2007.02.28.3.dar
-rw-r--r--  1 root root 2097152000 Feb 28 01:22 2007.02.28.4.dar
-rw-r--r--  1 root root 2097152000 Feb 28 01:23 2007.02.28.5.dar
-rw-r--r--  1 root root 2097152000 Feb 28 01:24 2007.02.28.6.dar
-rw-r--r--  1 root root 2097152000 Feb 28 02:04 2007.02.28.7.dar
-rw-r--r--  1 root root 2097152000 Feb 28 02:56 2007.02.28.8.dar
-rw-r--r--  1 root root 2097152000 Feb 28 03:34 2007.02.28.9.dar
-rw-r--r--  1 root root 2097152000 Mar  1 01:04 2007.03.01.1.dar
-rw-r--r--  1 root root  702938099 Mar  1 03:48 2007.03.01.10.dar
-rw-r--r--  1 root root 2097152000 Mar  1 01:07 2007.03.01.2.dar
-rw-r--r--  1 root root 2097152000 Mar  1 01:20 2007.03.01.3.dar
-rw-r--r--  1 root root 2097152000 Mar  1 01:22 2007.03.01.4.dar
-rw-r--r--  1 root root 2097152000 Mar  1 01:23 2007.03.01.5.dar
-rw-r--r--  1 root root 2097152000 Mar  1 01:25 2007.03.01.6.dar
-rw-r--r--  1 root root 2097152000 Mar  1 02:04 2007.03.01.7.dar
-rw-r--r--  1 root root 2097152000 Mar  1 02:56 2007.03.01.8.dar
-rw-r--r--  1 root root 2097152000 Mar  1 03:34 2007.03.01.9.dar

Total target disk space usage:
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             230G  129G   90G  60% /media/usbdisk

Executing post-backup event
Dismounting /media/usbdisk

Now, i readed about backup sets works in this contrib and seems that dar2 calculate one set every 24 hours, in this case last backup has started on 1st march 1am so, if i understood how it works, 5 sets backup expiring on 24th feb 1 am, right?
Why on my backup disk 23th feb backup is still there?

I wanna say that 23th feb backup was a manual backup taken just after the installation of this contrib. Maybe backups that aren't involved in the same job doesn't count in the number of sets of the backup job?

Thanx to dmay for his powerful contrib and all contrib's forum who gave me a lot of hints to set up my server.

Max from Rome  

[wjhobbs]

I expected dar2 to retain 'n' days worth of backups if I specified 'n' on the panel.

However, what I am experiencing consistently on several servers is that dar2 retains 'n +2' days of backups.

What I want is 3 days so I have specified '1' in the panel.

It's strange.

John

[dmay]

Try making a simple change dropping a '+' sign in front of the $EXPIRY variable in /etc/e-smith/events/actions/dar2-backup as follows:

Original code:

/usr/bin/find $MOUNT/$ID/$DCF/*.dar -type f -mtime +$EXPIRY -exec /bin/rm -f {} \; > /dev/null 2>&1

New code:

/usr/bin/find $MOUNT/$ID/$DCF/*.dar -type f -mtime $EXPIRY -exec /bin/rm -f {} \; > /dev/null 2>&1

Report your results.

Darrell

[greatty]

I will try next week with your tip darrel

I have to confirm what john said, cause today 23th backup is finally expired.

Thanx to all for your replies

[wjhobbs]

Hi Darrell,

I tried the script change you suggested and got an unexpected result. I will keep the change in place to see what happens; but for now let me explain what I observed.

My normal setup is to do a daily backup at 01:15hrs. I had specified '1' as the retention value. As of yesterday (March 5) I had three days worth of backups on the drive, dated March 3, 4 and 5.

Last night (March 5) I applied your change and the backup job ran this morning (01:15 on March 6). What is now reported on the drive according to the dar2 email is backups for March 3, 5 and 6.

I will keep the current settings and see what happens tomorrow. I'll let you know.

John

[wjhobbs]

As promised, here is an update.

With the '+' removed from the script what seems to be happening is that dar2 is removing the specific backup set for 'n+2' days ago. It is ignoring anything older (which is a problem for anything other than daily backups).

With the retention set to '1' I now have backup sets for today (March 7), yesterday (March 6) and the old backup for March 3.

With the '+' in the script it seems to be removing any sets *greater than* 'n+2' days ago. What would be ideal is for it to remove sets *greater than or equal to* 'n+2' days ago.

Hope this helps.

John

[dmay]

# man find

Numeric arguments can be specified as:

Code: [Select]

-mtime -n   = File's data was last modified less then n*24 hours ago.
-mtime n    = File's data was last modified n*24 hours ago.
-mtime +n   = File's data was last modified greater then n*24 hours ago.
Try all three arguments and see which one works best. Default I chose to use was +n.

Darrell

[greatty]

Seems that the problem source is not in the script but in the behavior. Since we want to delete old backups greater than n*24 hours we have to use "+" option in the script. The reason about it works like "n+2" has to be searched in the behavior, when i have to delete backup files older than n*24 hours i usually consider time in a continuos way, so if i set 'n' to 1 i should delete all files older than n*24 starting from the time when backup starts, so if backup is starting at 1 am i should delete all backups modified before yesterday at 1 am, in this case i'll keep yesterday set because files was created after 1 am but, with continuos time, i have to delete the day before yesterday set for the same reason. But script doesn't work like this.

In my opinion we have to consider time in a discrete way, let us think about time blocks of 24 hours each, in this case we should delete all backup sets whose creation time is not included in the 1st,2nd,3rd,...'n'th block. So with only 1 set we have to consider 1 block, let me enumerate blocks starting from 0. The 0 block starting from 1am yesterday to 1 am today, the 1st and then the last block of my sets starting from 1 am the day before yesterday to 1 am yesterday, because n=1 the script keeps a total of 3 backup sets corresponding to: 1)today backup, 2)backup belonging to the 0 block 3)backup belonging to the 1st block.

Imho, this is the reason cause script works in a "strange" way keeping 'n+2' instead of 'n' backup sets.