Topic: Strange cron problem

[toothandnail]

I've hit a very odd problem, which appears to be cron, and have no idea what the cause is, or for that matter, how to fix it.

SME 7.5.1, running on an AMD Athlon machine. The server is connected to a network which runs POS using some horrible Windows software called Easitill. This server is purely a backup server, has no internet connection and spends most of its time idle.

It runs three backups. One (very small) one uses Synbak to backup the server setup itself. The other two backup data from a Windows server. There is a daily backup, which runs on a 28 day cycle, plus a monthly backup which will run on a 36 month cycle. The latter two are run by very similar scripts, though the daily has to count and increment the day count to locate where the backup archives should be placed.

The backups are executed by the following template fragment:

Code: [Select]

30 0 * * * root /root/bin/tarback
15 1 * * * root /root/bin/doback
30 3 1 * * root /root/bin/monthly
The fragment expands quite correctly, and all three events are visible in /etc/crontab.

The problem I have is that the monthly backup is never executed. Looking at the logs, I can find both tarback and doback being executed at the correct time every day, but, on the first of each month, there is no sign that the monthly script is executed at all.

I use very similar template fragments with a number of other SME servers, and have hit no problems with them, so I'm at a loss to explain why the monthly event doesn't work on this machine. I'm hoping I'm missing something obvious, but so far, I'm not seeing it at all. To make matters worse, because the network that this server is part of has no internet connection, I have to physically visit it to find out what is going on...

Anyone got any ideas as to what stupidity I've managed to inflict on myself?

Paul.

[TerryF]

The cron jobs look fine..you would think it has to be further along the process..what are the doback and monthly contents..

[cactus]

Did you check /var/log/cron* files to see it really is not kicked off? I also suspect there might be an error in your script, although that should normally be reported to the admin mailbox. Did you already check the admin mailbox?

[toothandnail]

The cron jobs look fine..you would think it has to be further along the process..what are the doback and monthly contents..

Should have mentioned - I've had no problems running the monthly script manually, either directly or using a detached screen session. The doback is a similar script, and runs fine.

Contents of the monthly script is as below:

Code: [Select]

#!/bin/bash
#Backup is performed to a 1 TB drive, mounted in a caddy. First mount the disk..
mount -t ext3 /dev/sdb1 /var/backup
sleep 20
echo 'Backup drive mounted'

#----------------------------------------------------------------------
# Mount server resouces, create backup, umount, sequential.
# Set 'now' variable to allow correct dating of backup archives
# Set 'ym' variable to 'year-month', change to monthly backup directory,
# create 'year-month' directory and change into it.
#----------------------------------------------------------------------

now=`date +%G%m`
ym=`date +%G-%m`

cd /var/backup/monthly
mkdir $ym
cd $ym

for i in easiback Sage accounts Amanda Tracy Pat database ProgramData
    do
mount -t cifs //192.168.100.254/$i /mnt/backup -o user=xxxxx%yyyyy
    sleep 20
    tar czf $i-$now.tgz /mnt/backup
umount /mnt/backup; sleep 60; echo "$i completed"
    done

cd /root
sleep 60
umount /var/backup
echo 'Backup drive unmounted'

exit 0
The daily script (doback) is very similar, just keeps a tally of days 1 - 28 and increments the day count on each start. It runs without any problems....

Paul.

[cactus]

Should have mentioned - I've had no problems running the monthly script manually, either directly or using a detached screen session. The doback is a similar script, and runs fine.

Running manually might be very different from running as a cron script as for instance the path settings are much stricter for security than the normal user environment.

The daily script (doback) is very similar, just keeps a tally of days 1 - 28 and increments the day count on each start. It runs without any problems....

Just stating it is very similar does not help us much in doing diagnosis, perhaps it is caused by just that tiny difference.

Did you already check my earlier suggestion?

Did you check /var/log/cron* files to see it really is not kicked off? I also suspect there might be an error in your script, although that should normally be reported to the admin mailbox. Did you already check the admin mailbox?

[toothandnail]

Did you check /var/log/cron* files to see it really is not kicked off? I also suspect there might be an error in your script, although that should normally be reported to the admin mailbox. Did you already check the admin mailbox?

I've checked /var/log/cron logs to see if I could work out what is going on. In the logs, there is a clear indication of the two daily backups starting, but nothing at all for the monthly. Also had a look at the admin mailbox and don't see any errors reported there either. It just seems that the cron event is being ignored.

I suspected an error in the scripts, but, as already mentioned, the daily backup script (which is largely similar) runs without errors and the monthly script can be run from an SSH login, also without errors.

One possibility that I wondered about was power outages. The system is in a relatively rural area, and there have been a number of power cuts. But this has happened over several months, and the cron logs don't indicate any power outages that I can see....

Paul.

[cactus]

I've checked /var/log/cron logs to see if I could work out what is going on. In the logs, there is a clear indication of the two daily backups starting, but nothing at all for the monthly. Also had a look at the admin mailbox and don't see any errors reported there either. It just seems that the cron event is being ignored.

Did you also check to make sure the first off the month is in there?

I suspected an error in the scripts

That should show up in logs and admin mailbox IIRC.

but, as already mentioned, the daily backup script (which is largely similar) runs without errors and the monthly script can be run from an SSH login, also without errors.

Please stop argumenting that it is largely similar, the error might be in the differences and could be as simple as not setting a full path to a file outside the cron home environment. We are trying to help you but you are making it hard.

One possibility that I wondered about was power outages. The system is in a relatively rural area, and there have been a number of power cuts. But this has happened over several months, and the cron logs don't indicate any power outages that I can see....

I find that hard to believe, cron logs only log con related stuff. They are no general logfile and will not contain any events or messages related to power outages, reboots and the like.

[toothandnail]

Running manually might be very different from running as a cron script as for instance the path settings are much stricter for security than the normal user environment.
Just stating it is very similar does not help us much in doing diagnosis, perhaps it is caused by just that tiny difference.

True. As requested, the daily (doback) script:

Code: [Select]

#!/bin/bash
# Backup script. Backup MS shares on a 28 day cycle.

mount -t ext3 /dev/sdb1 /var/backup
sleep 10
echo 'Backup drive mounted'

# Check for position in the backup cycle, initialise if necessary

if [ ! -e /root/backups/last_back ]; then
        echo "0"> /root/backups/last_back
fi

# Done. Check for position in cycle

read dayno </root/backups/last_back

# Position verified, cleanup old files and proceed to backup
rm -rf /var/backup/daily/day${dayno}/*
cd /var/backup/daily/day${dayno}

#----------------------------------------------------------------------
# Mount server resources, create backup, unmount, sequential.
# Set 'now' variable to allow correct dating of backup archives
#----------------------------------------------------------------------

now=`date +%G%m%e`

for i in easiback Sage accounts Amanda Tracy Pat database ProgramData
    do
mount -t cifs //192.168.100.254/$i /mnt/backup -o user=xxxx%yyy
    sleep 20
    tar czf ${i}-${now}.tgz /mnt/backup
umount /mnt/backup; sleep 30; echo "$i completed"
    done

if [[ $dayno = 28 ]]; then
    dayno=0;
fi
cd /root
dayno=$(($dayno + 1))
echo $dayno >/root/backups/last_back
# backdir="/var/backup/day"$dayno
sleep 60
umount /var/backup
echo 'Backup drive unmounted'
exit 0

Did you already check my earlier suggestion?

Yes, no sign of the monthly job being started, though I see indications of the two daily backups and other cron events in the logs.

Paul.

[toothandnail]

Did you also check to make sure the first off the month is in there?

There are indications of other jobs starting on the first of the month.

That should show up in logs and admin mailbox IIRC.

I would have expected that also.

Please stop argumenting that it is largely similar, the error might be in the differences and could be as simple as not setting a full path to a file outside the cron home environment. We are trying to help you but you are making it hard.

Sorry, didn't mean to. See my last message for the daily script.

I find that hard to believe, cron logs only log con related stuff. They are no general logfile and will not contain any events or messages related to power outages, reboots and the like.

Sorry, badly expressed. What I should have said is that there are indications of events either side of the time the monthly backup should have run, so a power outage seems unlikely. While the server is set to restart after a power failure, and has a UPS connected, I would expect a longer interval between reported events than I'm seeing if there had been an outage.

Paul.

[jphilip]

I may not be remembering correctly but I seem to recall a similar problem with cron jobs I had a few months ago was solved by making sure there was a carriage return after the last entry in the template fragment.

[toothandnail]

I may not be remembering correctly but I seem to recall a similar problem with cron jobs I had a few months ago was solved by making sure there was a carriage return after the last entry in the template fragment.

  Funny you should say that. I spent some time going through the crontab and template fragment, and comparing them to other, similar ones from other systems. And that is the only difference that I've been able to spot - all the others have a newline at the end of the file, but the problem system doesn't - there is no newline at the end of the template fragment, and there is no newline at the end of the crontab either.

I'll be testing that sometime next week - will post back if that solves the problem. I'm keeping my fingers crossed.....

Paul.

[toothandnail]

As a followup to my original post, I've now solved the problem. Just took a while to test it....

The original template fragment for the cron event to run the monthly backup did not end in a newline. All I ended up doing was adding a newline to the template fragement, expanding it again, and restarting cron. Got back to the site this morning, and found that the cron event had run on time and the monthly backup had been made.

I'll report a bug on this, can't do it from here though.

Paul.