Koozali.org: home of the SME Server

Legacy Forums => General Discussion (Legacy) => Topic started by: Samer Pharaon on November 05, 2003, 10:18:29 AM

Title: Squid Log files - automatic conversion
Post by: Samer Pharaon on November 05, 2003, 10:18:29 AM
Hi,

Is there any way to automatically run a script every day at midnight to have the squid log files converted into a normal time and date and then stored in a special folder.

I usually do this manually by typing the following order:

/var/log/squid/access.log | /timeconvert.pl > /var/log/squid/access-11-05-2003.log

timeconvert.pl reads as follows:
#! /usr/bin/perl -p
s/^\d+\.\d+/localtime $&/e;

What I need is an automatic way to do this everyday at midnight. Any ideas?

Thanks for your help.

Samer
Title: Re: Squid Log files - automatic conversion
Post by: Nathan Fowler on November 05, 2003, 07:36:43 PM
You could create a perl script in /etc/cron.daily, I think that runs at 2:00 AM (Haven't verified though).

pico /etc/cron.daily/squidlogconvert

[add]
#!/bin/sh
/var/log/squid/access.log | /timeconvert.pl > /var/log/squid/access-11-05-2003.log

[save]
chmod +x squidlogconvert

You probably need to substitute the 11-05-2003 with the [today], you can do this programatically by invoking /sbin/date I believe.
Title: Re: Squid Log files - automatic conversion
Post by: jeroen on November 06, 2003, 12:58:26 AM
Nathan Fowler wrote:
>
> You could create a perl script in /etc/cron.daily, I think
> that runs at 2:00 AM (Haven't verified though).
>

Make that 4:00 AM  :-)

/jeroen
Title: Re: Squid Log files - automatic conversion
Post by: Samer Pharaon on November 06, 2003, 01:01:36 AM
Thanks a lot Nathan for your reply. Would you please explain how to substitute the 11-05-2003 with [today] and how to invoke the /sbin/date. This point was not clear for me.

Thank you again,
Samer
Title: Re: Squid Log files - automatic conversion
Post by: Paul Nesbit on November 06, 2003, 01:28:27 AM
Samer Pharaon wrote:
>
>
> Thanks a lot Nathan for your reply. Would you please explain
> how to substitute the 11-05-2003 with [today] and how to
> invoke the /sbin/date. This point was not clear for me.

timeconvert.pl > /var/log/squid/access-$(date +%d-%m-%Y).log

(See 'man date' for more info on format controls for the date utility.)

  Paul
Title: Re: Squid Log files - automatic conversion
Post by: Samer Pharaon on November 06, 2003, 12:53:31 PM
Thank you Paul. That was helpful.

One more question:
Is it possible to add something to the script to make it send the converted file automatically to one specific email address. May be this will be difficult given the fact that the file name will be changing every day. What would be the procedure and the script for doing this?

Many thanks,
Samer
Title: Re: Squid Log files - automatic conversion
Post by: Nathan Fowler on November 06, 2003, 05:18:01 PM
#!/bin/sh
cd /var/log/squid
/var/log/squid/access.log | /timeconvert.pl > /var/log/squid/access-$(date +%d-%m-%Y).log
uuencode access-$(date +%d-%m-%Y).log access-$(date +%d-%m-%Y).log | mail -s "Squid Access Log" username@domain.com
Title: Re: Squid Log files - automatic conversion
Post by: Samer Pharaon on November 06, 2003, 11:53:22 PM
I forgot to say that the command "cat" should appear in front of the text, as follows:

cat /var/log/squid/access.log | /timeconvert.pl > /var/log/squid/access-$(date +%d-%m-%Y).log

I added the file "squidlogconvert" to the "/etc/cron.daily" folder and then issued the "chmod" command. However, I received the following error msg through the Cron Daemon email, which reads as follows:

/etc/cron.daily/squidlogconvert:

/etc/cron.daily/squidlogconvert: line 2: syntax error near unexpected token >'
/etc/cron.daily/squidlogconvert: line 2: cat /var/log/squid/access.log | /timeconvert.pl > '

The same command executes without any problems if issued directly on the command line. Any ideas what went wrong. ??

Thanks,
Samer
Title: Re: Squid Log files - automatic conversion
Post by: Nathan Fowler on November 07, 2003, 12:45:11 AM
Paste me the contents of your file.
Title: Re: Squid Log files - automatic conversion
Post by: Samer Pharaon on November 08, 2003, 10:17:50 AM
I found the problem. The text after ">" came on the second line. So the script was not able to complete the command.

Thanks for your help.
Samer