Topic: squid log files rotation time of the day? day of the week? day of the month?

[purvis]

inside the squid file in the directory /etc/logrotate.d you can make changes how the access.log file is archived.
although rotating log files can be done manually, i plan to let my access.log file have many months of backups where the log rotation will be done weekly automatically.
i noticed that last week the rotation took place on tuesday which i felt it would rotate on maybe a sunday in the wee hours of the morning.

how can i predict the times and days when those files will rotate if set daily, weekly, or monthly
thanks

 

[cactus]

On a standard SME Server the logrotate script is kicked of daily (/etc/cron.daily/logrotate). Most likely the conditions to rotate logs where met at Tuesday in your case, IIRC normally logrotation is done on a weekly basis. My guess is you installed your machine on a Tuesday and the weekly count started from there.

From 'man logrotate':

weekly: Log  files  are  rotated  if the current weekday is less then the weekday of the last rotation or if more then a week has passed since the last rotation. This is normally the same as rotating logs on the first day of the week, but it works better if  logrotate is not run every night.

I guess the easiest fix in this case would be to run it in a Sunday forcefully either by a custom cron job or by manual intervention, with something like this:

Code: [Select]

/usr/sbin/logrotate -f /etc/logrotate.conf
Concerning the 'wee hours', since the logrotate is in the cron.daily folder it is kicked of at the time the cron.daily jobs are queued, which is done through /etc/crontab, which in most cases will have a time of 4:02 AM local time.

[purvis]

Thanks cactus.
I was also looking into the file timestamp as well.
I hate to write my own code to do things my way by but that seems to maybe be the path.
I did a forced log rotate last night on Monday morning at 2am so that may cover it.
I am mostly interested in the access.log file only.

[cactus]

I did a forced log rotate last night on Monday morning at 2am so that may cover it.

I am curious to know if this works as I suggested. Could you check and report back in a week?

[purvis]

Cactus i will try and remember to report back.
I have a lot of learning to do to in order to check things out.
I am rolling out some servers to do logging with squid and it is going to be a slow process.
When i get more out there, i only have 2 now in running mode, i will be able to get more info on what is happening.
It would appear that linux does not keep a file creation date.
I have turned off file time stamping of last time accessed with noatime to help my lower speed hard drives perform faster.

My rotation period is set to weekly and looking already at a file using the stat command, the files last accessed file time is over a week.
In testing the squid server, i deleted the squid log files to see if squid would recreate the file. I found out real quick not to do that because it would keep squid from not working. So i created a new access.log file(i had backed them up before i deleted them). It appears the log files chmod need to be set to 644 and that is probably what it is from a sme linux installation.

During my recreation of the squid log files, i set the file's chmod to 777 to quickly get my system back up and going.
I see now my log files older that a week and already after an automatic file rotation, the chmod is still 777 and the last access time is two weeks old and the log files did rotate automatically at least once.
 
here is a quick view of what i have now in the directory of /var/log/squid from a server i placed into service about two weeks ago.
i turned off compression of the access.log files that get rotated until i get  accustom to working with compressed files in an automated fashion.

ls -l

total 6380
-rwxrwxrwx 1 squid squid  897261 Aug  8 20:14 access.log
-rwxrwxrwx 1 squid squid 1831581 Aug  7 04:02 access.log.1
-rwxrwxrwx 1 squid squid  543387 Aug  2 17:49 access.log.2
-rwxrwxrwx 1 squid squid  863269 Aug  2 04:02 access.log.3
-rwxrwxrwx 1 squid squid     999 Aug  8 17:29 cache.log
-rwxrwxrwx 1 squid squid     322 Aug  7 04:02 cache.log.1.gz
-rwxrwxrwx 1 squid squid     176 Aug  2 17:49 cache.log.2.gz
-rwxrwxrwx 1 squid squid  164801 Aug  2 04:02 cache.log.3.gz
-rw-r--r-- 1 squid squid       5 Aug  1 23:11 squid.pid
-rwxrwxrwx 1 squid squid 1296158 Aug  9 03:11 store.log
-rwxrwxrwx 1 squid squid  525336 Aug  7 04:02 store.log.1.gz
-rwxrwxrwx 1 squid squid  165646 Aug  2 17:49 store.log.2.gz
-rwxrwxrwx 1 squid squid  169244 Aug  2 04:02 store.log.3.gz



stat access.log

  File: `access.log'
  Size: 897261          Blocks: 1768       IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 2884135     Links: 1
Access: (0777/-rwxrwxrwx)  Uid: (   23/   squid)   Gid: (   23/   squid)
Access: 2011-07-28 22:12:50.000000000 -0500
Modify: 2011-08-08 20:14:06.000000000 -0500
Change: 2011-08-08 20:14:06.000000000 -0500

 

[janet]

purvis

chmod 644 is correct for those files

[purvis]

Thanks Mary on concurring with the chmod of the squid log files being the default of 644

I am disappointed to find that most all ix's oses do not maintain the creation date of a file. Which ones do, i do not know.

With out further searching on how squid's log file rotation keeps up with dates on knowing when to rotate the files based on day,weekly, or monthly.
I would think squid  looks in the log file and uses the date/time  of the first entry in log file to determine if the proper time has passed.
I will be testing that thought out.

[cactus]

I am disappointed to find that most all ix's oses do not maintain the creation date of a file. Which ones do, i do not know.

By nature of unix and linux, virtually none I think, why do you care so much?

[cactus]

My rotation period is set to weekly and looking already at a file using the stat command, the files last accessed file time is over a week.
In testing the squid server, i deleted the squid log files to see if squid would recreate the file. I found out real quick not to do that because it would keep squid from not working. So i created a new access.log file(i had backed them up before i deleted them). It appears the log files chmod need to be set to 644 and that is probably what it is from a sme linux installation.

Are you sure you did not override the defaults which are in /etc/logrotate.d/squid IIRC?

[purvis]

Cactus.
Date of when a file is created has more importance for me usually than a last file access date.
The only time a global last accesssed date has done me any good is for in searching for a programs that should not be running in a windows system.
But it is handy too for finding backup files that where created at certain point.   

Who knows what does cause or does not cause the last accessed file date to be changed. That is question in my mind and i do not feel i could count on any knowledge 100 percent to what describes that. I mean, does a search function update that, or how about a backup routine. So can i depend on that last file time accessed? Probably not.

To your second question on /etc/logrotate.d/squid changes.
Yes i did make some changes for my set up, but small.
In the access.log section where parameters can be changed.
I removed compress. I changed the number of rotations to be 160.

I want to store up to three years of log files on my servers without them being purged out with a rotation routine. Actually, two years will probably give me what i really need. The extra year is for buffer. The rotation is set for the default weekly.

I right now have a cron procedure that copies all these files over to another location each 5 minutes. We will see how all this turns out. I expect the files to become large and as you know, transfer time depends on drive write and read speed.
If the squid  log file rotation would not change file names, i would not have keep so many.

The location that i copy the files to is the same internet accessable ibay location that i am using
 to run my programs at.