Topic: an alternate method to sme virusscans

[purvis]

My virsus scans take a long time and use a lot of cpu and disk resources.
Using the sme server-manager panel is just not working the best for me.
Using the server-manager panel activates clamscan to do scans daily, weekly, or no virus scans at all.

It is taking almost 8 hours to do these virus scans.
Why so long?
A large number of files, a single core CPU, disk not the fastest speed demons, and also I feel the clamscan process is hogging the system.

I did some testing. I found that if clamscan is given less priority, it will help smooth out the other needs of the computer.
One of those is reading and righting to files and other active processes need attention also.
Trying to work with files or doing any other computer chores, including the server's own chores it has to do while the server is doing a full virus scan,  is not what i call too productive.

What is likely best for me is !
1 Run a full virus scan on Sunday when nobody is working.
2 Try to change the way scanning software is run, reduce the computer resources it uses as much as possible.
3 Do virus scans at certain times of the day and set a limit on the number of files that I should think need scanning.

OK I did not do all of that but I now have a better solution for me and it will need to be tweaked more in the future.
I feel like I have tweaked the virus scans process enough.
I have written a bash file that will allow me to turn off all the virus scans that the sme server-manager panel uses or I
can have sme server-manager panel do the virus scans only weekly. Weekly is on Sunday.

For our use, we do not have viruses automatically removed or deleted, we just want viruses reported, except on emails.
I have already placed code for bash file to delete email viruses in the forum. Search for "clamscanemails".
So viruses scans regarding emails should not be much of a worry using the clamav software installed in the sme server.

Here is a not so perfect, but workable bash file to do virus scans for daily and weekly virus scans if placed into the /etc/cron.daily directory.

[purvis]

This bash routine will scan for viruses using the clamav antivirus scanner software already running on the sme server.
This routine will place the scanned results in a file on the server where it can be viewed from a web browser.
http://yourserver/serverstatus/clamscandaily.txt

If this bash file is run as is, then you need to disable(turn off) the daily and weekly virus scanning from inside the sme server-manager panel.

This program can be easily edited to:
Make no log entries to the clamscandaily.txt file by commenting out one line and making the variable logit="/dev/null".
 

This bash routine will only check files for viruses that are less than 73 days old or have access times less than 73 days old for daily scan and
scan all files on Sunday. 
The 73 days can be changed to another number but i would get to extreme with a large number of days.

A lot of operating systems are now turning off the date last accessed files attributes.
This can be a problem when scanning for virus in files with by using a date delimiter.

I will likely be changing this bash routine where it can also run a full virus scan in the evening, but not sure how I want to do that.
Anybody using this program should already know how long it takes to run a full virus scan on their servers. If not, it is time to find out.
But for me it is a long intensive process to run a full virus scan on the equipment I have.
If you have a great newer processor and lots of memory, maybe you do not even need a routine like this.

This routine on daily scans, places the names and directory of files into a temporary file, then clamscan is run against the files listed in the temporary file.
The temporary file holding the file list is deleted after the routine completes it job. This program also deletes(cleans) older temporary files with a similar name in the case the routine is does not finish.

If i am not mistaken, if you want to send an email to the admin and do not want the information logged another way.
The procedure would be to make the variable logit="/dev/null", by editing out the comment character and then
removing the ">> $logit" and removing the option "--no-summary" from the lines where clamscan is run.
From my testing, I think you will only get an email if this bash routine is run by the server in an automatic fashion when
the bash file is placed in the /etc/cron.daily folder.   If you manually run the bash files from a console, you will likely not
get a email going to the root account.

Code: [Select]

#!/bin/bash

# routine created on 03-11-2013 19:13
# routine scans for viruses in files using clamscan
# on sunday days, it does a full scan
# on days not being sunday, only files less than filesxdaysold days old are scanned

#                WARNING!!!
# REVIEW THE VIRUS SCANNING SCHEDULE IN THE SERVER MANAGER PANEL
# YOU SHOULD NOT HAVE THE SERVER SET TO DO DAILY VIRUS SCANS

# IF YOU SET THE SCHEDULE TO DO A VIRUS SCAN ONCE A WEEK, ON SUNDAYS
# YOU SHOULD COMMENT OUT THE IF BLOCK BELOW IF SCANNNG WEEKLY, ON SUNDAYS


routinename="clamscandaily"
filesxdaysold="73"

/usr/bin/renice 19 -p $$ > /dev/null

# create a uuid in lower case
uuidtext="$(echo $(uuidgen) | tr '[A-Z]' '[a-z]')"
uuidtext=$(echo ${uuidtext//[-._]/})


locationoflogfile="/home/e-smith/files/ibays/Primary/html/serverstatus"
logfilename="clamscandaily.txt"
logit="$locationoflogfile/$logfilename"
mkdir -p $locationoflogfile
chmod 755 $locationoflogfile


# REMOVE THE COMMENT ON THE NEXT LINE TO FORCE NO OUTPUT TO THE LOG FILE
#logit="/dev/null"


tempdirectory="/tmp"
mkdir -p $tempdirectory
chmod 777 $tempdirectory

#delete any old temporary files that might exist
find $tempdirectory/ -name "$routinename*.tmp" -type f -mtime +3 -delete

filewithlisttoscan=$routinename'_'$uuidtext".tmp"
listoffilestoscan="$tempdirectory/$filewithlisttoscan"

# the next line will delete the log file for testing purposes
#rm -f $logit > /dev/null

TODAY=$(date +"%Y%m%d %T")
echo "$TODAY $routinename started" >> $logit

DAYOFWEEK=$(date +"%w")

# THIS IF BLOCK WILL DO VIRUS SCANS ON NON SUNDAYS AND ONLY FILES X DAYS OLD
if [ "$DAYOFWEEK" != "0" ]
then
echo "$TODAY finding files < $filesxdaysold days"  >> $logit
/bin/nice -n 19 /usr/bin/ionice -c3 -n7 /usr/bin/find /home/e-smith/files/ -name "*" -type f -mtime -$filesxdaysold -ctime -$filesxdaysold > $listoffilestoscan
TODAY=$(date +"%Y%m%d %T")
echo "$TODAY scanning files found" >> $logit
/bin/nice -n 19 /usr/bin/ionice -c3 -n7 /usr/bin/clamscan --no-summary  --infected  -f $listoffilestoscan >> $logit
rm -f $listoffilestoscan > /dev/null
fi



# THIS IF BLOCK WILL DO VIRUS SCANS ON SUNDAYS
# COMMENT THIS SECTION OUT IF YOU ARE DOING FULL VIRUS SCANS WEEKLY
# FOUND IN THE VIRUS SECTION OF THE SME SERVER MANAGER PANEL
if [ "$DAYOFWEEK" == "0" ]
then
cd /home/e-smith/files
TODAY=$(date +"%Y%m%d %T")
echo "$TODAY scanning all files" >> $logit
/bin/nice -n 19 /usr/bin/ionice -c3 -n7 /usr/bin/clamscan  --no-summary --infected -r  >> $logit
fi

TODAY=$(date +"%Y%m%d %T")
echo "$TODAY $routinename ended"  >> $logit
echo '----------------------------------'  >> $logit

exit 0


this routine is found on my server under the freebies sub url as mentioned in a previous post today.

This script will be changed in the future to something better. I am thinking about how I want to do that.

As a side note, I want to mention that originally altered the clamscanemails bash script that was posted.
It actually ran faster using clamdscan rather than clamscan on my system the way I was scanning emails.
The server's CPU resources where much lower using clamdscan than clamscan.
I decided to used clamscan for now only because the summary clamscan can produce and the more simplicity
of having the a emailed report if the bash file is edited.  Also I wanted others to be able to monitor their server's resources
as compared to the way clamscan is being used on a sme server with the regular daily or weekly virus scanning.
I more or less just listen to my server's fan that tells me the CPU is getting a heavy work out and I visually watch the
console with the "top" command.

For those wanting to run a bash routine in the background while at the console, just add a space and a ampersand sign to end of your command line
like "/directory/command &" or "./command &" if you are at the current directory of the bash routine.

[purvis]

Here is another version similar to the above that will use either clamscan or clamdscan depending on a variable setting in the file bash file
Set the variables scanprogram and/or  filesxdaysold
If you are testing for speed, be sure to run both settings multiple times before changing. Why, because of system file caching likely will cause some differences.
You can watch the CPU using the 
You can also set the variable DAYOFWEEK to "0" to force a full scan.

Code: [Select]

#!/bin/bash


# routine created on 03-13-2013 02:13a

# routine scans for viruses in files using clamscan or clamdscan
# set scan program used to 1 or 2 below see variable scanprogram
# set number of days from today on files to be scannned see variable filesxdaysold

# on sunday days, it does a full scan
# on days not being sunday, only files less than filesxdaysold days old are scanned

#                WARNING!!!
# REVIEW THE VIRUS SCANNING SCHEDULE IN THE SERVER MANAGER PANEL
# YOU SHOULD NOT HAVE THE SERVER SET TO DO DAILY VIRUS SCANS

# IF YOU SET THE SCHEDULE TO DO A VIRUS SCAN ONCE A WEEK, ON SUNDAYS
# YOU SHOULD COMMENT OUT THE IF BLOCK BELOW IF SCANNNG WEEKLY, ON SUNDAYS

#set the scan method
#scan program used  1 is using clamscan and 2 is using clamdscan
scanprogram="2"
 
#set the number of days from today that are included in a dailyscan
filesxdaysold="73"



routinename="clamscandaily"
/usr/bin/renice 19 -p $$ > /dev/null

locationoflogfile="/home/e-smith/files/ibays/Primary/html/serverstatus"
logfilename="clamscandaily.txt"
logit="$locationoflogfile/$logfilename"
mkdir -p $locationoflogfile
chmod 755 $locationoflogfile

# remove the commment below to delete the log file for testing purposes only
#rm -f $logit > /dev/null

# REMOVE THE COMMENT ON THE NEXT LINE TO FORCE NO OUTPUT TO THE LOG FILE
#logit="/dev/null"

scanprogramdescription=""
if [ "$scanprogram" == "1" ]
then
scanprogramdescription="clamscan"
fi
if [ "$scanprogram" == "2" ]
then
scanprogramdescription="clamdscan"
fi


DAYOFWEEK=$(date +"%w")
#to force a full virus scan, remove the comment character out the next line
#DAYOFWEEK="0"

echo '----------------------------------'  >> $logit
TODAY=$(date +"%Y%m%d %T")
echo "$TODAY $routinename started" >> $logit

if [ "$scanprogram" != "1" ] && [ "$scanprogram" != "2" ]
then
echo "$TODAY no program used" >> $logit
fi

################################################### start of using the clamscan method
if [ "$scanprogram" == "1" ]
then

echo "$TODAY $scanprogramdescription used" >> $logit
tempdirectory="/tmp"
mkdir -p $tempdirectory
chmod 777 $tempdirectory
#delete any old temporary files that might exist
find $tempdirectory/ -name "$routinename*.tmp" -type f -mtime +3 -delete
# create a uuid in lower case
uuidtext="$(echo $(uuidgen) | tr '[A-Z]' '[a-z]')"
uuidtext=$(echo ${uuidtext//[-._]/})
filewithlisttoscan=$routinename'_'$uuidtext".tmp"
listoffilestoscan="$tempdirectory/$filewithlisttoscan"

# THIS IF BLOCK WILL DO VIRUS SCANS ON NON SUNDAYS AND ONLY FILES X DAYS OLD
if [ "$DAYOFWEEK" != "0" ]
then
echo "$TODAY finding files < $filesxdaysold days"  >> $logit
/bin/nice -n 19 /usr/bin/ionice -c3 -n7 /usr/bin/find /home/e-smith/files/ -name "*" -type f -mtime -$filesxdaysold -ctime -$filesxdaysold > $listoffilestoscan
TODAY=$(date +"%Y%m%d %T")
echo "$TODAY scanning files found" >> $logit
/bin/nice -n 19 /usr/bin/ionice -c3 -n7 /usr/bin/clamscan --no-summary  --infected  -f $listoffilestoscan >> $logit
rm -f $listoffilestoscan > /dev/null
fi
# THIS IF BLOCK WILL DO VIRUS SCANS ON SUNDAYS
# COMMENT THIS SECTION OUT IF YOU ARE DOING FULL VIRUS SCANS WEEKLY
# FOUND IN THE VIRUS SECTION OF THE SME SERVER MANAGER PANEL
if [ "$DAYOFWEEK" == "0" ]
then
TODAY=$(date +"%Y%m%d %T")
echo "$TODAY scanning all files" >> $logit
cd /home/e-smith/files
/bin/nice -n 19 /usr/bin/ionice -c3 -n7 /usr/bin/clamscan  --no-summary --infected -r  >> $logit
fi
fi
################################################## done using clamscan method


################################################## start of using the clamdscan method
if [ "$scanprogram" == "2" ]
then

echo "$TODAY $scanprogramdescription used" >> $logit
pidclamd=$(ps -C clamd -o pid=)
/usr/bin/renice 19 -p $pidclamd > /dev/null

# THIS IF BLOCK WILL DO VIRUS SCANS ON NON SUNDAYS AND ONLY FILES X DAYS OLD
if [ "$DAYOFWEEK" != "0" ]
then
TODAY=$(date +"%Y%m%d %T")
echo "$TODAY scanning files < $filesxdaysold days" >> $logit
/bin/nice -n 19 /usr/bin/ionice -c3 -n7 find /home/e-smith/files/ -name "*" -mtime -$filesxdaysold -ctime -$filesxdaysold -type f -exec /usr/bin/clamdscan {} --no-summary --infected --stdout --fdpass \; >> $logit
fi
# THIS IF BLOCK WILL DO VIRUS SCANS ON SUNDAYS
# COMMENT THIS SECTION OUT IF YOU ARE DOING FULL VIRUS SCANS WEEKLY
# FOUND IN THE VIRUS SECTION OF THE SME SERVER MANAGER PANEL
if [ "$DAYOFWEEK" == "0" ]
then
TODAY=$(date +"%Y%m%d %T")
echo "$TODAY scanning all files" >> $logit
/bin/nice -n 19 /usr/bin/ionice -c3 -n7 find /home/e-smith/files/ -name "*" -type f -exec /usr/bin/clamdscan {} --no-summary --infected --stdout --fdpass \; >> $logit
fi
/usr/bin/renice 0 -p $pidclamd > /dev/null
fi
#################################################### done using clamdscan method

TODAY=$(date +"%Y%m%d %T")
echo "$TODAY $routinename ended"  >> $logit

exit 0