Topic: bash routine to update clamd virus definitions

[purvis]

This routine will download and update the clamd virus definition files main.cvd, daily.cvd, and bytecode.cvd files.
You can set the web url location of where to get the files.
You can set a few variables in the routine to tell the routine to do a few things.
The routine downloads two sets of files and compares them for completeness.
The two sets of files are not deleted after the routine but left for testing and the
next time the routine is run.
When downloading the cvd files, if this routine finds the cvd files downloaded in the first set match the second set
from a previously download, then it will not download the second set of cvd files to save some time.
I am sure there are other fancy ways of getting the same results,  but this is what I know how to do.

I would prefer bash routines had GOTO statements and LABELS,  those would make bash routines a little easier for me to write small routines.
My coding is not perfect but it apparently gets the job done.
This program will keep two sets

Code: [Select]

see a following post


[Stefano]

purvis..

thank you for your effort..

all your work/routines should be in the wiki

[larieu]

Stefano

I have formatted the code for wiki


http://wiki.contribs.org/Talk:Clamav:freshclam_update

[purvis]

Thank you both Stefano and larieu
Stefano, I had requested to be able to add a wiki some time ago and that request was either not processed or not honored for whatever reason.
That was fine with me. That is why i do not mind posting in this section of the forums.
Plus if I post a script or something here. It has an opportunity of being improved, checked, and tested by others before a wiki.

I am sorry about the wiki needing to be edited larieu.

I am also working on a script to not send any output except when an update fails.

[purvis]

I found 1 error and 3 lines that need to be added in the above script.


2 lines added here
Freshclam update creates files that needed to be deleted
The code written did not do a full wildcard in the case the variable $clamdir was wrongly edited.
The safe approach was delete specific files.
Newer freshclam update files where not coded to be deleted which needed to be.
near the bottom of the routine, 2 lines need to be added below the line

Code: [Select]

rm  -f $clamdir/*.cvd 1> /dev/null 2> /dev/null


Code: [Select]

rm  -f $clamdir/*.cld 1> /dev/null 2> /dev/null
rm  -f $clamdir/mirrors.dat 1> /dev/null 2> /dev/null




1 error fixed here and 1 line added here
very shortly after the first line of

Code: [Select]

comparecvdfiles
downloaddirectory=$downloaddirectory1  should be changed to
downloaddirectory=$downloaddirectory2

then under that line add

Code: [Select]

   downloadcvdfiles
before the line of
dodownloadfilesexist
then the small section of code will read as

Code: [Select]

comparecvdfiles
if [ $result == "0" ]
   then
   downloaddirectory=$downloaddirectory2
   downloadcvdfiles
   dodownloadfilesexist


[purvis]

clamcvdget bash script

Code: [Select]

#!/bin/bash

# clamcvdget
# routine created on 03-18-2013 16:00
# this routine will update the clamav virus signature datafiles

######################################## VARIABLES FOR TESTING
# SET skipdownload TO 1 WILL SKIP DOWNLOADING OF NEW CVD FILES
# IF FILES NEED TO BE DOWNLOADED THEY WILL BE AUTOMATICALLY
# FOR TESTING PURPOSES
  skipdownload="0"

# SET dovirustest TO 1 WILL CAUSE A SHORT CLAMSACN VIRUS
# TEST TO DISPLAY SHOWING SUMMARY AND KNOWN VIRUS COUNT
  doclamtest="1"
#SET showversion TO 1 TO DISPLAY VERSION OF CLAMD AND CLAMSCAN
  showversion="1"
#############################################################


####################### THESE VARIABLES MUST BE SET CORRECTLY
  downloadurl="db.us.clamav.net"
  file1="main.cvd"
  file2="daily.cvd"
  file3="bytecode.cvd"
  clamdir="/var/clamav"
  downloaddirectory=""
  downloaddirectory1='/var/clamav/download/1'
  downloaddirectory2='/var/clamav/download/2'
#############################################################

/usr/bin/renice 20 -p $$ > /dev/null


############################### begin of functions
#  FUNCTION TO COMPARE DOWNLOADED CVD FILES
function comparecvdfiles {
result="0"
if ! diff \
   "$downloaddirectory1/$file1" \
   "$downloaddirectory2/$file1" > /dev/null
   then
   return 0
fi
if ! diff \
   "$downloaddirectory1/$file2" \
   "$downloaddirectory2/$file2" > /dev/null
   then
   return 0
fi
if ! diff \
   "$downloaddirectory1/$file2" \
   "$downloaddirectory2/$file2" > /dev/null
   then
   return 0
fi
result="1"
}

# FUNCTION TO DOWNLOAD CVD FILES
function downloadcvdfiles {
   result="0"
   cd /tmp
   currentdir=$PWD
   if [ $currentdir != "/tmp" ]
      then
      return 0
   fi
   if [ $downloaddirectory == "" ] | [ $downloaddirectory == "/" ]
      then
      return 0
   fi
   mkdir -p  $downloaddirectory
   chmod 755 $downloaddirectory
   rm -f $downloaddirectory/* > /dev/null
   cd $downloaddirectory
   currentdir=$PWD
   if [ $currentdir == "/tmp" ] | [ $currentdir ==  "/" ]  | [ $currentdir  == "/root" ]
      then
      return 0
   fi
   wget -q $downloadurl/$file3
   wget -q $downloadurl/$file1
   wget -q $downloadurl/$file2
   echo "done with downloading"
   cd /tmp
   result="1"
}

#FUNCITON TO CHECK FOR EXISTENCE OF CVD FILES
function  dodownloadfilesexist {
   result="0"
   if [ ! -f "$downloaddirectory/$file1" ] | [ ! -f "$downloaddirectory/$file2" ] \
      | [ ! -f "$downloaddirectory/$file3" ]
      then
      return 0
      else
      result="1"
   fi
}


################################## end of functions


#TODAY=$(date +"%Y%m%d %T")
#echo "$TODAY started" 


if [ $skipdownload == "0" ]
then
   echo "downloding new clamav cvd files"
   downloaddirectory=$downloaddirectory1
   downloadcvdfiles
fi


# DOWNLOAD FIRST SET OF CVD FILES IF NOT EXIST
downloaddirectory=$downloaddirectory1
dodownloadfilesexist
if [ $result == "0" ]
   then
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   sleep 90
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   sleep 90
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   echo "Update cvd files do not exist"
   echo "Cannot update Clamav"
   exit 0
fi

# DOWNLOAD SECOND SET OF CVD FILES IF NOT EXIST
downloaddirectory=$downloaddirectory2
dodownloadfilesexist
if [ $result == "0" ]
   then
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   sleep 90
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   sleep 90
   downloadcvdfiles
fi
if [ $result == "0" ]
   then
   echo "Update cvd files do not exist"
   echo "Cannot update Clamav"
   exit 0
fi



comparecvdfiles
if [ $result == "0" ]
   then
   downloaddirectory=$downloaddirectory2
   dodownloadfilesexist
  if [ $result == "0" ]
      then
      downloadcvdfiles
   fi
  if [ $result == "0" ]
      then
      sleep 90
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      sleep 90
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      echo "Update cvd files do not exist"
      echo "Cannot update Clamav"
      exit 0
   fi
fi


comparecvdfiles
if [ $result == "0" ]
   then
   downloaddirectory=$downloaddirectory2
   downloadcvdfiles
   dodownloadfilesexist
  if [ $result == "0" ]
      then
      downloadcvdfiles
   fi
  if [ $result == "0" ]
      then
      sleep 90
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      sleep 90
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      echo "Update cvd files do not exist"
      echo "Cannot update Clamav"
      exit 0
   fi
fi

comparecvdfiles
if [ $result == "0" ]
then
echo "Update files are not complete"
echo "Cannot update ClamAV"
exit 0
fi

echo "updating Clamav"
service clamd stop > /dev/null
cd $clamdir
rm  -f $clamdir/*.cvd 1> /dev/null 2> /dev/null
rm  -f $clamdir/*.cld 1> /dev/null 2> /dev/null
rm  -f $clamdir/mirrors.dat 1> /dev/null 2> /dev/null
cp -p $downloaddirectory1/*.cvd $clamdir
service clamd start > /dev/null


# THE FOLLOWING LINE IS JUST A TEST TO SEE
if [ $showversion == "1" ]
then
clamd -V
clamscan -V
fi

if [ $doclamtest == "1" ] 
then
echo "running clamscan test"
mkdir -p $clamdir/temp
clamscan $clamdir/temp
rm -rf $clamdir/temp
fi

#TODAY=$(date +"%Y%m%d %T")
#echo "$TODAY ended"

exit 0


[purvis]

I have added a few more features to this script.

You can set the script to wait until a certain time of the same day to do its processing.
The program makes use of the sleep statement for this process and the script will renice itself, so there should not be issues there.

You can set the script to run only one instance at a time.
This is so that there will not be any conflicts with the above in setting a certain time of day to do the update.

I inserted 3 variables to be used as flags to cause most messages to be displayed or not.

The clamscan test is not needed at the bottom of the script but is ok for test purposes.
There is a flag to turn on and off the clamscan test.

In an effort to not cause a conflict with any ClamAV process running at the time the script is trying to do clamav cvd file updates.
The script will try to identify any ClamAV processes running(clamdscan, clamscan, or freshclam)  over a short period of time before stopping the clamd and freschclam service to copy cvd files to the /var/clamav directory.

For my purposes, I only want to run this script once a day and before midnight, or while testing.

The goal I had in mind when creating this script was to update the clamav cvd files any time i wanted manually and fill a gap where freshclam may not be running.

In using the 3 wget statements, I would like to do retries and write over file options. I had some issues using the wget while testing.
Inside the function where downloading of cvd files is, that function could be improved.  I need more testing. But if the two sets of cvd files are not equal.
This script will abort itself.
If you set outputerr to 1 and set quiet to 0, you will get messages of failures.

Code: [Select]

#!/bin/bash

# clamcvdget
# routine created on 03-18-2013 22:00:00
# this routine will update the clamav virus signature datafiles

######################################## START OF VARIABLES
# SET onlyoneinstance TO 1 TO FORCE ONLY ONE INSTANCE OF THIS
# ROUTINE TO RUN AT A TIME
    onlyoneinstance=0

# SET skipdownload TO 1 WILL SKIP DOWNLOADING OF NEW CVD FILES
# IF FILES NEED TO BE DOWNLOADED THEY WILL BE AUTOMATICALLY
# FOR TESTING PURPOSES
    skipdownload=0

# SET dovirustest TO 1 WILL CAUSE A SHORT CLAMSACN VIRUS
# TEST TO DISPLAY SUMMARY AND KNOWN VIRUS COUNT
    doclamtest=0

# SET showversion TO 1 TO DISPLAY VERSION OF CLAMD AND CLAMSCAN
    showversion=1

# SET outputmsg TO 1 TO DISPLAY REGUALAR MESSAGES IN ROUTINE
    outputmsg=1

# SET outputerr TO 1 TO DISPLAY ERROR MESSAGES IN ROUTINE
    outputerr=1

# SET quiet TO 1 TO NOT DISPLAY ANY MESSAGES AND DO NO TEST
    quiet=0

# SET runonlyattime FOR THE ROUTINE TO WAIT FOR A SPECFIC TIME
# IF YOU SET runonlyattime TO 1 YOU MUST SET runathourminsec
#    AND SET onlyoneinstance TO 1 DURING PRODUCTION USE
    runonlyattime=0

# SET runathourminsec TO "HH:MM:SS" FOR THE TIME OF TODAY
# THE TIME OF DAY IS ONLY FOR TODAY AND NO OTHER DATES
# THE ROUTINE HAS TO START BEFORE THIS TIME OF THE DAY
# IF THE TIME HAS PASSED THEN THIS PROCESS WILL ABORT
# runathoursminsec VARIABLE WILL NOT BE CHECKED FOR CORRECTNESS
    runathourminsec="03:30:00"
#######################

####################### THESE VARIABLES MUST BE SET CORRECTLY
  downloadurl="db.us.clamav.net"
  file1="main.cvd"
  file2="daily.cvd"
  file3="bytecode.cvd"
  clamdir="/var/clamav"
  downloaddirectory=""
  downloaddirectory1='/var/clamav/download/1'
  downloaddirectory2='/var/clamav/download/2'
######################################## END OF VARIABLES



############################### begin of functions
#  FUNCTION TO COMPARE DOWNLOADED CVD FILES
function comparecvdfiles {
result="0"
if ! diff \
   "$downloaddirectory1/$file1" \
   "$downloaddirectory2/$file1" > /dev/null
   then
   return 0
fi
if ! diff \
   "$downloaddirectory1/$file2" \
   "$downloaddirectory2/$file2" > /dev/null
   then
   return 0
fi
if ! diff \
   "$downloaddirectory1/$file2" \
   "$downloaddirectory2/$file2" > /dev/null
   then
   return 0
fi
result="1"
}

# FUNCTION TO DOWNLOAD CVD FILES
function downloadcvdfiles {
   result="0"
   cd /tmp
   currentdir=$PWD
   if [ $currentdir != "/tmp" ]
      then
      return 0
   fi
   if [ $downloaddirectory == "" ] | [ $downloaddirectory == "/" ]
      then
      return 0
   fi
   mkdir -p  $downloaddirectory
   chmod 755 $downloaddirectory
   rm -f $downloaddirectory/* > /dev/null
   cd $downloaddirectory
   currentdir=$PWD
   if [ $currentdir == "/tmp" ] | \
      [ $currentdir ==  "/" ]  |  \
      [ $currentdir  == "/root" ]
      then
      return 0
   fi
   if [ $outputmsg == 1 ]
      then
      echo "downloding new clamav cvd files in $downloaddirectory"
   fi

   wget -q $downloadurl/$file3
   wget -q $downloadurl/$file1
   wget -q $downloadurl/$file2
   if [ $outputmsg == 1 ]
      then
      echo "done with downloading in $downloaddirectory"
   fi
   cd /tmp
   result="1"
}

#FUNCITON TO CHECK FOR EXISTENCE OF CVD FILES
function  dodownloadfilesexist {
   result="0"
   if [ ! -f "$downloaddirectory/$file1" ] | \
      [ ! -f "$downloaddirectory/$file2" ] | \
      [ ! -f "$downloaddirectory/$file3" ]
      then
      return 0
      else
      result="1"
   fi
}


################################## end of functions


routinename=$(basename $(readlink -nf $0))

if [ $onlyoneinstance == 1 ]
   then
   processname=$(basename $(readlink -nf $0))   
   if [ $(pidof -x $processname | wc -w) -gt 2 ]
      then
      exit 0
   fi
fi

/usr/bin/renice 20 -p $$ > /dev/null

if [ $quiet == 1 ]
  then
  doclamtest=0
  showversion=0
  outputmsg=0
  outputerr=0
  quiet=1
fi

if [ $outputmsg == 1 ]
   then
   TODAY=$(date +"%Y%m%d %T")
   echo "$TODAY $routinename bash routine started"
fi


if [ $runonlyattime == 1 ]
   then
   current_epoch=$(date +%s)
   target_epoch=$(date -d $runathourminsec +%s)
   sleep_seconds=$(($target_epoch - $current_epoch))
   if [ $outputmsg == 1 ]
      then
      echo -n "This routine is set to run at "
      echo $(date -d @$target_epoch +"%m-%d-%Y %T")
   fi

   if [ $sleep_seconds -lt 0 ]
      then
      if [ $outputmsg == 1 ]
         then
         echo "That time has already past."
         echo "Aborting $routinename bash routine"
      fi
      exit 0
      else
      if [ $outputmsg == 1 ]
         then
         echo "Waiting on time before continuing"
      fi
      sleep  $sleep_seconds
      if [ $outputmsg == 1 ]
         then
         TODAY=$(date +"%Y%m%d %T")
         echo "$TODAY $routinename bash routine continuing now"
      fi
   fi
fi


if [ $skipdownload == 0 ]
then
   downloaddirectory=$downloaddirectory1
   downloadcvdfiles
fi


# DOWNLOAD FIRST SET OF CVD FILES IF NOT EXIST
downloaddirectory=$downloaddirectory1
dodownloadfilesexist
if [ $result == "0" ]
   then
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   sleep 90
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   sleep 90
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   if [ $outputerr == 1 ]
      then
      echo
      TODAY=$(date +"%Y%m%d %T")
      echo "$TODAY routine $routinename"
      echo "Update cvd files do not exist"
      echo "  in the directory of $downloaddirectory."
      echo "Cannot update Clamav. Aborting update routine."
   fi
   exit 0
fi

# DOWNLOAD SECOND SET OF CVD FILES IF NOT EXIST
downloaddirectory=$downloaddirectory2
dodownloadfilesexist
if [ $result == "0" ]
   then
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   sleep 90
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   sleep 90
   downloadcvdfiles
fi
if [ $result == "0" ]
   then
   if [ $outputerr == 1 ]
      then
      echo
      TODAY=$(date +"%Y%m%d %T")
      echo "$TODAY routine $routinename" 
      echo "Update cvd files do not exist"
      echo "  in the directory of $downloaddirectory."
      echo "Cannot update Clamav. Aborting update routine."
   fi
   exit 0
fi


comparecvdfiles
if [ $result == "0" ]
   then
   downloaddirectory=$downloaddirectory2
   dodownloadfilesexist
  if [ $result == "0" ]
      then
      downloadcvdfiles
   fi
  if [ $result == "0" ]
      then
      sleep 90
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      sleep 90
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      if [ $outputerr == 1 ]
         then
         TODAY=$(date +"%Y%m%d %T")
         echo "$TODAY routine $routinename"
         echo "Update cvd files do not exist"
         echo "  in the directory of $downloaddirectory."
         echo "Cannot update Clamav. Aborting update routine."
       fi
      exit 0
   fi
fi

# COMPARE THE TWO SETS OF CVD FILES
# IF THE FILES ARE NOT EQUAL THEN DOWNLOAD THE SECOND SET
comparecvdfiles
if [ $result == "0" ]
   then
   downloaddirectory=$downloaddirectory2
   downloadcvdfiles
   dodownloadfilesexist
  if [ $result == "0" ]
      then
      downloadcvdfiles
   fi
  if [ $result == "0" ]
      then
      sleep 90
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      sleep 90
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      if [ $outputerr == 1 ]
         then
         TODAY=$(date +"%Y%m%d %T")
         echo "$TODAY routine $routinename"
         echo "Update cvd files do not exist"
         echo "  in the directory of $downloaddirectory."
         echo "Cannot update Clamav. Aborting update routine."
      fi
   exit 0
   fi
fi

# COMPARE THE TWO SETS OF CVD FILES FOR A SECOND AND FINAL TRY
# IF THE TWO SETS ARE NOT EQUAL THEN THE PROGRAM ABORTS
comparecvdfiles
if [ $result == "0" ]
then
if [ $outputerr == 1 ]
      then
      echo
      echo "Update cvd files do not exist"
      echo "Update cvd files are not complete"
      echo "Cannot update ClamAV"
   fi
exit 0
fi




# UPDATING THE CLAMAV SIGNATURE CVD FILES

# MAKE AN EFFORT TO CHECK IF CLAMSCAN IS RUNNING
# TEST TO MAKE SURE CLAMSCAN IS NOT RUNNING
#   FOR 15 SECONDS FIRST THEN IF CLAMSCAN IS
#   RUNNING TEST ONCE EVERY 15 SECONDS THERE AFTER
# WHEN CLAMSCAN HAS NOT BEEN RUNNING FOR 1
#   MINUTE THEN PROCEED TO UPDATE CLAMAV
if [ $outputmsg == 1 ];then echo "Verifying or waiting on freed up Clamav processes";fi
clamprocesses=1
until [  $clamprocesses -lt 1 ]; do
clamprocesses=0
counter=15
processname="clamscan"
until [ $counter -lt 1 ]; do
      if [ $(pidof -x $processname | wc -w) -gt 0 ]
         then
         sleep 1
         counter=30
         clamprocesses=1
         else
         let counter-=1
         sleep 1
      fi
done
counter=5
processname="clamdscan"
until [ $counter -lt 1 ]; do
      if [ $(pidof -x $processname | wc -w) -gt 0 ]
         then
         sleep 1
         counter=30
         clamprocesses=1
         else
         let counter-=1
         sleep 1
      fi
done
counter=5
processname="freshclam"
until [ $counter -lt 1 ]; do
      if [ $(pidof -x $processname | wc -w) -gt 1 ]
         then
         sleep 1
         counter=20
         clamprocesses=1
         else
         let counter-=1
         sleep 1
      fi
done
done

#STOPPING THE FRESHCLAM SERVICE
if [ $outputmsg == 1 ]
   then
   TODAY=$(date +"%Y%m%d %T")
   echo "$TODAY stopping freshclam service"
fi
service freshclam stop > /dev/null
counter=1
processname="freshclam"
until [ $counter -lt 1 ]; do
      if [ $(pidof -x $processname | wc -w) -gt 0 ]
         then
         sleep 2
         else
         if [ $outputmsg == 1 ]
             then
             TODAY=$(date +"%Y%m%d %T")
             echo "$TODAY freshclam service stopped"
         fi
         let counter=0
      fi
done

#STOPPING THE CLAMD SERVICE
if [ $outputmsg == 1 ]
   then
   TODAY=$(date +"%Y%m%d %T")
   echo "$TODAY stopping clamd service"
fi
service clamd stop > /dev/null
counter=1
processname="clamd"
until [ $counter -lt 1 ]; do
      if [ $(pidof -x $processname | wc -w) -gt 0 ]
         then
         sleep 2
         else
         if [ $outputmsg == 1 ]
         then
         TODAY=$(date +"%Y%m%d %T")
         echo "$TODAY clamd service stopped"
         fi
         let counter=0
      fi
done

#COPYING THE DOWNLOADED CLAMAV CVD FILES TO
#THE CLAMAV CVD WORKING DIRECTORY
if [ $outputmsg == 1 ]
   then
   TODAY=$(date +"%Y%m%d %T")
   echo "$TODAY updating the Clamav cvd files"
fi
cd $clamdir
rm  -f $clamdir/*.cvd 1> /dev/null 2> /dev/null
rm  -f $clamdir/*.cld 1> /dev/null 2> /dev/null
rm  -f $clamdir/mirrors.dat 1> /dev/null 2> /dev/null
cp -p $downloaddirectory1/*.cvd $clamdir 1> /dev/null 2> /dev/null

#STARTING THE CLAMD SERVICE
if [ $outputmsg == 1 ]
   then
   TODAY=$(date +"%Y%m%d %T")
   echo "$TODAY starting clamd service"
fi
service clamd start > /dev/null
counter=1
processname="clamd"
until [ $counter -lt 1 ]; do
      if [ $(pidof -x $processname | wc -w) -gt 1 ]
         then
         sleep 2
         else
         let counter=0
         if [ $outputmsg == 1 ]
            then
            TODAY=$(date +"%Y%m%d %T")
            echo "$TODAY clamd service started"
         fi
      fi
done

#STARTING THE FRESHCLAM SERVICE
if [ $outputmsg == 1 ]
   then
   TODAY=$(date +"%Y%m%d %T")
   echo "$TODAY starting freshclam service"
fi
service freshclam start > /dev/null
counter=1
processname="freshclam"
until [ $counter -lt 1 ]; do
      if [ $(pidof -x $processname | wc -w) -gt 1 ]
         then
         sleep 2
         else
         TODAY=$(date +"%Y%m%d %T")
         echo "$TODAY freshclam service started"
         let counter=0
      fi
done
if [ $outputmsg == 1 ]
   then
   TODAY=$(date +"%Y%m%d %T")
   echo "$TODAY updated Clamav"
fi


# THE SHOW THE VERSION OF CLAMAV
if [ $showversion == 1 ]
   then
   echo -n "ClamAV version - "
   clamd -V
fi

# THE FOLLOWING IS TO JUST RUN A SHORT TEST OF CLAMSCAN
if [ $doclamtest == 1 ] 
   then
   counter=15
   processname="clamscan"
   until [ $counter -lt 1 ]; do
      if [ $(pidof -x $processname | wc -w) -gt 0 ]
         then
         sleep 1
         counter=30
         else
         let counter-=1
         sleep 1
     fi
    done
    echo
    echo "Running a short clamscan test on an empty directory."
    echo "No files or viruses should be found in the summary."
    mkdir -p $clamdir/temp
    clamscan $clamdir/temp
    echo
    rm -rf $clamdir/temp
fi


if [ $outputmsg == 1 ]
   then
   TODAY=$(date +"%Y%m%d %T")
   echo "$TODAY $routinename bash routine ended successfully"
fi

exit 0


[purvis]

I just could not leave the script working half way.
I made the script check for all running processes before it updates the cvd files in the /var/clamav directory.
Code is never perfect.
The program now will wait on any clamscan, clamd or freshclam processes running, at least it is suppose to.
After all those processes have no activity(freshclam runs 2 processes while updating and 1 process when idle no updating ClamAVand).
The script will stop the service of freshclam and clamd,  then update the cvd files, then restart the clamd and freshclam services in that order.

The timing in the checking for freed up clamd processes maybe too loose for a busy server, specially where heavy ClamAV activity maybe accessing  and checking emails.
If outputmsg is set 1, the output of the routine should give some valuable information. If ClamAV processes never free up. This script will never end the way it is written.
Because some clamscan virus scans can take hours. This program script was desgined not to give up while clamscan is running, but the script will wait on clamscan to finish, then update the cvd files when there is no activity. Once again, that is where I put in a time of day variable for the script to actually do its work if the user wants to use that option.

[purvis]

Here is my newest clamcvdget script
I am still testing this script
This script has many changes.
The previous script would stop the clamav service then restart it.
This script gives you option to stop the clamav service before updating the cvd files.
This script also lets you set a period of time in seconds of how long a clamscan or clamdscan routine has not been run before it updates the cvd files.
If a clamscan, clamdscan, or freshclam routine is running,  this script will not update the cvd files because at this point I am still not
I am not convinced NOW you have to stop the clamav service, but the script does stop the freshclam service before updating the cvd files, then restarts the freshclam service after the cvd files have been updated.

I am also not too sure my route to catch more than one instance of this script is running properly.
I use the same function in another routine and for some reason it did not work a few times.
I am concerned over that some in this program and more so in other scripts that i would like the program to catch multiple instances of itself trying to run.

The routine is a bit chatty but sometimes that I what I need for testing

Code: [Select]

removed code
see below


[purvis]

I think that I am now done with this project on scripts updating the ClamAV with newer virus signature files(cvd files)

There is now 2 new scripts to improve this project. clamcvdupdate and a reworked clamcvdget script called calmcvdget2

clamcvdupdate which i place in /etc/cron.hourly to be run hourly. It will run clamcurrentck to compare the clamav cvd version running against the clamav web site.
If there is a different regardless in numbers it will run clamcvdget.

I also created a new version of clamcvdget. I call the new one clamcvdget2 and the old one clamcvdget1.
The new version can have up to 7 clamav database locations to try to download the clamav cvd files and it will only download one set of cvd files.
Rather than downloading 2 sets of cvd files and comparing them. It downloads 1 set and runs sigtool against the cvd files. If the returned result from using sigtool on each cvd file returns the string "Verification OK". Then it is assumed that those cvd files are ok to update ClamAV with.
The newer version should be quicker and more dependable if sigtool does not change the text of "Verification OK" being returned on properly formated cvd files.

I place clamcvdupdate in /etc/cron.hourly and the other files  clamcurrentck, clamcvdget1, and clamcvdget2, in a directory named /opt/myscripts. Then I create clamcvdget by copying  clamcvdget1 or clamcvdget2 to it.
You need to edit clamcvdupdate to where you place clamcurrentck and clamcvdget

Each script will can now be run in quiet mode

If you are going to use clamcvdupdate in a daily or hourly cron. You might want to set quietall to equal 1.
Clamcvdupdate has two options "-q" and "-qall"

I am not getting any emails to the root account from routines the way they are set. I like the peace.
If you ever want to test how up to date clamav is , use the clamcurrentck script.



clamcvdupdate

Code: [Select]

#!/bin/bash

### clamavupdate
### date of routine 04-02-2013 13:16:00
### script updates the Clamav cvd signature files
### script first determines if Clamav is up todate 
### if cvd files are not up todate then a update will  be tried
### the files for number of tries that is set below


################################# BEGIN OF VARIABLES
# set quiet to  1 to not display any messages
    quiet=1

# set quietall to  1 to not display any messages from sub scripts
    quietall=1

# set number of tries to update ClamAV with new cvd signature files
   numberoftries=5
################################# END OF VARIABLES

############################### begin of functions
#   function to display help
function displayhelp {
  if [ "$1" != "-q" ] && [ "$1" != "-qall" ]
     then
     echo "NAME: $routinename"
     echo "  Updates Clamav cvd signature files only if not current."
     echo "options:"
     echo "   -q    sub scripts quiet only"
     echo "   -qall all scripts quiet"
     exit 0
   fi
}

#   function to display messages without date
function display {
  if [ $quietall == 0 ];then echo "$1";fi
}

#   function to display messages with date
function displaywdate {
   if [ $quietall == 0 ]
      then
      TODAY=$(date +"%Y%m%d %T")
      echo "$TODAY $1"
   fi
}

############################### end of functions

############################### start of main script

   routinename=$(basename $(readlink -nf $0))

   /usr/bin/renice 20 -p $$ > /dev/null

   if [ ! -z $1 ];then displayhelp $1;fi
   if [ "$1" == "-q" ];then quiet=1;fi
   if [ "$1" == "-qall" ];then quietall=1;quiet=1;fi

   quietoption=""
   if [ $quiet == 1 ] || [ $quietall == 1 ]
      then
      quietoption="-q"
   fi

   xxexitcode=0
   clamavstatus=0
   until [ $numberoftries -lt 1 ]; do
      /opt/myscripts/clamcurrentck $quietoption
      clamavstatus=$?
      if [ $clamavstatus == 1 ] || \
         [ $clamavstatus == 2 ]
         then xxexitcode=3
         break
      fi
      /opt/myscripts/clamcvdget $quietoption
      if [ $? == 1 ]
         then
         let xxexitcode=1
         let numberoftries=0
         break
         else
         let numberoftries-=1
      fi
    sleep 180
   done

   if [ $xxexitcode == 1 ]
      then
         displaywdate "$routinename updated the ClamAV virus signatures."
      else
      if [ $clamavstatus == 1 ] || [ $clamavstatus == 2 ]
         then
         displaywdate "$routinename found the ClamAV virus signatures current."
         xxexitcode=1
         else
         /opt/myscripts/clamcurrentck
         if [ $? == 1 ];then xxexitcode=1;fi
         if [ $? == 2 ];then xxexitcode=1;fi
      fi
   fi
 
exit $xxexitcode



clamcurrentck

Code: [Select]

#!/bin/bash


###
###  clamcurrentck is bash script name
###  date 04-02-2013 13:17:00
###  this bash routine will check to see if the current clamav virus
###  database is the most current from the internet site
###

########################################  VARIABLES THAT NEED SETTING
###
###  command to provie version of clamav service
###  this variable is set to "clamd -V" by default
     commandtocheckclamavservice="clamd -V"
###  command to provide the current version from clamav internet site
     commandtocheckclamavsite="host -t txt current.cvd.clamav.net"

### set outputmsg to  1 to display messages in the routine
     outputmsg=1
### set outputmsg to  1 to display messages in the routine
     outputerr=1
### set quiet to 1 to not display any messagess
     quiet=0

################################################ END OF VARIABLES

################################################ START OF FUNCTIONS

###  function to display help
function displayhelp {
  if [ $1 != "-q" ]
     then
     echo "NAME: $routinename"
     echo "  Compare ClamAV service cvd signature version to ClamAV website"
     echo "options:"
     echo "   -q   quiet no output"
     exit 0
  fi
}



###  function to display messages without a date
function display {
  if [ $outputmsg == 1 ];then echo  "$1";fi
}
function displayerr {
  if [ $outputerr == 1 ];then echo  "$1" >&2;fi
}

function parseiteminstr () {
   local i=0
   local x=0
   local arr2=""
   local itemnumber=""
   parseiteminstring=""
 
   if [ -z "$1" ] || [ -z "$2" ] || [ -z "$3" ];then return;fi
   OIFS=$IFS;IFS="$2"
   i=0
   itemnumber="$3"
   arr2="$1"
   for x in $arr2
      do
         let i+=1
         if [ $i == $itemnumber ];then parseiteminstring=$x;fi
      done
   IFS=$OIFS
}

###  function returns a value if the clamav db is current
###  0 is not current and  1 is current and 2 is current

function isclamdbversioncurrent () {
   local tempstring=""
   local clamdbversionfromservice=""
   local clamdbversionfromsite=""
   clamdbversionuptodate=0;parseiteminstring=""

   tempstring=$($commandtocheckclamavservice)
   parseiteminstr "$tempstring" "/" "2"
   clamdbversionfromservice=$parseiteminstring
###  the next 4 lines are for testing the return variable

###  print length of variable
   #display $parseiteminstring
###  print the parseiteminstring variable length
   #display  $(expr length "$parseiteminstring")

   clamdbversionservice=$parseiteminstring
   parseiteminstring=""
   tempstring=""
   tempstring=$(echo $($commandtocheckclamavsite) | tr -d '"')
   parseiteminstr "$tempstring"  ":" "3"
   clamdbversionfromsite=$parseiteminstring

   if [ -z "$clamdbversionfromsite" ];then  let clamdbversionuptodate-=2;fi
   if [ -z "$clamdbversionfromservice" ];then let clamdbversionuptodate-=1;fi
   if [ $clamdbversionuptodate -ne 0 ];then return 0;fi
   if [ "$clamdbversionfromservice" == "$clamdbversionfromsite" ]
      then
       let clamdbversionuptodate=1
   fi
   if [ $clamdbversionuptodate -eq 0 ]
      then
      if [ ${clamdbversionfromservice#0} -gt ${clamdbversionfromsite#0} ]
         then
         let clamdbversionuptodate=2
      fi
   fi
   if [ $clamdbversionuptodate -ge 0 ]
      then
      display "Clamav database version from service is $clamdbversionfromservice"
      display "Clamav database version from website is $clamdbversionfromsite"
fi
}

################################################ END OF FUNCTIONS

################################################ START OF MAIN SCRIPT


   routinename=$(basename $(readlink -nf $0))

   if [ ! -z $1 ]; then
      displayhelp $1
   fi

   /usr/bin/renice 20 -p $$ > /dev/null


### be quiet if -q is on the command tail
   if [ "$1" == "-q" ]
      then
      quiet=1
   fi

   if [ $quiet == 1 ]
      then
      outputmsg=0
      outputerr=0
   fi

   display "$routinename bash routine started"

   clamdbversionuptodate=0
   isclamdbversioncurrent

   case "$clamdbversionuptodate" in
      2)  display "Clamav virus database is up to date"
        ;;
      1)  display "Clamav virus database is up to date"
        ;;
      0)  display "ClamAV virus database is NOT up to date"
        ;;
     -1)  displayerr "Error cannot get ClamAV db version from local command"
        ;;
     -2)  displayerr "Error cannot get ClamAV db version from internet"
       ;;
     -3)  displayerr "Error cannot get ClamAV db version from both command and internet"
       ;;
      *) displayerr  "Error returned value of $clamdbversionuptodate is not defined"
       ;;
   esac

 display "$routinename bash routine ended"
exit $clamdbversionuptodate

################################################ END OF MAIN SCRIPT



[purvis]

clamcvdget1

Code: [Select]

#!/bin/bash

# clamcvdget
# routine created on 03-21-2013 22:00:00
# this routine will update the clamav virus signature datafiles

######################################## START OF VARIABLES
# SET onlyoneinstance TO 1 TO FORCE ONLY ONE INSTANCE OF THIS
# ROUTINE TO RUN AT A TIME
    onlyoneinstance=1

# SET skipdownload TO 1 WILL SKIP DOWNLOADING OF NEW CVD FILES
# IF FILES NEED TO BE DOWNLOADED THEY WILL BE AUTOMATICALLY
# FOR TESTING PURPOSES
    skipdownload=0

# SET dovirustest TO 1 WILL CAUSE A SHORT CLAMSACN VIRUS
#   TEST TO DISPLAY SUMMARY AND KNOWN VIRUS COUNT
# USUALLY THIS IS ALWAYS SET TO 0 IN A PRODUCTON EVIRONMENT
    doclamtest=0

# SET showversion TO 1 TO DISPLAY VERSION OF CLAMD AND CLAMSCAN
    showversion=1

# SET outputmsg TO 1 TO DISPLAY MESSAGES IN ROUTINE
# IF  outputerr IS SET TO 1 THEN outputmsg WILL BE SET TO 1
    outputmsg=1

# SET outputerr TO 1 TO DISPLAY ERROR MESSAGES IN ROUTINE
# IF YOU outputerr to 1  THEN outputmsg WILL BE SET TO 1
    outputerr=1

# SET quiet TO 1 TO NOT DISPLAY ANY MESSAGES AND DO NO TEST
    quiet=0

# SET runonlyattime FOR THE ROUTINE TO WAIT FOR A SPECFIC TIME
# IF YOU SET runonlyattime TO 1 YOU MUST SET runathourminsec
#    AND SET onlyoneinstance TO 1 DURING PRODUCTION USE
    runonlyattime=0

# SET runathourminsec TO "HH:MM:SS" FOR THE TIME OF TODAY
# THE TIME OF DAY IS ONLY FOR TODAY AND NO OTHER DATES
# THE ROUTINE HAS TO START BEFORE THIS TIME OF THE DAY
# IF THE TIME HAS PASSED THEN THIS PROCESS WILL ABORT
# runathoursminsec VARIABLE WILL NOT BE CHECKED FOR CORRECTNESS
    runathourminsec="03:30:00"

# SET stopclamdsevice TO 1 WILL STOP THE CLAMAV SERVICE WHILE UPDATING
# IT IS NOT NECESSARY TO STOP THE CLAMAV SERVICE AND STOPPING
# COULD POSSIBLY CAUSE CONFLICT PROBLEMS WITH PROGRAMS NEEDING CLAMAV
# IT IS RECOMMENDED TO LEAVE THIS AT 0
    stopclamdservice=0

# THIS PROGRAM WILL TRY TO SEE IF CLAMAV IS BEING USED BY OTHER
#   ROUTINES.  SETTING THESE TO A LOWER NUMBER ON A HEAVILY USED
#   SERVER THAT MAKES CONSTAT USE OF THE CLAMAV SERVICE MAYBE NECESSARY
# SET checkclamactiveseconds TO SECONDS TO CHECK FOR ACTIVE
#    CLAMDSCAN OR CLAMD SCAN RUNNING 30 IS GOOD
    checkclamactiveseconds=30

# number of seconds between failures trying to download cvd files
    secondsbetweendownloadtries=90

#######################

####################### THESE VARIABLES MUST BE SET CORRECTLY
  downloadurl="db.us.clamav.net"
  file1="main.cvd"
  file2="daily.cvd"
  file3="bytecode.cvd"
  clamdir="/var/clamav"
  downloaddirectory=""
  downloaddirectory1='/var/clamav/download/1'
  downloaddirectory2='/var/clamav/download/2'
################################################ END OF VARIABLES



############################### begin of functions

#   function to display help
function displayhelp {
  if [ $1 != "-q" ]
     then
     echo "NAME: $routinename"
     echo "  Updates Clamav cvd signature files from the internet"
     echo "options:"
     echo "   -q   quiet"
     exit 0
   fi
}


#   FUNCTION TO DISPLAY MESSAGES WITH DATE
function displaywdate {
   if [ $outputmsg == 1 ]
      then
      TODAY=$(date +"%Y%m%d %T")
      echo "$TODAY $1"
   fi
}

function displaywdateerr {
   if [ $outputerr == 1 ]
      then
      TODAY=$(date +"%Y%m%d %T")
      echo "$TODAY $1"
   fi
}

#   FUNCTION TO DISPAY MESSAGES WITHOUT DATE
function display {
  if [ $outputmsg == 1 ];then echo "$1";fi
}

function displayerr {
  if [ $outputerr == 1 ];then echo "$1" >&2;fi
}



#   FUNCTION TO COMPARE DOWNLOADED CVD FILES
function comparecvdfiles {
result="0"
if ! diff \
   "$downloaddirectory1/$file1" \
   "$downloaddirectory2/$file1" > /dev/null
   then
   return 0
fi
if ! diff \
   "$downloaddirectory1/$file2" \
   "$downloaddirectory2/$file2" > /dev/null
   then
   return 0
fi
if ! diff \
   "$downloaddirectory1/$file2" \
   "$downloaddirectory2/$file2" > /dev/null
   then
   return 0
fi
result="1"
}

# FUNCTION TO DOWNLOAD CVD FILES
function downloadcvdfiles {
   result="0"
   cd /tmp
   currentdir=$PWD
   if [ $currentdir != "/tmp" ]
      then
      return 0
   fi
   if [ $downloaddirectory == "" ] | [ $downloaddirectory == "/" ]
      then
      return 0
   fi
   mkdir -p  $downloaddirectory
   chmod 755 $downloaddirectory
   rm -f $downloaddirectory/* > /dev/null
   cd $downloaddirectory
   currentdir=$PWD
   if [ $currentdir == "/tmp" ] | \
      [ $currentdir ==  "/" ]  |  \
      [ $currentdir  == "/root" ]
      then
      return 0
   fi
   displaywdate "download url site is $downloadurl"
   displaywdate "downloading cvd files in $downloaddirectory"
   wget -q $downloadurl/$file3
   wget -q $downloadurl/$file1
   wget -q $downloadurl/$file2
   ### displaywdate "done downloading in $downloaddirectory"
   cd /tmp
   result="1"
}

#FUNCITON TO CHECK FOR EXISTENCE OF CVD FILES
function  dodownloadfilesexist {
   result="0"
   if [ ! -f "$downloaddirectory/$file1" ] | \
      [ ! -f "$downloaddirectory/$file2" ] | \
      [ ! -f "$downloaddirectory/$file3" ]
      then
      return 0
      else
      result="1"
   fi
}
################################## end of functions

routinename=$(basename $(readlink -nf $0))

if [ $onlyoneinstance == 1 ]
   then
   processname=$(basename $(readlink -nf $0))   
   if [ $(pidof -x $processname | wc -w) -gt 2 ]
      then
      exit 0
   fi
fi

/usr/bin/renice 20 -p $$ > /dev/null
if [ ! -z $1 ]; then
   displayhelp $1
fi

if [ "$1" == "-q" ];then quiet=1;fi
if [ $outputerr == 1 ];then  outputmsg=1;fi
if [ $quiet == 1 ]
  then
  doclamtest=0
  showversion=0
  outputmsg=0
  outputerr=0
  quiet=1
fi

displaywdate "$routinename bash routine started"

if [ $runonlyattime == 1 ]
   then
   current_epoch=$(date +%s)
   target_epoch=$(date -d $runathourminsec +%s)
   sleep_seconds=$(($target_epoch - $current_epoch))
   displaywdateerr "This routine is set to run at $(date -d @$target_epoch +"%m-%d-%Y %T")"

   if [ $sleep_seconds -lt 0 ]
      then
      displaywdate "That time has already past."
      displaywdate "Aborting $routinename bash routine"
      exit 0
      else
      displaywdate "Waiting on time before continuing"
      sleep $sleep_seconds
      displaywdate "$routinename bash routine continuing now"
   fi
fi


# DOWNLOAD FIRST SET OF CVD FILES IF NOT EXIST
downloaddirectory=$downloaddirectory1
if [ $skipdownload == 0 ]
   then
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   displaywdate "cvd files missing in $downloaddirectory"
   displaywdate "will try to download in $secondsbetweendownloadtries seconds"
   sleep $secondsbetweendownloadtries
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   displaywdate "cvd files missing in $downloaddirectory"
   displaywdate "will try to download in $secondsbetweendownloadtries seconds"
   sleep $secondsbetweendownloadtries
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   displaywdate "cvd files missing in $downloaddirectory"
   displaywdate "will try to download in $secondsbetweendownloadtries seconds"
   sleep $secondsbetweendownloadtries 
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   displaywdateerr "routine $routinename"
   displaywdateerr "Update cvd files do not exist"
   displaywdateerr "in the directory of $downloaddirectory."
   displaywdateerr  "Cannot update Clamav. Aborting update routine."
   exit 0
fi

# DOWNLOAD SECOND SET OF CVD FILES IF NOT EXIST
downloaddirectory=$downloaddirectory2
dodownloadfilesexist
if [ $result == "0" ]
   then
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   displaywdate "cvd files missing in $downloaddirectory"
   displaywdate "will try to download in $secondsbetweendownloadtries seconds"
   sleep $secondsbetweendownloadtries
   downloadcvdfiles
fi
dodownloadfilesexist
if [ $result == "0" ]
   then
   displaywdate "cvd files missing in $downloaddirectory"
   displaywdate "will try to download in $secondsbetweendownloadtries seconds"
   sleep $secondsbetweendownloadtries
   downloadcvdfiles
fi
if [ $result == "0" ]
   then
   if [ $outputerr == 1 ]
      then
      displaywdateerr "routine $routinename" 
      displaywdateerr "Update cvd files do not exist"
      displaywdateerr "  in the directory of $downloaddirectory."
      displaywdateerr "Cannot update Clamav. Aborting update routine."
   fi
   exit 0
fi


comparecvdfiles
if [ $result == "0" ]
   then
   downloaddirectory=$downloaddirectory2
   dodownloadfilesexist
  if [ $result == "0" ]
      then
      downloadcvdfiles
   fi
  if [ $result == "0" ]
      then
      displaywdateerr "cvd files missing in $downloaddirectory"
      displaywdateerr "will try to download in $secondsbetweendownloadtries seconds"
      sleep $secondsbetweendownloadtries
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      displaywdateerr "cvd files missing in $downloaddirectory"
      displaywdateerr "will try to download in $secondsbetweendownloadtries seconds"
      sleep $secondsbetweendownloadtries
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      if [ $outputerr == 1 ]
         then
         displaywdateerr "routine $routinename"
         displaywdateerr "Update cvd files do not exist"
         displaywdateerr "  in the directory of $downloaddirectory."
         displaywdateerr "Cannot update Clamav. Aborting update routine."
       fi
      exit 0
   fi
fi

# COMPARE THE TWO SETS OF CVD FILES
# IF THE FILES ARE NOT EQUAL THEN DOWNLOAD THE SECOND SET
comparecvdfiles
if [ $result == "0" ]
   then
   downloaddirectory=$downloaddirectory2
   downloadcvdfiles
   dodownloadfilesexist
  if [ $result == "0" ]
      then
      displaywdateerr "cvd files missing in $downloaddirectory"
      displaywdateerr "will try to download in $secondsbetweendownloadtries seconds"
      sleep $secondsbetweendownloadtries
      downloadcvdfiles
   fi
  if [ $result == "0" ]
      then
      displaywdateerr "cvd files missing in $downloaddirectory"
      displaywdateerr "will try to download in $secondsbetweendownloadtries seconds"
      sleep $secondsbetweendownloadtries
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      displaywdateerr "cvd files missing in $downloaddirectory"
      displaywdateerr "will try to download in $secondsbetweendownloadtries seconds"
      sleep $secondsbetweendownloadtries
      downloadcvdfiles
   fi
   dodownloadfilesexist
   if [ $result == "0" ]
      then
      if [ $outputerr == 1 ]
         then
         displaywdateerr "routine $routinename"
         displaywdateerr "Update cvd files do not exist"
         displaywdateerr "  in the directory of $downloaddirectory."
         dispalywdateerr "Cannot update Clamav. Aborting update routine."
      fi
   exit 0
   fi
fi

# COMPARE THE TWO SETS OF CVD FILES FOR A SECOND AND FINAL TRY
# IF THE TWO SETS ARE NOT EQUAL THEN THE PROGRAM ABORTS
comparecvdfiles
if [ $result == "0" ]
then
if [ $outputerr == 1 ]
      then
      displaywdateerr "Update cvd files do not exist"
      displaywdateerr "Update cvd files are not complete"
      displaywdateerr "Cannot update ClamAV"
   fi
exit 0
fi




# UPDATING THE CLAMAV SIGNATURE CVD FILES

# MAKE AN EFFORT TO CHECK IF CLAMSCAN IS RUNNING
# TEST TO MAKE SURE CLAMSCAN IS NOT RUNNING
#   FOR 15 SECONDS FIRST THEN IF CLAMSCAN IS
#   RUNNING TEST ONCE EVERY 15 SECONDS THERE AFTER
# WHEN CLAMSCAN HAS NOT BEEN RUNNING FOR 1
#   MINUTE THEN PROCEED TO UPDATE CLAMAV

displaywdate "Verifying or waiting on freed up Clamav processes"

let counter=$checkclamactiveseconds*4
until [ $counter -lt 1 ]; do
      if [ $(pidof -x "clamscan" | wc -w) -gt 0 ]  || \
         [ $(pidof -x "clamdscan" | wc -w) -gt 0 ] || \
         [ $(pidof -x "freshclam" | wc -w) -gt 1 ]
         then
         let counter=$checkclamactiveseconds*4
         sleep 2
         else
         let counter-=1
         sleep .25
      fi
done

#STOPPING THE FRESHCLAM SERVICE
displaywdate "stopping freshclam service"
service freshclam stop > /dev/null
counter=1
until [ $counter -lt 1 ]; do
      if [ $(pidof -x "freshclam" | wc -w) -gt 0 ]
         then
         sleep 2
         else
         displaywdate "freshclam service stopped"
         counter=0
      fi
done


#STOPPING THE CLAMD SERVICE
if [ $stopclamdservice == 1 ]
   then
   displaywdate "stopping clamd service"

service clamd stop > /dev/null
counter=1
until [ $counter -lt 1 ]; do
      if [ $(pidof -x "clamd" | wc -w) -gt 0 ]
         then
         sleep 2
         else
         displaywdate "clamd service stopped"
          counter=0
      fi
done
fi

#COPYING THE DOWNLOADED CLAMAV CVD FILES TO
#THE CLAMAV CVD WORKING DIRECTORY
displaywdate "updating the Clamav cvd files"
cd $clamdir
rm  -f $clamdir/*.cvd 1> /dev/null 2> /dev/null
rm  -f $clamdir/*.cld 1> /dev/null 2> /dev/null
rm  -f $clamdir/mirrors.dat 1> /dev/null 2> /dev/null
cp -p $downloaddirectory1/*.cvd $clamdir 1> /dev/null 2> /dev/null


#STARTING THE CLAMD SERVICE
if [ $stopclamdservice == 1 ]
   then
   displaywdate "starting clamd service"
   service clamd start > /dev/null
   counter=1
   until [ $counter -lt 1 ]; do
      if [ $(pidof -x "clamd" | wc -w) -gt 1 ]
         then
         sleep .5
         else
         counter=0
         displaywdate "clamd service started"
      fi
done
fi

#RELOADING THE CLAMAV VIRUS DEFINITION FILES IN THE CLAMD SERVICE

/usr/bin/clamdscan $downloaddirectory1/$file3 \
   --no-summary --infected --reload 1>/devnull 2>/dev/null


#STARTING THE FRESHCLAM SERVICE
displaywdate "starting freshclam service"
service freshclam start > /dev/null
counter=1
until [ $counter -lt 1 ]; do
      if [ $(pidof -x "freshclam" | wc -w) -gt 1 ]
         then
         sleep .5
         else
         counter=0
         displaywdate "freshclam service started"
      fi
done

displaywdate "updated Clamav"

# THE SHOW THE VERSION OF CLAMAV
if [ $showversion == 1 ]
   then
   tempstring=$(clamd -V)
   displaywdate   "ClamAV version - $tempstring"
fi

# THE FOLLOWING IS TO JUST RUN A SHORT TEST OF CLAMSCAN
if [ $doclamtest == 1 ] 
   then
   until [ $counter -lt 1 ]; do
      if [ $(pidof -x "clamscan" | wc -w) -gt 0 ]
         then
         counter=60
         sleep 2
         else
         let counter-=1
         sleep .25
      fi
    done
    echo
    echo "Running a short clamscan test on an empty directory."
    echo "No files or viruses should be found in the summary."
    mkdir -p $clamdir/temp
    clamscan $clamdir/temp
    echo
    rm -rf $clamdir/temp
fi

displaywdate "$routinename bash routine ended successfully"
exit 1



[purvis]

clamcvdget2

Code: [Select]

#!/bin/bash

# clamcvdget
# routine created on 04-02-2013 12:10:00
# this routine will update the clamav virus signature datafiles

######################################## START OF VARIABLES
# SET onlyoneinstance TO 1 TO FORCE ONLY ONE INSTANCE OF THIS
# ROUTINE TO RUN AT A TIME
    onlyoneinstance=1

# SET skipdownload TO 1 WILL SKIP DOWNLOADING OF NEW CVD FILES
# IF FILES NEED TO BE DOWNLOADED THEY WILL BE AUTOMATICALLY
# FOR TESTING PURPOSES
    skipdownload=0

# SET dovirustest TO 1 WILL CAUSE A SHORT CLAMSACN VIRUS
#   TEST TO DISPLAY SUMMARY AND KNOWN VIRUS COUNT
# USUALLY THIS IS ALWAYS SET TO 0 IN A PRODUCTON EVIRONMENT
    doclamtest=0

# SET showversion TO 1 TO DISPLAY VERSION OF CLAMD AND CLAMSCAN
    showversion=1

# SET outputmsg TO 1 TO DISPLAY MESSAGES IN ROUTINE
# IF  outputerr IS SET TO 1 THEN outputmsg WILL BE SET TO 1
    outputmsg=1

# SET outputerr TO 1 TO DISPLAY ERROR MESSAGES IN ROUTINE
# IF YOU outputerr to 1  THEN outputmsg WILL BE SET TO 1
    outputerr=1

# SET quiet TO 1 TO NOT DISPLAY ANY MESSAGES AND DO NO TEST
    quiet=0

# SET runonlyattime FOR THE ROUTINE TO WAIT FOR A SPECFIC TIME
# IF YOU SET runonlyattime TO 1 YOU MUST SET runathourminsec
#    AND SET onlyoneinstance TO 1 DURING PRODUCTION USE
    runonlyattime=0

# SET runathourminsec TO "HH:MM:SS" FOR THE TIME OF TODAY
# THE TIME OF DAY IS ONLY FOR TODAY AND NO OTHER DATES
# THE ROUTINE HAS TO START BEFORE THIS TIME OF THE DAY
# IF THE TIME HAS PASSED THEN THIS PROCESS WILL ABORT
# runathoursminsec VARIABLE WILL NOT BE CHECKED FOR CORRECTNESS
    runathourminsec="03:30:00"

# SET stopclamdsevice TO 1 WILL STOP THE CLAMAV SERVICE WHILE UPDATING
# IT IS NOT NECESSARY TO STOP THE CLAMAV SERVICE AND STOPPING
# COULD POSSIBLY CAUSE CONFLICT PROBLEMS WITH PROGRAMS NEEDING CLAMAV
# IT IS RECOMMENDED TO LEAVE THIS AT 0
    stopclamdservice=0

# THIS PROGRAM WILL TRY TO SEE IF CLAMAV IS BEING USED BY OTHER
#   ROUTINES.  SETTING THESE TO A LOWER NUMBER ON A HEAVILY USED
#   SERVER THAT MAKES CONSTAT USE OF THE CLAMAV SERVICE MAYBE NECESSARY
# SET checkclamactiveseconds TO SECONDS TO CHECK FOR ACTIVE
#    CLAMDSCAN OR CLAMD SCAN RUNNING 30 IS GOOD
    checkclamactiveseconds=30

# number of seconds between failures trying to download cvd files
    secondsbetweendownloadtries=60

#######################

####################### THESE VARIABLES MUST BE SET CORRECTLY
### set the locations where the cvd files will be downloaded from
### local can be substutied can be any of these abbeviations found at
### this webstite http://www.clamav.net/mirrors.html
### here are the current local abbreivations as of the date at the top
### at au ba be br by ca ch cn cz de dk ee es fr gl gr hk hu id
### ie in it jp lt lv mt nl no pl pt ro ru se sg si sk th tr tw ua uk us za
### there needs to at least 1 location
### the number of locations are unlimited and will be used in reverse order
  declare -a downloadsite=(  \
     db.us.clamav.net        \
     db.local.clamav.net     \
     db.us.clamav.net        \
     db.local.clamav.net     \
     db.us.clamav.net        \
     db.local.clamav.net     \
     db.us.clamav.net        \
     db.xx.clamav.net        \
     )

  file1="main.cvd"
  file2="daily.cvd"
  file3="bytecode.cvd"
  clamdir="/var/clamav"
  downloaddirectory=""
  downloaddirectory='/var/clamav/download'
################################################ END OF VARIABLES



############################### begin of functions

#   function to display help
function displayhelp {
  if [ $1 != "-q" ]
     then
     echo "NAME: $routinename"
     echo "  Updates Clamav cvd signature files from the internet"
     echo "options:"
     echo "   -q   quiet"
     exit 0
   fi
}


#   FUNCTION TO DISPLAY MESSAGES WITH DATE
function displaywdate {
   local TODAY=""
   if [ $outputmsg == 1 ]
      then
      TODAY=$(date +"%Y%m%d %T")
      echo "$TODAY $1"
   fi
}

function displaywdateerr {
   local TODAY=""
   if [ $outputerr == 1 ]
      then
      TODAY=$(date +"%Y%m%d %T")
      echo "$TODAY $1"
   fi
}

#   FUNCTION TO DISPAY MESSAGES WITHOUT DATE
function display {
  if [ $outputmsg == 1 ];then echo "$1";fi
}

function displayerr {
  if [ $outputerr == 1 ];then echo "$1" >&2;fi
}

# function to test cvd files downloaded
function testcvdfiles {
local strikeout=0
local tempcount=""
result="0"

if [ ! -f $downloaddirectory/$file1 ];then let strikeout+=1;fi
if [ -f $downloaddirectory/$file1 ]
   then
   tempstring=""
   tempstring=$(sigtool --info=$downloaddirectory/$file1)
   sleep 2
   if [ -z "$tempstring" ]
      then
      let strikeout+=1
      else
      tempcount=""
      tempcount==$(echo $tempstring |grep -P -i -c "verification\s+ok")
      if [ "$tempcount" == "=0" ] || [ "$tempcount" == "" ]
         then
         let strikeout+=1
      fi
   fi
fi
sleep 1

if [ ! -f $downloaddirectory/$file2 ];then let strikeout+=1;fi
if [ -f $downloaddirectory/$file2 ]
   then
   tempstring=""
   tempstring=$(sigtool --info=$downloaddirectory/$file2)
   sleep 2
   if [ -z "$tempstring" ]
      then
      let strikeout+=1
      else
      tempcount=""
      tempcount==$(echo $tempstring |grep -P -i -c "verification\s+ok")
      if [ "$tempcount" == "=0" ] || [ "$tempcount" == "" ]
         then
         let strikeout+=1
      fi
   fi
fi

if [ ! -f $downloaddirectory/$file3 ];then let strikeout+=1;fi
if [ -f $downloaddirectory/$file3 ]
   then
   tempstring=""
   tempstring=$(sigtool --info=$downloaddirectory/$file3)
   sleep 2
   if [ -z "$tempstring" ]
      then
      let strikeout+=1
      else
      tempcount=""
      tempcount==$(echo $tempstring |grep -P -i -c "verification\s+ok")
      if [ "$tempcount" == "=0" ] || [ "$tempcount" == "" ]
         then
         let strikeout+=1
      fi
   fi
fi

if [ $strikeout -eq 0 ]
   then
   result="1"
fi
}



# FUNCTION TO DOWNLOAD CVD FILES
function downloadcvdfiles {
   result="0"
   cd /tmp
   currentdir=$PWD
   if [ $currentdir != "/tmp" ]
      then
      return 0
   fi
   if [ $downloaddirectory == "" ] | [ $downloaddirectory == "/" ]
      then
      return 0
   fi
   mkdir -p  $downloaddirectory
   chmod 755 $downloaddirectory
   rm -f $downloaddirectory/$file1 2> /dev/null
   rm -f $downloaddirectory/$file2 2> /dev/null
   rm -f $downloaddirectory/$file3 2> /dev/null
   rm -f $downloaddirectory/$file1* 2> /dev/null
   rm -f $downloaddirectory/$file2* 2> /dev/null
   rm -f $downloaddirectory/$file3* 2> /dev/null

 
 cd $downloaddirectory
   currentdir=$PWD
   if [ $currentdir == "/tmp" ] | \
      [ $currentdir ==  "/" ]  |  \
      [ $currentdir  == "/root" ]
      then
      return 0
   fi
   displaywdate "download url site is $downloadurl"
   displaywdate "downloading cvd files in $downloaddirectory"
   wget -q $downloadurl/$file3
   if [ ! -f "$downloaddirectory/$file3" ]
      then
      displaywdate "download failed for file $file3"
      displaywdate "aborting this download instance"
      return 0
   fi
   wget -q $downloadurl/$file1
   if [ ! -f "$downloaddirectory/$file1" ]
      then
      displaywdate "download failed for file $file1"
      displaywdate "aborting this download instance"
      return 0
   fi
   wget -q $downloadurl/$file2
     if [ ! -f "$downloaddirectory/$file2" ]
      then
      displaywdate "download failed for file $file2"
      displaywdate "aborting this download instance"
      return 0
   fi
   ### displaywdate "done downloading in $downloaddirectory"
   cd /tmp
   result="1"
}

#FUNCITON TO CHECK FOR EXISTENCE OF CVD FILES
function  dodownloadfilesexist {
   result="0"
   if [ ! -f "$downloaddirectory/$file1" ] | \
      [ ! -f "$downloaddirectory/$file2" ] | \
      [ ! -f "$downloaddirectory/$file3" ]
      then
      return 0
      else
      result="1"
   fi
}
################################## end of functions

routinename=$(basename $(readlink -nf $0))

if [ $onlyoneinstance == 1 ]
   then
   processname=$(basename $(readlink -nf $0))   
   if [ $(pidof -x $processname | wc -w) -gt 2 ]
      then
      exit 0
   fi
fi

/usr/bin/renice 20 -p $$ > /dev/null
if [ ! -z $1 ]; then
   displayhelp $1
fi

if [ "$1" == "-q" ];then quiet=1;fi
if [ $outputerr == 1 ];then  outputmsg=1;fi
if [ $quiet == 1 ]
  then
  doclamtest=0
  showversion=0
  outputmsg=0
  outputerr=0
  quiet=1
fi

displaywdate "$routinename bash routine started"

if [ $runonlyattime == 1 ]
   then
   current_epoch=$(date +%s)
   target_epoch=$(date -d $runathourminsec +%s)
   sleep_seconds=$(($target_epoch - $current_epoch))
   displaywdateerr "This routine is set to run at $(date -d @$target_epoch +"%m-%d-%Y %T")"

   if [ $sleep_seconds -lt 0 ]
      then
      displaywdate "That time has already past."
      displaywdate "Aborting $routinename bash routine"
      exit 0
      else
      displaywdate "Waiting on time before continuing"
      sleep $sleep_seconds
      displaywdate "$routinename bash routine continuing now"
   fi
fi




cvdfilesgood=0
counter=${#downloadsite[@]}
let counter-=1
until [ $counter -lt 0 ]; do
     downloadurl=${downloadsite[$counter]}
     downloadcvdfiles
     dodownloadfilesexist
     if [ $result == "0" ]
        then
        displaywdateerr "cvd files missing in $downloaddirectory"
        displaywdateerr "will retry download in $secondsbetweendownloadtries seconds"
        sleep $secondsbetweendownloadtries
        let counter-=1
     else
        testcvdfiles
        if [ $result == "1" ]
           then
           let counter=-1
           cvdfilesgood=1
        fi
     fi
done

dodownloadfilesexist
   if [ $result == "0" ]
    then
    displaywdateerr "routine $routinename"
    displaywdateerr "Update cvd files do not exist"
    displaywdateerr "in the directory of $downloaddirectory."
    displaywdateerr  "Cannot update Clamav. Aborting update routine."
    exit 0
  fi

if [ $cvdfilesgood -eq 0 ]
   then
   displaywdateerr "routine $routinename"
   displaywdateerr "Cvd files do not verify proper"
   displaywdateerr "in the directory of $downloaddirectory."
   displaywdateerr  "Cannot update Clamav. Aborting update routine."
   exit 0
fi


# UPDATING THE CLAMAV SIGNATURE CVD FILES

# MAKE AN EFFORT TO CHECK IF CLAMSCAN IS RUNNING
# TEST TO MAKE SURE CLAMSCAN IS NOT RUNNING
#   FOR 15 SECONDS FIRST THEN IF CLAMSCAN IS
#   RUNNING TEST ONCE EVERY 15 SECONDS THERE AFTER
# WHEN CLAMSCAN HAS NOT BEEN RUNNING FOR 1
#   MINUTE THEN PROCEED TO UPDATE CLAMAV

displaywdate "Verifying or waiting on freed up Clamav processes"

let counter=$checkclamactiveseconds*4
until [ $counter -lt 1 ]; do
      if [ $(pidof -x "clamscan" | wc -w) -gt 0 ]  || \
         [ $(pidof -x "clamdscan" | wc -w) -gt 0 ] || \
         [ $(pidof -x "freshclam" | wc -w) -gt 1 ]
         then
         let counter=$checkclamactiveseconds*4
         sleep 2
         else
         let counter-=1
         sleep .25
      fi
done

#STOPPING THE FRESHCLAM SERVICE
displaywdate "stopping freshclam service"
service freshclam stop > /dev/null
counter=1
until [ $counter -lt 1 ]; do
      if [ $(pidof -x "freshclam" | wc -w) -gt 0 ]
         then
         sleep 2
         else
         displaywdate "freshclam service stopped"
         counter=0
      fi
done

#STOPPING THE CLAMD SERVICE
if [ $stopclamdservice == 1 ]
   then
   displaywdate "stopping clamd service"

service clamd stop > /dev/null
counter=1
until [ $counter -lt 1 ]; do
      if [ $(pidof -x "clamd" | wc -w) -gt 0 ]
         then
         sleep 2
         else
         displaywdate "clamd service stopped"
          counter=0
      fi
done
fi

#COPYING THE DOWNLOADED CLAMAV CVD FILES TO
#THE CLAMAV CVD WORKING DIRECTORY
displaywdate "updating the Clamav cvd files"
cd $clamdir
rm  -f $clamdir/*.cvd 1> /dev/null 2> /dev/null
rm  -f $clamdir/*.cld 1> /dev/null 2> /dev/null
rm  -f $clamdir/mirrors.dat 1> /dev/null 2> /dev/null
cp -p $downloaddirectory/*.cvd $clamdir 1> /dev/null 2> /dev/null


#STARTING THE CLAMD SERVICE
if [ $stopclamdservice == 1 ]
   then
   displaywdate "starting clamd service"
   service clamd start > /dev/null
   counter=1
   until [ $counter -lt 1 ]; do
      if [ $(pidof -x "clamd" | wc -w) -gt 1 ]
         then
         sleep .5
         else
         counter=0
         displaywdate "clamd service started"
      fi
done
fi

#RELOADING THE CLAMAV VIRUS DEFINITION FILES IN THE CLAMD SERVICE

/usr/bin/clamdscan $downloaddirectory/$file3 \
   --no-summary --infected --reload 1>/devnull 2>/dev/null


#STARTING THE FRESHCLAM SERVICE
displaywdate "starting freshclam service"
service freshclam start > /dev/null
counter=1
until [ $counter -lt 1 ]; do
      if [ $(pidof -x "freshclam" | wc -w) -gt 1 ]
         then
         sleep .5
         else
         counter=0
         displaywdate "freshclam service started"
      fi
done

displaywdate "updated Clamav"

# THE SHOW THE VERSION OF CLAMAV
if [ $showversion == 1 ]
   then
   tempstring=$(clamd -V)
   displaywdate   "ClamAV version - $tempstring"
fi

# THE FOLLOWING IS TO JUST RUN A SHORT TEST OF CLAMSCAN
if [ $doclamtest == 1 ] 
   then
   until [ $counter -lt 1 ]; do
      if [ $(pidof -x "clamscan" | wc -w) -gt 0 ]
         then
         counter=60
         sleep 2
         else
         let counter-=1
         sleep .25
      fi
    done
    echo
    echo "Running a short clamscan test on an empty directory."
    echo "No files or viruses should be found in the summary."
    mkdir -p $clamdir/temp
    clamscan $clamdir/temp
    echo
    rm -rf $clamdir/temp 2> /dev/null
fi

displaywdate "$routinename bash routine ended successfully"
exit 1



[purvis]

Late yesterday doing some final testing.
I found out that it is possible that the current clamav datebase version number retrieved from a clamav website(host -t txt current.cvd.clamav.net) maybe be less than the version of the currently loaded database in the ClamAV service being run.
I would just never had expected that.
I figured the clamav website would of always been up to date and provided a higher version number or equal number to ClamAV service.
But never a lower version number.

The script clamcurrentck only compares the two version numbers in string format that are returned from the ClamAV service and the website.
If the two strings do not match, the clamcurrentck will return they do not match.
In the clamavupdate script, if a returned value from the clamcurrentck script indicates there is no match, then clamavdget is run to update the cvd files even if the most current cvd files had already been updated.

In order to improve the scripts and not cause unnecessary updates.
I will make a change to both clamcvdupdate and clamcurrentck scripts.
I will have clamcurrentck return a newly added value of 2 if the running ClamAV service reports a larger version number than the returned version from the ClamAV website.
Right now the returned value from clamcurrentck would be a 0(zero).

I am hoping to just update the posting of scripts above and I will make a posting they where updated.

[purvis]

The changes where made to the above scripts.

While I was making those changes I posted on the previous post.
I also made some improvement to downloading that occurred to me over night to clamcvdget2.
Now clamcvdget2 can have an unlimited number of url sites to be added to try in downloading the cvd files.

With an unlimited number of url's to be added, then the variable secondsbetweendownloadtries maybe lowered if you have a long list of urls to be used.
You could have something like 1 set of urls duplicated about 4 or more times then set secondsbetweendownloadtries to equal something like 15 seconds or less.

Inside clamcvdget2 just place the urls to be used in reverse order near the top under the line

Code: [Select]

declare -a downloadsite=(  \

Also when the three cvd files are trying to be downloaded from a url site, any file not downloaded will break the effort to download any remaining cvd files.
This process will help speed up the retrying a download of the cvd files from the next url location.

[Drifting]

Have I missed the point here ? I thought SME server did that on it's own?

[purvis]

Freshclam apparently is having issues.
This is a work around.
I am running this on all my servers with no issues to keep the databases up to date.
Nothing is perfect and things change. One day, this code may not work either.