Koozali.org: home of the SME Server
Obsolete Releases => SME Server 7.x => Topic started by: Jontu Kontar on December 17, 2008, 06:47:37 PM
-
Info:
1 Hard Drive: Contains everything except iBays and User files.
2 Hard Drives (Raid 0): Contains the iBay and User files. (soft linked to their normal location).
In it's default configuration, /usr/bin/clamscan ignores all symlinks and thus ignores all the files that need to be scanned. It appears that there is an option "FollowDirectorySymlinks" and "FollowFileSymlinks" that can be added to /etc/clamd.conf. Altering that is counter-productive, in that the server scan ignores it and it interferes with the virus scanning of e-mail. Editing /sbin/e-smith/smeserver-clamscan to add an option to follow the symlinks doesn't seem to be possible because no such switch exists for /usr/bin/clamscan. Creating a custom configuration also doesn't seem to be possible.
So that leaves me with the possibility of adding or changing the location where /usr/bin/clamscan is directed to check. Looking at /sbin/e-smith/smeserver-clamscan it appears that a database is selected and then the file systems are populated by querying for "clamav" and "FilesystemScanFilesystems" (which returns /home/e-smith/files).
In the end, I'm looking for a way to scan beyond the symlink that will be preserved if the system is updated or upgraded. I'm left with two possibilities: edit /sbin/e-smith/smeserver-clamscan to add the additional file system or edit the database that returns the file system. Which would be the better way to go? In the case of the database, how to you edit that?
-
Don't use symlinks. Mount your extra drive/file system at /home/e-smith/files.
-
While for Jontu I think Charlie's answer is the best, I'm looking at being more selective about which directories go onto my other disks. So I was planning to symlink only selected portions of my hierarchy.
The obvious question beyond finding a solution to this is... are there any other gotchas?
If it gets too whacky then I will likely admit defeat and simply mount the spare drive in place of my most dominant ibay.
Now for clam, I think one way is to start searching from "/" and then exclude a tonne of directories as it doesn't appear clam can take multiple file systems as an argument with the prop "FilesystemScanFilesystems".
The docs (http://wiki.contribs.org/SME_Server:Documentation:FAQ#Virus_Scanning) says you can and the code appears to allow it but I think clam itself won't take it.
Another way may be to create your own version of "smeserver-clamscan" to run on another hierarchy driven by cron.
-
Don't use symlinks. Mount your extra drive/file system at /home/e-smith/files.
I will keep that in mind if I need to do this again. As it stands now, that isn't a practical possibility. Unless you know a way to mount a specific folder on a given device?
The docs (http://wiki.contribs.org/SME_Server:Documentation:FAQ#Virus_Scanning) says you can and the code appears to allow it but I think clam itself won't take it.
Thanks for the link. It might work if the comma isn't returned with the values (as per the link). Clam takes space delimited directories as paths along which to scan. Which brings up the question as to whether the comma is required when inputting the information?
Another way may be to create your own version of "smeserver-clamscan" to run on another hierarchy driven by cron.
I could do that, but I know that certain files are replaced when the system is updated. I wouldn't want my change to disappear when that happens.
-
I could do that, but I know that certain files are replaced when the system is updated. I wouldn't want my change to disappear when that happens.
I agree with your concern and wasn't suggesting to replace smeserver-clamscan but to create your own script based on it to run driven by cron for the additional drive only.
However, if in fact the FilesystemScanFilesystems variable can work then that should be used. Can you confirm this? I've seen noise in the forums about it not working (supported by my own read of the clamscan man page); but you believe clamscan can so perhaps it is more about correcting usage method. If you can confirm, I'll ensure the FAQ is updated accordingly.
Once we determine this, I can also update the AddExtraDisk How-to point to this as an additional optional stop.
-
However, if in fact the FilesystemScanFilesystems variable can work then that should be used. Can you confirm this? I've seen noise in the forums about it not working (supported by my own read of the clamscan man page); but you believe clamscan can so perhaps it is more about correcting usage method. If you can confirm, I'll ensure the FAQ is updated accordingly.
Changing that property according to the above linked template doesn't work. That property returns exactly what you type into it (commma and all).
WARNING: Can't access file /home/e-smith/files,/user_files
Changing it to the following seems to work.
config setprop clamav FilesystemScanFilesystems "/home/e-smith/files /user_files"
Well, normally the script is done well within 60 seconds of startup. However with the update, it is continuing to run. The results of ps auxc | grep clamscan reveal the following snippets which seem accurate (or at the least they were expected).
sh -c nice /usr/bin/clamscan --recursive --infected --stdout --log /var/log/clamd/clamscan.log --exclude=/proc --exclude=/sys --exclude=/usr/share --exclude=/var --exclude=/var/spool/clamav/quarantine --move=/var/spool/clamav/quarantine /home/e-smith/files /user_files 2> /var/log/clamd/smeserver-clamscan.log
/usr/bin/clamscan --recursive --infected --stdout --log /var/log/clamd/clamscan.log --exclude=/proc --exclude=/sys --exclude=/usr/share --exclude=/var --exclude=/var/spool/clamav/quarantine --move=/var/spool/clamav/quarantine /home/e-smith/files /user_files
/usr/sbin/lsof -c clamscan reveals the following (which is new but expected behavior):
clamscan 27639 root 16r REG 253,2 13190862 96944676 /user_files/ibays/it_software/files/Utility/Windows/Patch Management/ctupdate4/5.0/client/wsus/wsusscn2.cab
-
That works!
----------- SCAN SUMMARY -----------
Known viruses: 915381
Engine version: 0.94.1
Scanned directories: 5223
Scanned files: 108622
Infected files: 0
Data scanned: 64423.45 MB
Time: 14488.970 sec (241 m 28 s)
-
That works!
Thanks for confirming. I've updated the FAQ and will make reference in AddExtraHardDisk.
Christian
-
Which command shows what clamav is going to scan?
-
Which command shows what clamav is going to scan?
The opposite to the "config setprop" identified in Jontu's comment above.
config getprop clamav FilesystemScanFilesystemsAnd note the excluded directories:
config getprop clamav FilesystemScanExclude
To see all attributes:
config show clamav
note "config" is short for "db configuration"; either will work.
EDIT: See also http://wiki.contribs.org/DB_Variables_Configuration#Clam_AntiVirus_.28clamav.29 which is part of the wiki page: http://wiki.contribs.org/DB_Variables_Configuration
-
Thank you.
-
I got this back this morning after using this.
config setprop clamav FilesystemScanFilesystems "/home/e-smith/files mnt/tracy"
I guess I will try.
config setprop clamav FilesystemScanFilesystems "/home/e-smith/files /mnt/tracy"
Then I do signal-event clamav-update
Question which command makes it do an immediate scan?
WARNING: Can't access file mnt/tracy
----------- SCAN SUMMARY -----------
Known viruses: 922370
Engine version: 0.94.2
Scanned directories: 670
Scanned files: 14334
Infected files: 0
Data scanned: 1285.98 MB
Time: 956.398 sec (15 m 56 s)
-
Question which command makes it do an immediate scan?
Did you try:
/sbin/e-smith/smeserver-clamscan
Jontu makes note of this in his first comment in this thread. I believe that will work.
-
Did you try:
/sbin/e-smith/smeserver-clamscan
Jontu makes note of this in his first comment in this thread. I believe that will work.
I tried that but not adviseable on slow VPN connection. I timed out.
I did get this back this morning.
----------- SCAN SUMMARY -----------
Known viruses: 922398
Engine version: 0.94.2
Scanned directories: 1372
Scanned files: 26513
Infected files: 0
Data scanned: 8648.59 MB
Time: 6246.497 sec (104 m 6 s)
This is telling me it is not scanning the mnt/tracy which has about 40 gb of users files on it.
config getprop clamav FilesystemScanFilesystems is showing
[root@shpdserver ~]# config getprop clamav FilesystemScanFilesystems
/home/e-smith/files /mnt/tracy
I have also tried it as /home/e-smith/files mnt/tracy with same results I'm not understanding something.
[root@shpdserver ~]# config getprop clamav FilesystemScanExclude
/proc,/sys,/usr/share,/var
[root@shpdserver ~]# config show clamav
clamav=service
/opt,=mnt/tracy
ArchiveBlockEncrypted=no
ArchiveBlockMax=no
ArchiveMaxCompressionRatio=300
Checks=24
DNSDatabaseInfo=current.cvd.clamav.net
DatabaseMirror=db.local.clamav.net
Debug=no
DetectBrokenExecutables=no
FilesystemScan=daily
FilesystemScanExclude=/proc,/sys,/usr/share,/var
FilesystemScanFilesystems=/home/e-smith/files /mnt/tracy
FilesystemScanReportTo=admin
Foreground=yes
HTTPProxyPassword=
HTTPProxyPort=
HTTPProxyServer=
HTTPProxyUsername=
IdleTimeout=60
LeaveTemporaryFiles=no
LogClean=no
LogFileUnlock=yes
LogTime=no
LogVerbose=yes
MaxAttempts=6
MaxConnectionQueueLength=30
MaxDirectoryRecursion=20
MaxFileSize=15M
MaxFiles=1500
MaxRecursion=8
MaxThreads=20
Quarantine=enabled
QuarantineDirectory=/var/spool/clamav/quarantine
ReadTimeout=300
ScanArchive=yes
ScanHTML=yes
ScanMail=yes
ScanOLE2=yes
ScanPE=yes
ScanRAR=no
SelfCheck=1800
ShowProxySettings=no
ShowUpdateSettings=no
SignaturesUpdated=unknown
UpdateNonOfficeHrs=disabled
UpdateOfficeHrs=disabled
UpdateWeekend=disabled
status=enabled
[root@shpdserver ~]#
-
FilesystemScanExclude=/proc,/sys,/usr/share,/var
FilesystemScanFilesystems=/home/e-smith/files /mnt/tracy
I note that there are commas (",") between the ScanExcludes values. Perhaps you need commans between the ScanFilesystems values as well.
i.e.
db configuration setprop clamav FilesystemScanFilesystems "/home/e-smith/files,/mnt/tracy"
-
I note that there are commas (",") between the ScanExcludes values. Perhaps you need commans between the ScanFilesystems values as well.
Per the rest of this thread, that shouldn't work. This is why we also changed the FAQ.
-
I tried that but not adviseable on slow VPN connection. I timed out.
You could run it in the background
This is telling me it is not scanning the mnt/tracy which has about 40 gb of users files on it.
That appears to be true given your facts.
[root@shpdserver ~]# config show clamav
clamav=service
/opt,=mnt/tracy
ArchiveBlockEncrypted=no
ArchiveBlockMax=no
...
You have an error here (note the line /opt,=mnt/tracy). I don't believe this has any effect but you should clean it up.
config delprop clamav "/opt,"
The fact that clamav is not complaining about /mnt/tracy is bothering me if it is in fact not scanning.
Could you print out the result of:
cat /etc/mtab
I would try to eliminate things from here:
- verify that you have the right path to the mount point
- check your log files in /var/log/ notably "messages" and logs in "clamd"
- try doing a scan with only /mnt/tracy on the scan line
- etc
-
tviles,
the other thing that occurs to me as a I look at your scanned data is that there is very little on the main disks as opposed to the extra disks.
Another method to consider which may simplify your life is to mount the new disk as /home/e-smith/files. This will allow SME to work with no other mods complicated by the symlink.
The net effect is all user/ibay data is on the extra disk and SME config data and OS is on your main disk.
Christian
-
Last login: Sun Jan 4 04:10:01 2009 from pc-00249.shpd.local
[root@shpdserver ~]# cat /etc/mtab
/dev/mapper/main-root / ext3 rw,usrquota,grpquota 0 0
none /proc proc rw 0 0
none /sys sysfs rw 0 0
none /dev/pts devpts rw,gid=5,mode=620 0 0
usbfs /proc/bus/usb usbfs rw 0 0
/dev/md1 /boot ext3 rw 0 0
none /dev/shm tmpfs rw 0 0
/dev/sdf1 /mnt/jeremy ext3 rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
/dev/sdh1 /media/usbdisk1 ext3 rw,nosuid,nodev 0 0
/dev/sdg1 /mnt/tracy ext3 rw 0 0
[root@shpdserver ~]#
-
Last login: Mon Jan 5 05:00:30 2009 from pc-00249.shpd.local
[root@shpdserver ~]# config delprop clamav "/opt,"
[root@shpdserver ~]# cat /etc/mtab
/dev/mapper/main-root / ext3 rw,usrquota,grpquota 0 0
none /proc proc rw 0 0
none /sys sysfs rw 0 0
none /dev/pts devpts rw,gid=5,mode=620 0 0
usbfs /proc/bus/usb usbfs rw 0 0
/dev/md1 /boot ext3 rw 0 0
none /dev/shm tmpfs rw 0 0
/dev/sdf1 /mnt/jeremy ext3 rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
/dev/sdh1 /media/usbdisk1 ext3 rw,nosuid,nodev 0 0
/dev/sdg1 /mnt/tracy ext3 rw 0 0
[root@shpdserver ~]# config getprop clamav FilesystemScanFilesystems
/home/e-smith/files /mnt/tracy
[root@shpdserver ~]# config getprop clamav FilesystemScanExclude
/proc,/sys,/usr/share,/var
[root@shpdserver ~]# config show clamav
clamav=service
ArchiveBlockEncrypted=no
ArchiveBlockMax=no
ArchiveMaxCompressionRatio=300
Checks=24
DNSDatabaseInfo=current.cvd.clamav.net
DatabaseMirror=db.local.clamav.net
Debug=no
DetectBrokenExecutables=no
FilesystemScan=daily
FilesystemScanExclude=/proc,/sys,/usr/share,/var
FilesystemScanFilesystems=/home/e-smith/files /mnt/tracy
FilesystemScanReportTo=admin
Foreground=yes
HTTPProxyPassword=
HTTPProxyPort=
HTTPProxyServer=
HTTPProxyUsername=
IdleTimeout=60
LeaveTemporaryFiles=no
LogClean=no
LogFileUnlock=yes
LogTime=no
LogVerbose=yes
MaxAttempts=6
MaxConnectionQueueLength=30
MaxDirectoryRecursion=20
MaxFileSize=15M
MaxFiles=1500
MaxRecursion=8
MaxThreads=20
Quarantine=enabled
QuarantineDirectory=/var/spool/clamav/quarantine
ReadTimeout=300
ScanArchive=yes
ScanHTML=yes
ScanMail=yes
ScanOLE2=yes
ScanPE=yes
ScanRAR=no
SelfCheck=1800
ShowProxySettings=no
ShowUpdateSettings=no
SignaturesUpdated=unknown
UpdateNonOfficeHrs=disabled
UpdateOfficeHrs=disabled
UpdateWeekend=disabled
status=enabled
[root@shpdserver ~]#
-
How do I get rid of these?
[root@shpdserver ~]#
[root@shpdserver ~]#
[root@shpdserver ~]#
[root@shpdserver ~]# cd /var
[root@shpdserver var]# cd spool/clamav
[root@shpdserver clamav]# dir
quarantine
[root@shpdserver clamav]# cd quarantine
[root@shpdserver quarantine]# dir
backup.pst EvelynTrue[1].htm o_Lix_o[1].htm ptcielo[1].htm
[root@shpdserver quarantine]#
OK I just did rm -i * and that got rid of them.
-
clamd / current
last two days
2009-01-03 04:14:08.429923500 Shutting down the main socket.
2009-01-03 04:14:08.442578500 Closing the main socket.
2009-01-03 04:14:08.442586500 Socket file removed.
2009-01-03 04:14:08.442589500 --- Stopped at Sat Jan 3 04:14:08 2009
2009-01-03 04:14:18.548744500 Listening daemon: PID: 7344
2009-01-03 04:14:18.548833500 Limits: Global size limit set to 104857600 bytes.
2009-01-03 04:14:18.548837500 Limits: File size limit set to 15728640 bytes.
2009-01-03 04:14:18.548840500 Limits: Recursion level limit set to 8.
2009-01-03 04:14:18.548842500 Limits: Files limit set to 1500.
2009-01-03 04:14:18.548846500 Archive support enabled.
2009-01-03 04:14:18.548849500 Algorithmic detection enabled.
2009-01-03 04:14:18.548851500 Portable Executable support enabled.
2009-01-03 04:14:18.548854500 ELF support enabled.
2009-01-03 04:14:18.548857500 Mail files support enabled.
2009-01-03 04:14:18.548869500 OLE2 support enabled.
2009-01-03 04:14:18.548872500 PDF support enabled.
2009-01-03 04:14:18.548875500 HTML support enabled.
2009-01-03 04:14:18.548893500 Self checking every 1800 seconds.
2009-01-03 09:53:48.717576500 No stats for Database check - forcing reload
2009-01-03 09:53:48.717632500 Reading databases from /var/clamav
2009-01-03 09:53:57.356589500 Database correctly reloaded (922371 signatures)
2009-01-03 11:53:51.760044500 SelfCheck: Database modification detected. Forcing reload.
2009-01-03 11:53:51.760051500 Reading databases from /var/clamav
2009-01-03 11:54:04.267261500 Database correctly reloaded (922396 signatures)
2009-01-03 17:04:37.433091500 SelfCheck: Database status OK.
2009-01-03 19:53:55.475313500 SelfCheck: Database modification detected. Forcing reload.
2009-01-03 19:53:55.475321500 Reading databases from /var/clamav
2009-01-03 19:54:03.959591500 Database correctly reloaded (922398 signatures)
2009-01-04 00:11:00.990830500 SelfCheck: Database status OK.
2009-01-04 22:54:04.948259500 SelfCheck: Database modification detected. Forcing reload.
2009-01-04 22:54:04.961202500 Reading databases from /var/clamav
2009-01-04 22:54:56.736286500 Database correctly reloaded (922402 signatures)
2009-01-04 23:34:33.757277500 SelfCheck: Database status OK.
-
Trying this tonight. Will see in the morning.
[root@shpdserver quarantine]# rm -i *
rm: remove regular file `backup.pst'? y
rm: remove regular file `EvelynTrue[1].htm'? y
rm: remove regular file `o_Lix_o[1].htm'? y
rm: remove regular file `ptcielo[1].htm'? y
[root@shpdserver quarantine]# dir
[root@shpdserver quarantine]# cd /
[root@shpdserver /]# db configuration setprop clamav FilesystemScanFilesystems "/home/e-smith/files,/mnt/tracy"
[root@shpdserver /]# config getprop clamav FilesystemScanFilesystems
/home/e-smith/files,/mnt/tracy
[root@shpdserver /]# signal-event clamav-update
-
Got this back this morning. Will try /mnt/tracy tonight.
WARNING: Can't access file /home/e-smith/files,/mnt/tracy
----------- SCAN SUMMARY -----------
Known viruses: 923695
Engine version: 0.94.2
Scanned directories: 0
Scanned files: 0
Infected files: 0
Data scanned: 0.00 MB
Time: 16.469 sec (0 m 16 s)
[root@shpdserver ~]# db configuration setprop clamav FilesystemScanFilesystems "/mnt/tracy" [root@shpdserver ~]# config getprop clamav FilesystemScanFilesystems
/mnt/tracy
[root@shpdserver ~]# signal-event clamav-update
[root@shpdserver ~]#
I also got this email.
===
=== yum reports available updates:
===
atrpms.noarch 73-1 smeupdates
rsync.i386 3.0.5-1.el4.rf smeupdates
Where does one go to read about these updates?
-
Results from just scanning /mnt/tracy
----------- SCAN SUMMARY -----------
Known viruses: 924277
Engine version: 0.94.2
Scanned directories: 706
Scanned files: 12201
Infected files: 0
Data scanned: 7379.47 MB
Time: 6274.663 sec (104 m 34 s)
Last login: Tue Jan 6 11:54:24 2009
[root@shpdserver ~]# df -T
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/mapper/main-root
ext3 137530456 27133348 103410972 21% /
/dev/md1 ext3 101018 28137 67665 30% /boot
none tmpfs 1037408 0 1037408 0% /dev/shm
/dev/sdf1 ext3 70557052 86088 66886868 1% /mnt/jeremy
/dev/sdh1 ext3 153834852 46010652 100009784 32% /media/usbdisk1
/dev/sdg1 ext3 961432072 21789308 890804764 3% /mnt/tracy
[root@shpdserver ~]#
-
Got this back this morning. Will try /mnt/tracy tonight.
WARNING: Can't access file /home/e-smith/files,/mnt/tracy
----------- SCAN SUMMARY -----------
Known viruses: 923695
Engine version: 0.94.2
Scanned directories: 0
Scanned files: 0
Infected files: 0
Data scanned: 0.00 MB
Time: 16.469 sec (0 m 16 s)
That's what I expected.
-
In the absence of contribs, Tracy and I went off line and we determined that the issue was in the following:
[root@shpdserver ~]# config show clamav
clamav=service
...snip...
MaxFileSize=15M
...snip...
Turns out that he has a number of very large files on the disks. His last post with the df -T output clued me into it but as luck would have it contribs failed before I was able to hit the post button.
Clamav has a number of parameters which decide how it will handle certain files. More info on the parameters can be found with
man clamd.conf
and they can be set via SME's db for clamav.