Koozali.org: home of the SME Server
Obsolete Releases => SME Server 9.x => Topic started by: gbentley on February 28, 2020, 01:21:48 PM
-
Hi All,
Have an ongoing issue that has been increasing in frequency. Once a week or so outgoing emails with attachments get stuck in users Outlook outboxes. A while ago a quick refreshclam would fix it. However it is now becoming pretty frequent.
Against better advice I increased the size of attachments to 25MB however most of the above issues are created by emails that are less than 10MB.
Here is refreshclam output just now;
[root@mail ~]# refreshclam
Current working dir is /var/clamav
Max retries == 6
ClamAV update process started at Fri Feb 28 11:59:39 2020
Using IPv6 aware code
Querying current.cvd.clamav.net
TTL: 973
Software version from DNS: 0.102.2
WARNING: Your ClamAV installation is OUTDATED!
WARNING: Local version: 0.100.2 Recommended version: 0.102.2
DON'T PANIC! Read https://www.clamav.net/documents/upgrading-clamav
Retrieving http://db.local.clamav.net/main.cvd
Trying to download http://db.local.clamav.net/main.cvd (IP: 104.16.219.84)
Downloading main.cvd [100%]
Loading signatures from main.cvd
Properly loaded 4564902 signatures from new main.cvd
main.cvd updated (version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
Querying main.59.93.1.0.6810DB54.ping.clamav.net
Can't query main.59.93.1.0.6810DB54.ping.clamav.net
Retrieving http://db.local.clamav.net/daily.cvd
Trying to download http://db.local.clamav.net/daily.cvd (IP: 104.16.219.84)
Downloading daily.cvd [100%]
Loading signatures from daily.cvd
Properly loaded 2199661 signatures from new daily.cvd
daily.cvd updated (version: 25735, sigs: 2199661, f-level: 63, builder: raynman)
Querying daily.25735.93.1.0.6810DB54.ping.clamav.net
Can't query daily.25735.93.1.0.6810DB54.ping.clamav.net
Retrieving http://db.local.clamav.net/bytecode.cvd
Trying to download http://db.local.clamav.net/bytecode.cvd (IP: 104.16.219.84)
Downloading bytecode.cvd [100%]
Loading signatures from bytecode.cvd
Properly loaded 94 signatures from new bytecode.cvd
bytecode.cvd updated (version: 331, sigs: 94, f-level: 63, builder: anvilleg)
Querying bytecode.331.93.1.0.6810DB54.ping.clamav.net
Can't query bytecode.331.93.1.0.6810DB54.ping.clamav.net
Database updated (6764657 signatures) from db.local.clamav.net (IP: 104.16.219.84)
WARNING: Clamd was NOT notified: Can't connect to clamd through /var/clamav/clamd.socket: No such file or directory
[root@mail ~]#
I will investigate how to reset attachment db entry to defaults. I have also noticed that qpsmtpd log when the above happens almost always includes 'virus::clamdscan 902 unable to scan for viruses msg denied before qued' and 'virus::clamdscan cannot ping clamd server could not establish connection, tried Unix domain and TCP socket at /usr/share/perl5/vendor_perl/ClamAV/Client.pm line 471'
Thanks in advance of any help :)
-
I don't know if this helps or not however from the troubleshooting on Clamav
[root@mail ClamAV]# host -t txt current.cvd.clamav.net
current.cvd.clamav.net descriptive text "0.102.2:59:25736:1582892940:1:63:49191:331"
[root@mail ClamAV]# dig @ns1.clamav.net db.us.big.clamac.net
; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.68.rc1.el6_10.3 <<>> @ns1.clamav.net db.us.big.clamac.net
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 54016
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;db.us.big.clamac.net. IN A
;; Query time: 69 msec
;; SERVER: 193.28.86.61#53(193.28.86.61)
;; WHEN: Fri Feb 28 12:48:04 2020
;; MSG SIZE rcvd: 38
-
I've reset to default all of the config properties listed here;
https://wiki.contribs.org/SME_Server:Documentation:FAQ:Section04#Set_max_email_size
I don't think I have ever changed the below settings but the warning sounds alarming.
Assume the defaults are fine?
These attributes could result in the rejection of a compressed attachment on a SME server:
ArchiveMaxCompressionRatio (default 300)
MaxFiles (default 1500)
MaxRecursion (default 8)
I am now running with mail scanning off as the warnings in my original post occur as soon as its enabled.
-
gbentley
Show us the output of these:
config show qpsmtpd
config show php
config show qmail
config show clamd
config show clamav
config show spamassassin
-
[root@mail ~]# config show qpsmtpd
qpsmtpd=service
Bcc=disabled
BccMode=cc
BccUser=maillog
DNSBL=enabled
LogLevel=6
MaxScannerSize=25000000
RBLList=zen.spamhaus.org,bl.spamcop.net,multi.surbl.org,black.uribl.com,rhsbl.sorbs.net
RHSBL=enabled
RelayRequiresAuth=disabled
SBLList=dbl.spamhaus.org,multi.surbl.org,black.uribl.com,rhsbl.sorbs.net
TlsBeforeAuth=1
UBLList=multi.surbl.org:8-16-64-128,black.uribl.com,rhsbl.sorbs.net
URIBL=enabled
access=public
qplogsumm=disabled
status=enabled
[root@mail ~]# config show php
php=service
AllowUrlFopen=Off
UploadMaxFilesize=10M
status=enabled
[root@mail ~]# config show qmail
qmail=service
MaxMessageSize=15000000
status=enabled
[root@mail ~]# config show clamd
clamd=service
MemLimit=1400000000
status=enabled
[root@mail ~]# config show clamav
clamav=service
ArchiveBlockEncrypted=no
Checks=24
DNSDatabaseInfo=current.cvd.clamav.net
DatabaseMirror=db.local.clamav.net
Debug=no
DetectBrokenExecutables=no
FilesystemScan=weekly
FilesystemScanExclude=/proc,/sys,/usr/share,/var
FilesystemScanFilesystems=/home/e-smith/files
FilesystemScanReportTo=admin
FilesystemScanUnofficialSigs=no
Foreground=yes
HTTPProxyPassword=
HTTPProxyPort=
HTTPProxyServer=
HTTPProxyUsername=
HeuristicScanPrecedence=yes
IdleTimeout=60
LeaveTemporaryFiles=no
LogClean=no
LogFileUnlock=yes
LogTime=no
LogVerbose=yes
MaxAttempts=6
MaxConnectionQueueLength=30
MaxDirectoryRecursion=20
MaxFileSize=15M
MaxFiles=1500
MaxRecursion=8
MaxThreads=20
Quarantine=enabled
QuarantineDirectory=/var/spool/clamav/quarantine
ReadTimeout=300
ScanArchive=yes
ScanHTML=yes
ScanMail=yes
ScanOLE2=yes
ScanPE=yes
ScanRAR=no
SelfCheck=1800
ShowProxySettings=no
ShowUpdateSettings=no
SignaturesUpdated=unknown
UpdateNonOfficeHrs=disabled
UpdateOfficeHrs=disabled
UpdateWeekend=disabled
status=enabled
[root@mail ~]# config show spamassassin
spamassassin=service
DNSAvailable=yes
MaxMessageSize=2000000
MessageRetentionTime=30
OkLanguages=all
OkLocales=all
RejectLevel=9
ReportSafe=0
Sensitivity=custom
SkipRBLChecks=0
SortSpam=enabled
Subject=[SPAM]
SubjectTag=enabled
TagLevel=5
UseBayes=0
status=enabled
[root@mail ~]#
-
When you run signal-event email-update or signal-event clamav-update (and maybe refreshclam) the clamd.socket can take up to 3 minutes to re-establish, during which time email transactions fail with; virus::clamdscan 902 unable to scan for viruses.
you can look for clamd.socket is running in /var/clamav; ll /var/clamav
srw-rw-rw- 1 clamav clamav 0 Feb 28 08:26 clamd.socket
Not sure this helps with your issue, but just to be aware this happens, sometimes I have had the clamd.socket not restart automatically, requires /etc/init.d/clamd start
-
You are right Gary, there is quite a delay that I was misinterpreting.
Now I am back to square one because its obviously an intermittent issue and I guess I will have to wait until one of the users reports it again and go back over the logs etc
-
In my tests there is also quite some delay in actually scanning / processing the email messages that have attachments.
I have heard users say that email with several drawings attached i.e. 6-8 Mb can sit in outbox for ages.
-
Thing with clamd is it needs to load the whole db in memory on every start.
As pointed it can take 3 minutes and sometimes way more. This depends on memory available, cpu and additional definition db you add to the load.
It can even prevent clamd to start if db is bigger than available memory or limit for memory in config db.
When this occurs the symptoms is that the smtp connexion will’ refuse the email and you need to send again latter.
Check for clamd log if there are any corrupted db alert or missing memory. In that case increase max memory.
It you use unofficial clamav db reevaluate if you need them.
Also instead of reloading clamd multiple time in the day you can change freshclam behaviour to only update once during the night so clamd is up during the day.
-
Check for clamd log if there are any corrupted db alert or missing memory. In that case increase max memory.
Cheers JP - is this the right config param?
config setprop clamd MemLimit
In which case I'll try;
db configuration setprop clamd MemLimit 1800000000
signal-event clamav-update
-
I've been thinking, as we have desktop Anti-Virus, it may be better to use the ISP outgoing server in the users Outlook setup.
-
gbentley
I feel you are sailing too close to the wind on some of those settings.
Increase ALL these (shown below) to say 50000000 (or 50M where appropriate) & see how you go for a while, you can adjust them down to say 30000000 (or 30M) after a while if you really want to limit message size (to something lower).
All parts of your system need to support the largest expected message size plus a considerable allowance for overheads etc.
Run the required signal-event commands after making changes.
config show qpsmtpd
MaxScannerSize=25000000
config show php
UploadMaxFilesize=10M
config show qmail
MaxMessageSize=15000000
config show clamav
MaxFileSize=15M
config show spamassassin
MaxMessageSize=2000000
-
Thanks Janet. My previous settings where increased from default to accommodate up to 25 Mb [which was against the general advice given in the forums]
This has mostly worked over the years but in recent months has been causing delays as described. If an email is sitting in the users outbox for more than a few minutes this creates a support call. Often the only way to 'clear it' has been to restart the server and restarted Outlook.
Anyway, this is happening more and more often [from once every few weeks to several times a week]
I increased the server [XEON 5110 1.6GHZ 2 Cores, 2 Threads] RAM to 8 MB RAM & Dual 240GB SSD's
Whilst this has improved performance generally it hasn't really gone any way to relieving the above symptoms.
I need to at least implement a workaround for now as I am 22 miles away from the office and remote desktop isn't something that is always 'agreeable' when the user is under pressure / deadlines etc
-
email with several drawings attached i.e. 6-8 Mb can sit in outbox for ages
Messages stuck in the users' outbox would be at the client<->qpsmtpd stage; I would expect problems to apper in /var/log/sqpsmtpd/*
When I was using SME for spam filtering at work the qpsmtpd logs only covered about 90 minutes by default; If you need more time, increase the qpsmtpd log retention by changing the number of log files:
config setprop qpsmtpd KeepLogFiles 30
config setprop sqpsmtpd KeepLogFiles 30
sv t qpsmtpd
sv t sqpsmtpd
-
I had a similar issue with outlook, by default I now set outlook server timeout to 3 minutes, under advanced settings.
-
I had a similar issue with outlook, by default I now set outlook server timeout to 3 minutes, under advanced settings.
this is one thing, as clamd will need to download the whole file and then scan it and finally give a result, it could take time when file are larger
also to avoid delay because of 24 update daily you can do this
# config setprop clamav Checks 2
# service freshclam restart
where it will only update the db twice daily.
the default from clamav is 12, SME default is 24, i.e. once hourly at least + multiple tries if it fails.