Topic: Spam Issue

[kruhm]

Hi Everyone,

I have a high volume email server and I feel like I'm getting a ton of spam lately. The messages are from servers not on a blacklist and they have a low spamassassin score.

If anyone has any recommendations either OSS or commercial, I'd be interested.

I have DNSBL & RHSBL enabled:

Code: [Select]

qpsmtpd=service
    DNSBL=enabled
    GeoIP=enabled
    LogLevel=6
    MaxScannerSize=25000000
    RBLList=zen.spamhaus.org:b.barracudacentral.org:whois.rfc-ignorant.org:dnsbl.njabl.org:bl.spamcop.net:dnsbl.ahbl.org:list.dsbl.org:multihop.dsbl.org:psbl.surriel.com:dnsbl-1.uceprotect.net:truncate.gbudb.net
    RHSBL=enabled
    RelayRequiresAuth=disabled
    SBLList=bogusmx.bl.rfc-ignorant.de:multi.surbl.org:black.uribl.com:rhsbl.sorbs.net:bulk.rhs.mailpolice.com:fraud.rhs.mailpolice.com:porn.rhs.mailpolice.com:adult.rhs.mailpolice.com:ex.dnsbl.org:dsn.bl.rfc-ignorant.de
    TlsBeforeAuth=1
    access=public
    qplogsumm=enabled
    status=enabled

I have spamassassin enabled:

Code: [Select]

# config show spamassassin
spamassassin=service
    DNSAvailable=yes
    MessageRetentionTime=15
    OkLanguages=all
    OkLocales=all
    PyzorTimeout=15
    RejectLevel=12
    ReportSafe=0
    Sensitivity=custom
    SkipRBLChecks=0
    SortSpam=enabled
    Subject=[SPAM]
    SubjectTag=enabled
    TagLevel=8
    UseBayes=0
    status=enabled


[brianr]

Tag level at 8 seems quite high, I normally run between 4 and 6.  Depends on your tolerance of false positives though.

Shouldn't skipRBLchecks be false?

[_alex]

This may help:
http://wiki.contribs.org/GeoIP

[kruhm]

Hi,

Thanks for the replies.

The messages are under the 4 spamassassin level.

SkipRBLChecks is defined in template:
./templates/etc/mail/spamassassin/local.cf/10skip_rbl_checks

{
    return "skip_rbl_checks " . ($spamassassin{SkipRBLChecks} || 0);
}

I'm familiar with GeoIP contrib and it's wiki page. I wrote it

Thanks,

[mmccarn]

Is there anything interesting in the spamassassin email headers?  Which rules *are* firing, and what scores are they assigning?

I have 'autowhitelist' turned on, which can have unexpected side-effects.

My "internal" mail server (running Kerio Connect) pretty regularly trashes its internal bayes database -- when this happens, the only solution is to delete the existing database and let it rebuild itself -- users always report improved spam filtering immediately -- even before the bayes learning threshold has been reached.  Perhaps some sort of bayesian poisoning is involved.

Also, I recently upgraded my SME spam filter from 7.6 to 9.0alpha3 -- several users independently reported improved spam filtering.  I don't know if this is due to differences in spamassassin and qpsmtpd beteween 7.6 and 9, or if it was caused because I did not transfer the bayes database to the new server.

[janet]

kruhm

I have a high volume email server and I feel like I'm getting a ton of spam lately.

You do not describe your server/network layout.
Is your sme server acting as a gateway, is it connected directly to the Internet via  bridged modem, do you have any other equipment between the server & the Internet ????

Best spam/antivirus performance is obtained when configured in standard server & gateway mode with a bridging modem & no other network devices etc or layers of configuration between sme server & the net.

[kruhm]

Again, thanks for the replies.

Is there anything interesting in the spamassassin email headers?  Which rules *are* firing, and what scores are they assigning?

Here's one from the latest round:

Code: [Select]

No, hits=6.4 required=8.0 tests=HTML_MESSAGE,INVALID_MSGID,MIME_HTML_ONLY,MSGID_SHORT,RAZOR2_CHECK,SPF_PASS,URIBL_BLACK,URIBL_RHS_DOB

Also, I recently upgraded my SME spam filter from 7.6 to 9.0alpha3 -- several users independently reported improved spam filtering.

No doubt I would love to but I can't take that chance with a production server.

I'll look into the bayes db.

You do not describe your server/network layout.

It's in server-only mode. Can't switch it due to outside regulations. Sonicwall as the gateway.

Best spam/antivirus performance is obtained when configured in standard server & gateway mode with a bridging modem & no other network devices etc or layers of configuration between sme server & the net.

I realize that. Thank you.

Any other ideas? I'm looking into a Sonicwall Email Security Appliance.

Thanks again.

[mmccarn]

http://wiki.apache.org/spamassassin/Rules/HTML_MESSAGE
http://wiki.apache.org/spamassassin/Rules/INVALID_MSGID   1.705 1.710 2.054 2.185
http://wiki.apache.org/spamassassin/Rules/MIME_HTML_ONLY
http://wiki.apache.org/spamassassin/Rules/MSGID_SHORT     0.001 0.337 0.001 0.001
http://wiki.apache.org/spamassassin/Rules/RAZOR2_CHECK
http://wiki.apache.org/spamassassin/Rules/SPF_PASS
http://wiki.apache.org/spamassassin/Rules/URIBL_BLACK      0 1.775 0 1.725
http://wiki.apache.org/spamassassin/Rules/URIBL_RHS_DOB  0 0.276 0 1.514
(Scores taken from http://spamassassin.apache.org/tests_3_3_x.html, except for 'INVALID_MSGID', which was taken from the rule URL above.)

If you can verify that one of these rules is common to all of the undesired email but is not fired by any messages you really want, you could solve your problem by creating a custom rule score.

I would start with the four rules for which I've shown the default scores (INVALID_MSGID, MSGID_SHORT, URIBL_BLACK, and possibly URIBL_RHS_DOB).

You can test your existing mail folders for messages matching each rule using:

Code: [Select]

cd /home/e-smith/files/users
find . -name *:* -exec grep -Hl MSGID_SHORT "{}" \;

(replace "MSGID_SHORT" with each test in turn).

Good luck.

(Note: I'm running this on my home mail server as I write -- the only message matching the sample query above is the forum message about this thread...)

[mmccarn]

Update: When done, the only messages on my server (~1.6Gb of email) that fired "MSGID_SHORT" were about this thread or from Dell in 2010.

[CharlieBrady]

Update: When done, the only messages on my server (~1.6Gb of email) that fired "MSGID_SHORT" were about this thread or from Dell in 2010.

Which suggests that the score for the rule should be increased.

I'm not surprised that OP is seeing a lot of spam with threshold of 8. I see that brianr already suggested adjusting the threshold.

[kruhm]

I see that brianr already suggested adjusting the threshold.

But the spamassassin scores on the problem messages seem to be low. The latest spam has a score of 1.1 & 3.9

Code: [Select]

No, hits=1.1 required=8.0 tests=HTML_MESSAGE,MIME_HTML_ONLY,SPF_PASS

Code: [Select]

No, hits=3.9 required=8.0 tests=HTML_MESSAGE,INVALID_MSGID,MIME_HTML_ONLY,MSGID_SHORT,RDNS_NONE,SPF_PASS

[mmccarn]

Sorry I didn't notice this earlier, but your spamassassin seems to have bayes disabled.  You probably want to enable it and 'train' it a bit.

# config show spamassassin
spamassassin=service
    ...
    UseBayes=0
    status=enabled

[kruhm]

Hi mmccarn,

Thank you very much. I'll be looking into it this weekend.

Thanks,

[kruhm]

I adjusted the taglevel to medium:

Code: [Select]

config setprop spamassassin Sensitivity medium
And I tweaked the scores a little by adding:

Code: [Select]

score SARE_ADULT2 20.000
score URIBL_DBL_SPAM 20.000
score RAZOR2_CF_RANGE_51_100 2.000
score RAZOR2_CF_RANGE_E8_51_100 2.000
score URIBL_BLACK 20.000
It seems to have helped. Thanks so much for your suggestions.

I still seem to get a lot of spam that isn't on a DNSBL and that is spamtagged under 2.0 without triggering many rules, maybe only 1-3 rules. My observation is that they come during the work day when a heavier load is placed on the server. They seem to be hit-and-run/snowshoe spam.

Outside of the work day, spam is closer to none.

Any other advice or suggestions?

[mmccarn]

I strongly recommend enabling Bayes with both manual and automatic training if you have not done so.

I have always setup bayes scoring using the Sonoracomm howto:
http://www.sonoracomm.com/support/19-inet-support/49-spam-filter-configuration-for-sme-7

But you can also set them up using the 'Learn' contrib:
http://wiki.contribs.org/Learn

Automatic training (for me) takes care of lots of emails that aren't caught by other rules.  Manual training lets me personalize my spam filter behavior without opening an ssh session to the server -- just drag the offending message into a training folder.