Topic: Spam Issue

[kruhm]

Hi Everyone,

I have a high volume email server and I feel like I'm getting a ton of spam lately. The messages are from servers not on a blacklist and they have a low spamassassin score.

If anyone has any recommendations either OSS or commercial, I'd be interested.

I have DNSBL & RHSBL enabled:

Code: [Select]

qpsmtpd=service
    DNSBL=enabled
    GeoIP=enabled
    LogLevel=6
    MaxScannerSize=25000000
    RBLList=zen.spamhaus.org:b.barracudacentral.org:whois.rfc-ignorant.org:dnsbl.njabl.org:bl.spamcop.net:dnsbl.ahbl.org:list.dsbl.org:multihop.dsbl.org:psbl.surriel.com:dnsbl-1.uceprotect.net:truncate.gbudb.net
    RHSBL=enabled
    RelayRequiresAuth=disabled
    SBLList=bogusmx.bl.rfc-ignorant.de:multi.surbl.org:black.uribl.com:rhsbl.sorbs.net:bulk.rhs.mailpolice.com:fraud.rhs.mailpolice.com:porn.rhs.mailpolice.com:adult.rhs.mailpolice.com:ex.dnsbl.org:dsn.bl.rfc-ignorant.de
    TlsBeforeAuth=1
    access=public
    qplogsumm=enabled
    status=enabled

I have spamassassin enabled:

Code: [Select]

# config show spamassassin
spamassassin=service
    DNSAvailable=yes
    MessageRetentionTime=15
    OkLanguages=all
    OkLocales=all
    PyzorTimeout=15
    RejectLevel=12
    ReportSafe=0
    Sensitivity=custom
    SkipRBLChecks=0
    SortSpam=enabled
    Subject=[SPAM]
    SubjectTag=enabled
    TagLevel=8
    UseBayes=0
    status=enabled


[brianr]

Tag level at 8 seems quite high, I normally run between 4 and 6.  Depends on your tolerance of false positives though.

Shouldn't skipRBLchecks be false?

[_alex]

This may help:
http://wiki.contribs.org/GeoIP

[kruhm]

Hi,

Thanks for the replies.

The messages are under the 4 spamassassin level.

SkipRBLChecks is defined in template:
./templates/etc/mail/spamassassin/local.cf/10skip_rbl_checks

{
    return "skip_rbl_checks " . ($spamassassin{SkipRBLChecks} || 0);
}

I'm familiar with GeoIP contrib and it's wiki page. I wrote it

Thanks,

[mmccarn]

Is there anything interesting in the spamassassin email headers?  Which rules *are* firing, and what scores are they assigning?

I have 'autowhitelist' turned on, which can have unexpected side-effects.

My "internal" mail server (running Kerio Connect) pretty regularly trashes its internal bayes database -- when this happens, the only solution is to delete the existing database and let it rebuild itself -- users always report improved spam filtering immediately -- even before the bayes learning threshold has been reached.  Perhaps some sort of bayesian poisoning is involved.

Also, I recently upgraded my SME spam filter from 7.6 to 9.0alpha3 -- several users independently reported improved spam filtering.  I don't know if this is due to differences in spamassassin and qpsmtpd beteween 7.6 and 9, or if it was caused because I did not transfer the bayes database to the new server.

[janet]

kruhm

I have a high volume email server and I feel like I'm getting a ton of spam lately.

You do not describe your server/network layout.
Is your sme server acting as a gateway, is it connected directly to the Internet via  bridged modem, do you have any other equipment between the server & the Internet ????

Best spam/antivirus performance is obtained when configured in standard server & gateway mode with a bridging modem & no other network devices etc or layers of configuration between sme server & the net.

[kruhm]

Again, thanks for the replies.

Is there anything interesting in the spamassassin email headers?  Which rules *are* firing, and what scores are they assigning?

Here's one from the latest round:

Code: [Select]

No, hits=6.4 required=8.0 tests=HTML_MESSAGE,INVALID_MSGID,MIME_HTML_ONLY,MSGID_SHORT,RAZOR2_CHECK,SPF_PASS,URIBL_BLACK,URIBL_RHS_DOB

Also, I recently upgraded my SME spam filter from 7.6 to 9.0alpha3 -- several users independently reported improved spam filtering.

No doubt I would love to but I can't take that chance with a production server.

I'll look into the bayes db.

You do not describe your server/network layout.

It's in server-only mode. Can't switch it due to outside regulations. Sonicwall as the gateway.

Best spam/antivirus performance is obtained when configured in standard server & gateway mode with a bridging modem & no other network devices etc or layers of configuration between sme server & the net.

I realize that. Thank you.

Any other ideas? I'm looking into a Sonicwall Email Security Appliance.

Thanks again.

[mmccarn]

http://wiki.apache.org/spamassassin/Rules/HTML_MESSAGE
http://wiki.apache.org/spamassassin/Rules/INVALID_MSGID   1.705 1.710 2.054 2.185
http://wiki.apache.org/spamassassin/Rules/MIME_HTML_ONLY
http://wiki.apache.org/spamassassin/Rules/MSGID_SHORT     0.001 0.337 0.001 0.001
http://wiki.apache.org/spamassassin/Rules/RAZOR2_CHECK
http://wiki.apache.org/spamassassin/Rules/SPF_PASS
http://wiki.apache.org/spamassassin/Rules/URIBL_BLACK      0 1.775 0 1.725
http://wiki.apache.org/spamassassin/Rules/URIBL_RHS_DOB  0 0.276 0 1.514
(Scores taken from http://spamassassin.apache.org/tests_3_3_x.html, except for 'INVALID_MSGID', which was taken from the rule URL above.)

If you can verify that one of these rules is common to all of the undesired email but is not fired by any messages you really want, you could solve your problem by creating a custom rule score.

I would start with the four rules for which I've shown the default scores (INVALID_MSGID, MSGID_SHORT, URIBL_BLACK, and possibly URIBL_RHS_DOB).

You can test your existing mail folders for messages matching each rule using:

Code: [Select]

cd /home/e-smith/files/users
find . -name *:* -exec grep -Hl MSGID_SHORT "{}" \;

(replace "MSGID_SHORT" with each test in turn).

Good luck.

(Note: I'm running this on my home mail server as I write -- the only message matching the sample query above is the forum message about this thread...)

[mmccarn]

Update: When done, the only messages on my server (~1.6Gb of email) that fired "MSGID_SHORT" were about this thread or from Dell in 2010.

[CharlieBrady]

Update: When done, the only messages on my server (~1.6Gb of email) that fired "MSGID_SHORT" were about this thread or from Dell in 2010.

Which suggests that the score for the rule should be increased.

I'm not surprised that OP is seeing a lot of spam with threshold of 8. I see that brianr already suggested adjusting the threshold.

[kruhm]

I see that brianr already suggested adjusting the threshold.

But the spamassassin scores on the problem messages seem to be low. The latest spam has a score of 1.1 & 3.9

Code: [Select]

No, hits=1.1 required=8.0 tests=HTML_MESSAGE,MIME_HTML_ONLY,SPF_PASS

Code: [Select]

No, hits=3.9 required=8.0 tests=HTML_MESSAGE,INVALID_MSGID,MIME_HTML_ONLY,MSGID_SHORT,RDNS_NONE,SPF_PASS

[mmccarn]

Sorry I didn't notice this earlier, but your spamassassin seems to have bayes disabled.  You probably want to enable it and 'train' it a bit.

# config show spamassassin
spamassassin=service
    ...
    UseBayes=0
    status=enabled

[kruhm]

Hi mmccarn,

Thank you very much. I'll be looking into it this weekend.

Thanks,

[kruhm]

I adjusted the taglevel to medium:

Code: [Select]

config setprop spamassassin Sensitivity medium
And I tweaked the scores a little by adding:

Code: [Select]

score SARE_ADULT2 20.000
score URIBL_DBL_SPAM 20.000
score RAZOR2_CF_RANGE_51_100 2.000
score RAZOR2_CF_RANGE_E8_51_100 2.000
score URIBL_BLACK 20.000
It seems to have helped. Thanks so much for your suggestions.

I still seem to get a lot of spam that isn't on a DNSBL and that is spamtagged under 2.0 without triggering many rules, maybe only 1-3 rules. My observation is that they come during the work day when a heavier load is placed on the server. They seem to be hit-and-run/snowshoe spam.

Outside of the work day, spam is closer to none.

Any other advice or suggestions?

[mmccarn]

I strongly recommend enabling Bayes with both manual and automatic training if you have not done so.

I have always setup bayes scoring using the Sonoracomm howto:
http://www.sonoracomm.com/support/19-inet-support/49-spam-filter-configuration-for-sme-7

But you can also set them up using the 'Learn' contrib:
http://wiki.contribs.org/Learn

Automatic training (for me) takes care of lots of emails that aren't caught by other rules.  Manual training lets me personalize my spam filter behavior without opening an ssh session to the server -- just drag the offending message into a training folder.

[Knuddi]

Hi kruhm,

You should be welcome to try out my filter solution as see whether it makes a difference - if it does I can maybe help finding the right combination to remove some more spam. Bayes is great but requires intelligent training, grey listing is annoying but sometimes does a great job. I am afraid that relying on SpamAssassin rules will be an endless walk in the spam desert

Contact me at jkn@scanmailx.com if interested.

Greetings,
Jesper

[Knyte]

I've had fantastic results by blocking spam at the firewall level, if that is an option for you.  pfsense (BSD based firewall) has a plugin (I think it's standard on the 2.x version) called Country Block that prevents any email from selected countries.  I noticed ~90% reduction in spam after configuring in this fashion.  Now I see ~5 spam a month, if that.  Of those, perhaps 1 or 2 are missed by SpamAssassin and make it to the Inbox.  Not bad at all.

[kruhm]

Hi Knyte,

Thanks for the info. I've subscribed to Knuddi's product as a stop-gap solution since it's cost effective and simple.

In the long run, the client has decided to move to MS Hosted Exchange. I've tried to convince them that it isn't going to solve the issues they're facing but I'll have to let them learn on their own.

Thanks to everyone involved in making suggestions.

[MSmith]

As for the machine being in server-only mode ... I haven't tried this out, but would it be possible to have the SME machine have two NICs with two IPs on the same /24, for instance, and run in server/gateway mode? The Sonicwall passes port 25 traffic to the "external" NIC, then your clients pull email from the "internal" NIC.

[MSmith]

Answering my own question ... I posted a bug and developers said "nope, can't have 2 NICS in same subnet." Oh well.

[janet]

MSmith

..... would it be possible to have the SME machine have two NICs with two IPs on the same /24, for instance, and run in server/gateway mode? The Sonicwall passes port 25 traffic to the "external" NIC, then your clients pull email from the "internal" NIC.

A similar configuration is available already.
Configure SME in gateway & server mode, then select the Static IP option (done in admin console menu Configure tbis server), that IP being the IP of the firewall, effectively creating a DMZ (AFAIK).
Whether doing that will gain you spam filtering advantages is questionable, but for many professional installers/users it is a "standard" arrangement to increase security.

[MSmith]

Thanks Janet, that's very interesting but I'm a bit lost. Are you saying that if, for instance, my Sonicwall had a public-facing IP of X.Y.Z.Z that I would configure the SME "behind" it with an "external" IP of X.Y.Z.Z? I'd be very interested to see how that would work. How would the Sonicwall know where to send the SMTP packets?

I may try setting up a 2nd IP range on the firewall and passing the traffic to the "external" NIC that way, but currently the machine seems to be working well enough in server-only mode.

[janet]

MSmith

Are you saying that if, for instance, my Sonicwall had a public-facing IP of X.Y.Z.Z that I would configure the SME "behind" it with an "external" IP of X.Y.Z.Z?

No, although it may/will depend on the type of connection protocol you are using.

More typically the Static IP will be the local IP of the gateway (eg your Sonicwall)

The comment was really related to the original poster.