Topic: HELP: From Spam Assassin To ASSP Question

[sc300t]

Hi,

I have a sme 6.01 box with ANTIVIRUS (ClamAV) by  http://www.pagefault.org/howto/e-smith-antivirus.shtml and SPAMFILTER (RBL and SpamAssassin) by http://sme.swerts-knudsen.dk/howtos/howto_29.htm

The antivirus works wonderfully, but the spam filter only has 30-40% effectiveness. I was thinking to switch to ASSP. I'm going to follow the installation by http://www.sonoracomm.com/sonoracomm/pdf/ASSP_HOWTO.pdf

My questions are:
1. How can uninstall the spamassassin? It was installed using jesper's script. If not, is it safe to just disable the spam filter from the panel and go ahead install assp?
2. Since the assp has the antivirus feature, should I uninstall the the one from pagefault.org?

Thanks.
Joe

[mach1_4fun]

If you are set on removing spamassassin ignore this post:
Just out of curiosity, what RBL(s) are you using and whats your spam threshold? I have had really good luck with the spamcop block list (I think I have stopped more like 95% with that combo).

before ditching spam assassin might try this
-try a different or multiple RBL'S
-lower the threshold even further, like 1.5
-ensure that spamassassin can connect to the RBL and the razor database.
-add some custom rules

[sc300t]

Thanks for the tips. I'm a newbie, and I dont understand some of the tips below:

-try a different or multiple RBL'S
I did try this. Here's what I have right now
dynablock.njabl.org Remove
sbl-xbl.spamhaus.org

-lower the threshold even further, like 1.5
Does treshold mean the spam assassin sensitivity? I set the sensitivity to High(5). BTW, I am using the control panel of the SpamFilter by Jesper Knudsen.

-ensure that spamassassin can connect to the RBL and the razor database.
How can I do this?

-add some custom rules
Can you please give me some pointers and examples?

Thanks.

[mach1_4fun]

Sorry about that, help me clarify a little.
Lets try the things that will do the most good first:

1) first lets try a different block list,
remove the old one and add this one:
bl.spamcop.net

2) also, try adjusting the sensitivity to 2 (very high).

3) Make sure to whitelist your own domain and any other important places that you get email from. so your own emails don't end up in the junkmail folder.
Example:
adding "*example.com" to the whitelist would make sure that any mail that ends in @example.com would go through without a problem. Also add "*localhost" to make sure that your webmail gets the same treatment.

Try that first and see if it makes a big difference, I think it will. Then we can talk about the other stuff.

I have noticed that I rarely see a legit email with a score above 2 even.
After making the changes look at the 12:00 email from spamassassin and see how many messages it stopped.

[sc300t]

I did change:
1. RBL to bl.spamcop.net
2. spamassassin sensitivity to very high(2)
3. I added our domain and other important domain in the white list.

It has been almost 24 hours since the change. They efficiency went up from 30-40% to 50-60%, but that is still not good enough.

Any other suggestion? Thanks.

[azche24]

Hi,

I did change:
It has been almost 24 hours since the change. They efficiency went up from 30-40% to 50-60%, but that is still not good enough.

I experienced ASSP and used it quite a long time both at home and at office.

ASSP was designed for larger usergroups (eg. > 10 Workstations sending > 5 Mails a day each) that send and receive mails with pretty similar topics and contents.

ASSP therefore will work better (maybe only_) in office environments with lots of similar mails. Simple reason is, that ASSP builds up it own rules and only does this effectively, when there is very similar stuff for training.

ASSP needs that and lots of training (in our case more than 14 days). After that it works ok. But it never worked at home in our 3-user and about 2 mails a day from newsgroup to office contents.

ASSP produced tons of false positives in our home environment. I hate that.

ASSP is perfect for the described office environment (about 98 % Spam blocked + about 5 - 10 % false positives) and to my opinion should better not be used by familys and novices.

SPAMASS (if configured properly) will do your job better.

[sc300t]

Thanks for the tip on the assp.

I guess question is still, how can I configure the spamassassin so that it will work effectively? I tried all different settings from the SpamFilter Control Panel already. Thanks.

[mach1_4fun]

Just out of curiosity how much email are we talking about? (Messages per day)
also, how do you know that the filter is only 50-60% efficient ?

Also, a copy of your spam report might be useful too.

I have an employee that got about 200-300 spams a day and I got it down to 6 a day or less. I am racking my brain as to why spamassassin is not catching more...

Try checking the spam assassin website, They have some good info on how it works and how to improve it. I am going to also look into how to lower the threshold to 1 manually.

http://spamassassin.apache.org/index.html

[djhomeless]

I just want to throw this out there.

I have an email address that I have just recently resurrected thanks to SA. I've had this addy for almost 10 years and you literally get thousands of 'hits' if you google it (including Groups).

Basically, I had to abandon this addy two years ago because I was literally receiving 300-400 pieces of SPAM a day.

Now, I have only one RBL (SpamHaus), and the sensitivity set to Very High (2), and SA nails about 90-95% of the SPAM.

Oddly enough, the RBL has caught absolutely zero mails so I may switch over to Spamcop to give it a go. Other than that, SAhas been a dream.

Geoffrey

[sc300t]

mach1_4fun,

how can I check if my rbl list is working?

[NickR]

Don't forget that RBL's can only work if you are running a mailserver which directly accepts mail from the 'net.  
If you use fetchmail or your ISP relays mail to your machine, the RBL won't help.

In fact, I recently experimented with not having any backup MX handlers for my domain & it has cut the spam that SA has to process by a factor of 3.

To check if the RBL is working, go to the server manager & use 'view log files' to look at smtpfront-qmail/current & filter on rblsmtp.

[djhomeless]

Hmm,
I do have a backup MX server, but I'm loathe to turn it off as I have just switched to a new ADSL provider and I want to make sure they are a stable provider.

I did check the logs as you suggested, but using the 'rblsmtp' as a filter yielded an empty log (the log is full).

Does that mean the RBL is not working, or that my mail is all going via the backup MX record??

Thanks!

Geoffrey

[mbachmann]

Jespers Spamfilter sents a report like this, there you can see if RBL is working. Do you get a report like this?

Code: [Select]



Period Beginning : Fri Jan 28 00:00:07 2005
Period Ending    : Sat Jan 29 00:00:07 2005

Reporting Period : 24.00 hrs
--------------------------------------------------

Total spam rejected   :       92 ( 36.08%)
       RBL rejected   :        1 (  0.39%)
     Score above 15   :       25 ( 27.17%)
Total ham accepted    :      163 ( 63.92%)
                        -------------------
Total emails processed:      255 (   11/hr)

Average spam threshold :        6.20
Average spam score     :       12.88
Average ham score      :       -9.44

Statistics by Hour
-------------------------------------
Hour                 Spam         Ham
-------------    --------    --------
2005-01-28, 00             6           2
2005-01-28, 01             7           1
2005-01-28, 02             0           3
2005-01-28, 03             1           2
2005-01-28, 04             3           5
2005-01-28, 05             4           6
2005-01-28, 06             1           6
2005-01-28, 07             4           3
2005-01-28, 08             0          18
2005-01-28, 09             5          24
2005-01-28, 10             7          15
2005-01-28, 11             4          15
2005-01-28, 12             3          14
2005-01-28, 13             2           7
2005-01-28, 14             4          12
2005-01-28, 15             9           6
2005-01-28, 16             4           3
2005-01-28, 17             9           3
2005-01-28, 18             2           1
2005-01-28, 19             3           7
2005-01-28, 20             7           5
2005-01-28, 21             2           0
2005-01-28, 22             3           2
2005-01-28, 23             2           3
2005-01-29, 00             0           0


[NickR]

I did check the logs as you suggested, but using the 'rblsmtp' as a filter yielded an empty log (the log is full).

Does that mean the RBL is not working, or that my mail is all going via the backup MX record??

Here's mine so far today:

2005-02-04 03:32:25.928037500 rblsmtpd: 140.212.207.152 pid 2524: 451 Blocked - see http://www.spamcop.net/bl.shtml?140.212.207.152
2005-02-04 03:32:30.338100500 rblsmtpd: 202.127.1.113 pid 2525: 451 Blocked - see http://www.spamcop.net/bl.shtml?202.127.1.113
2005-02-04 03:33:28.630625500 rblsmtpd: 61.98.40.56 pid 2526: 451 Blocked - see http://www.spamcop.net/bl.shtml?61.98.40.56
2005-02-04 03:33:38.113293500 rblsmtpd: 221.54.36.27 pid 2553: 451 http://dsbl.org/listing?221.54.36.27
2005-02-04 03:33:42.307449500 rblsmtpd: 24.27.68.87 pid 2554: 451 Blocked - see http://www.spamcop.net/bl.shtml?24.27.68.87
2005-02-04 03:33:48.412188500 rblsmtpd: 69.212.47.218 pid 2555: 451 Blocked - see http://www.spamcop.net/bl.shtml?69.212.47.218
2005-02-04 03:33:52.324171500 rblsmtpd: 24.24.148.76 pid 2556: 451 Blocked - see http://www.spamcop.net/bl.shtml?24.24.148.76
2005-02-04 03:34:06.523177500 rblsmtpd: 81.134.143.207 pid 2581: 451 Blocked - see http://www.spamcop.net/bl.shtml?81.134.143.207
2005-02-04 03:34:38.979597500 rblsmtpd: 67.130.34.81 pid 2582: 451 Blocked - see http://www.spamcop.net/bl.shtml?67.130.34.81
2005-02-04 03:34:41.778715500 rblsmtpd: 24.110.106.153 pid 2583: 451 Blocked - see http://www.spamcop.net/bl.shtml?24.110.106.153
2005-02-04 03:34:45.186563500 rblsmtpd: 24.210.239.82 pid 2584: 451 Blocked - see http://www.spamcop.net/bl.shtml?24.210.239.82
2005-02-04 09:52:28.548537500 rblsmtpd: 218.19.109.68 pid 24403: 451 Dynamic/Residential IP range listed by NJABL dynablock - http://njabl.org/dynablock.html
2005-02-04 09:59:18.995733500 rblsmtpd: 222.83.255.215 pid 24571: 451 http://dsbl.org/listing?222.83.255.215
2005-02-04 11:42:17.028912500 rblsmtpd: 81.4.168.139 pid 27753: 451 Blocked - see http://www.spamcop.net/bl.shtml?81.4.168.139
2005-02-04 11:42:25.779777500 rblsmtpd: 202.83.175.98 pid 27754: 451 Blocked - see http://www.spamcop.net/bl.shtml?202.83.175.98

It could mean that everything is going via the relays, but that would (a) be unusual & (b) be easy to spot - all your incoming mail would arrive via the relays & you should know what their IP's are.

BTW, I just checked & you don't have any other MX set other than your SME.  I see you're on Bulldog - how do you rate them?

[djhomeless]

I do but the RBL field always shows 0. I always found it odd considering the sheer amount of SPAM I receive.

Geoffrey

[hordeusr]

For our server ASSP puts much less load on the server.  I let it learn for a couple of weeks.  For us, it works better than any other SPAM solution.  It's very rare for anyone to get spam, when they do they use the "report as spam" link in the webmail (that forwards it to ASSP).  It makes heavy use of a whitelist, all outgoing receipients are added to the whitelist by ASSP.  A false positive is returned to the sender with instructions on what to do.  Oh, and it blocks whatever attachments you want (via the extension).  The only viruses CLAMAV ever see's is zip viruses, the rest are blocked by ASSP(I have it set to block all executable attachments).  Here is my current stats...
assp ver 1.1.0

ASSP Proxy Uptime:        329.886 days
SMTP Connections Received:  274704
Relay Attempts Rejected:    2719 (1.0%)
MESSAGE HANDLING Statistics
Messages Processed:        256917 (778.8 per day)
Rules Spams:                15476
Bayesian Spams:        84965
Spamlover Spams Passed:     2977
Attachments Blocked:        26516
Viruses Detected:        0
Spam Bombs Blocked:        0
Scripts Blocked:        0
Whitelisted Messages:       31643
External Spam Percentage:   56.8%

(I haven't tried the virus protection (uses clam database)

[Mumm-Ra]

I use ASSP on my home server.
I managed to configure it as how I like it pretty much straight away ( I already had a ruck of spam that I sent through the filter).
The only time I receive spam now is through my second mail server (hosted by my domain registrar) but I've now forwarded this mail to my gmail account (which does spam & av filtering) and I now pop it from there instead.

Perfect solution.

[djhomeless]

It could mean that everything is going via the relays, but that would (a) be unusual & (b) be easy to spot - all your incoming mail would arrive via the relays & you should know what their IP's are.

Agreed. I can see the IP of my backup MX every once in a while, but not often. The odd thing is if I go to DNSReport.com, they always show my mail server times out (I think it is a security measure). Regardless, I am going to switch over to spamcop for a few days to see the difference.

BTW, I just checked & you don't have any other MX set other than your SME.  I see you're on Bulldog - how do you rate them?

I literally just switched to Bulldog two weeks ago. So far the phone quality has been iffy, but improving. The broadband itself has been very fast, the only reason I took a change on them was because they offer 512 upstream. So far so good!

Geoffrey

[mach1_4fun]

mach1_4fun,

how can I check if my rbl list is working?

If you log in to your admin account through webmail and you will see that spam assassin sends a daily blocking report to that mail box.

heres what mine looks like:
Period Beginning : Fri Feb  4 00:00:01 2005
Period Ending    : Sat Feb  5 00:00:01 2005

Reporting Period : 24.00 hrs
--------------------------------------------------

Total spam rejected   :     1239 ( 57.98%)
    RBL rejected   :     1029 ( 48.15%)
     Score above 15   :       32 (  2.58%)
Total ham accepted    :      898 ( 42.02%)
                        -------------------
Total emails processed:     2137 (   89/hr)

Average spam threshold :        1.04
Average spam score     :        1.48
Average ham score      :      -73.65

[djhomeless]

24 hours after adding bl.spamcop.net to my rbl list, I still have zero RBL's in my daily update.

This is getting very odd. I swear I can see only a small percentage of mail going via the backup MX. Maybe I should turn off the backup for a few days to check?

Geoffrey

[azche24]

Hi, DJ,

i suffer from this at my home-server. No RBL at all. I am receiving lots of spam througth SpamAssassin now.

I already checked all the settings. Something fuxxed up when updating, because i had the RBL Installation by jesper earlier than the normal spamass panel.

So i will build up the server from the core again as soon as 6.5 RC is out.

[sc300t]

My RBL is also 0. This is very weird. We have more than 80 users. Here's the spam filter report:

Period Beginning : Fri Feb  4 00:00:00 2005
Period Ending    : Sat Feb  5 00:00:00 2005

Reporting Period : 24.00 hrs
--------------------------------------------------

Total spam rejected   :      128 ( 29.98%)
       RBL rejected   :        0 (  0.00%)
     Score above 15   :       48 ( 37.50%)
Total ham accepted    :      299 ( 70.02%)
                        -------------------
Total emails processed:      427 (   18/hr)

Average spam threshold :        2.00
Average spam score     :       13.92
Average ham score      :      -22.65

Statistics by Hour
-------------------------------------
Hour                 Spam         Ham
-------------    --------    --------
2005-02-04, 00             5           1
2005-02-04, 01             4           2
2005-02-04, 02             3           3
2005-02-04, 03             7           3
2005-02-04, 04             5           0
2005-02-04, 05             3           1
2005-02-04, 06             4           6
2005-02-04, 07             6          12
2005-02-04, 08             7          15
2005-02-04, 09             5          34
2005-02-04, 10             3          31
2005-02-04, 11             7          31
2005-02-04, 12             4          26
2005-02-04, 13             4          21
2005-02-04, 14             3          24
2005-02-04, 15             4          38
2005-02-04, 16             7          12
2005-02-04, 17            19           5
2005-02-04, 18             4           2
2005-02-04, 19             4           4
2005-02-04, 20             2           5
2005-02-04, 21             5           7
2005-02-04, 22             7          13
2005-02-04, 23             6           3

[sc300t]

I think my spam filter will work so much better if the rbl works. Can anybody help me?

[djhomeless]

Do you have a backup mail server?

[sc300t]

No, I dont have a backup mail server

[sonoracomm]

Hi,

Though I wrote the SME ASSP howto mentioned above, I don't profess to be an expert.  YMMV.

I have about two dozen ASSP/SME installations in the field in production.  All small/med businesses.

I have used RBLs in the past.  I found them less than sufficient, but simple to implement.  Not one customer thought the RBL alone was sufficient.

I find ASSP to be _very_ effective in the environments where I have installed it.  100% of the time.  Care does need to be taken to not 'poison' its databases...and it's easier to do than you might think.  For example, _never_ send a message to or reply to a spammer!

When I install ASSP, I don't even use test mode.  I found it was just to confusing for my clients and unnecessary.  However, using the sample spam/notspam databases, it _is_ pretty agresasive and will have a few false positives.  

I _always_ create a 'spambucket' account and have all filtered mail sent to this account.  That way, the 'spam administrator', just has to check the 'spambucket' account (via webmail, usually) once a day and no mail is ever lost.

I also always create a special code, like the companys phone number, that will allow messages to pass through  the spam filter.  That way road warriors just put their phone number in their signature and they never have to worry about their messages to other employees not getting through the spam filter.

In small businesses, ASSP quickly reaches a very high degree of accuracy with _very_ few false positives.  I can understand why it might not work as well with only a couple of messages per day, but I'll bet it would be a great improvement over nothing at all.  

Go to http://assp.sourceforge.net for a much better understanding.

G

p.s.  For now, it is not for 6.5b.  I'll update it soon...