Koozali.org: home of the SME Server

Obsolete Releases => SME Server 7.x => Topic started by: mark on September 09, 2006, 07:39:49 AM

Title: activating greylisting
Post by: mark on September 09, 2006, 07:39:49 AM: this seems to work ok
have I missed anything obviuos? (other than some people don't like greylisting:)

mkdir -p /usr/bin/config
chmod 777 /usr/bin/config
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/
echo greylisting black_timeout 60 > /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/09greylisting

/sbin/e-smith/expand-template /var/service/qpsmtpd/config/plugins

cheers

Mark
Title: activating greylisting
Post by: jvels on November 27, 2006, 09:22:53 PM: hello

is this the way how to enable graylisting?

best regrads
Jesper Vels
Title: activating greylisting
Post by: bjoyce on December 05, 2006, 11:05:20 PM: Hi,

Yes this does enable greylisting.

I sent a test email for the first time after doing the above and it had a delay of 37 minutes
The 2nd email from the same user and mail server was recieved straight away.

The question I have is where do you put the white list for servers that are legitimate but dont retry? e.g telstra in Australia.

I believe the IP's for telstra's smtp servers are:-

144.140.82.138
144.140.82.142
144.140.82.153
144.140.82.155
144.140.82.181
144.140.82.183
144.140.82.194
144.140.83.154
144.140.83.156
144.140.83.182
144.140.83.195
144.140.92.138
144.140.92.153
144.140.92.155
144.140.92.181
144.140.92.183
144.140.93.154
144.140.93.156
144.140.93.182
144.140.93.195
Title: activating greylisting
Post by: bjoyce on December 06, 2006, 04:32:31 AM: Hi,
RayMitchell has the answer to how to get a email server on the whitelist

Quote
db spamassassin setprop wbl.global *@*vonage.com White *domain2.com White *domain3.com Black
(all on one line)

edited - of course followed by
expand-template /etc/mail/spamassassin/local.cf
svc -t /service/spamd

I added a previously unused email account to the white list and then sent a test email. there was no delay.

Greylist seems to work well. SME7admin graph showed an average of .7 spams per 5 minutes before the implementation now the last 6 hours the average of spam has dropped to .08. a 90% reduction.

I'm impressed.

Brad[/img]
Title: activating greylisting
Post by: chris burnat on December 06, 2006, 04:43:55 AM: How to tweak the delayed period for first post? Could this be brought down say to 5 minutes?
I have been looking at milter-greylist and the response one receives for the first post is:
Remote_host_said:_451_4.7.1_Greylisting_in_action,_please_come_back_in_00:05:00/
Giving_up_on_84.16.68.xxx./
Title: activating greylisting
Post by: bjoyce on December 06, 2006, 06:28:01 AM: Im not absolutely positive but I think the time for the second email is totally dependant on the sending email server config which is obviously out of our hands.

Brad
Title: activating greylisting
Post by: chris burnat on December 06, 2006, 06:39:58 AM: "the time for the second email is totally dependant on the sending email server config which is obviously out of our hands. "

Agreed, how about the intial delay, first post? Bringing it down fro soem 36 min as repported in this thread to say 5 minutes?
Title: Re: activating greylisting
Post by: william_syd on December 06, 2006, 07:33:54 AM: Quote from: "mark"
this seems to work ok
have I missed anything obviuos? (other than some people don't like greylisting:)

chmod 777 /usr/bin/config

Is 777 a good idea ?

Could we just change ownership to qpsmtpd ?

Is a signal-event email-update required ?
Title: activating greylisting
Post by: raem on December 06, 2006, 08:49:26 AM: bjoyce

> RayMitchell has the answer to how to get a email server on the whitelist
> db spamassassin setprop wbl.global *@*vonage.com White

That command has nothing to do with greylisting.
That command adds White or Black list entries for the spamassassin spam filter, which will determine if a message is never identified as spam or always identified as spam.

The whitelist for greylisting is a totally different thing, a sender is whitelisted in order to bypass the greylisting delay time before accepting the message ie messages from a whitelisted sender are always accepted without being rejected the first time by the greylisting process.

I don't know where you set it in sme7.
Title: activating greylisting
Post by: bjoyce on December 06, 2006, 08:53:35 AM: Quote
="RayMitchell"]bjoyce

> RayMitchell has the answer to how to get a email server on the whitelist
> db spamassassin setprop wbl.global *@*vonage.com White

That command has nothing to do with greylisting.

I dont know why it works but it does, I sent a email from a new email account not recieved and the wait time is about 30 minutes, when whitelisted with your command I then sent from another new email and it is accepted immediatly

Brad
Title: activating greylisting
Post by: raem on December 06, 2006, 09:06:16 AM: bjoyce

> Greylist seems to work well.
>... now the last 6 hours the average of spam has dropped to .08. a 90% reduction.
> I'm impressed.

Greylisting does function as designed & will reduce spam to near zero, the problem is that not all mail servers function predictably. Unless you monitor mail that you have not received (and how do you do that ?) you are going to loose messages. People will complain about you not replying to their email and that's when you discover you never received their message.
Unreliable (ie broken) mail servers will not necessarily retry and even send from a different server IP, therefore disrupting the greylisting delay function ie the second message gets delayed again as it is from a different server IP and so on for large email systems with many servers.

Mails servers have different retry periods (some in days), so there is no guarantee how quickly you will receive the second message (after the first rejection).
You can set the delay time as low as you like, but that's not going to change the retry time of a broken mail server (and there are quite a few of them out there).

You can & will need to monitor rejections and then monitor the subsequent acceptance (whenever that occurs), so you will be busy constantly going through email log files, and then maintaining extensive whitelists to work around problematic mail servers.

Good luck !
Title: activating greylisting
Post by: bjoyce on December 06, 2006, 09:33:42 AM: There is a list of "broken" email servers that comes with the milter greylist.
http://hcpnet.free.fr/milter-greylist/
I am using this on a school email server. On the admin email list I have fellow colleges that have used greylisting effectivly and our recieved email is from a fairly limited user set. This set of mail servers from Milter and adding bigpond mail servers, our biggest internet provider in australia who have "broken" email servers, he has had great success for about 6 months now and still recommends it.
The amount of spam I was getting was about 90% with SME 6 so I upgraded to SME 7 with improved results but still getting an anoying amout, 500 spam email in the last 10 weeks on my account alone, so needed to do something more. Bayes filtering seemed to do nothing to improve the situation so now Im in to greylisting.

I can whitelist the milter list of email server and add bigpond and hopefully wont need to touch it again.

Regards Brad
Title: activating greylisting
Post by: raem on December 06, 2006, 09:39:53 AM: bjoyce

Greylisting will accept subsequent messages from the same sender immediately without delay (ie after the first rejection and the subsequent first acceptance within usually a 24 hour period).

The spamassassin plugin may be loaded/hooked into before the greylisting plugin, so if the sender is on the spam (junkmail) white list then the message is automatically accepted, before it is checked by the greylisting plugin.

I think you need to learn more about greylisting.
Title: activating greylisting
Post by: raem on December 06, 2006, 10:03:07 AM: bjoyce

>... our recieved email is from a fairly limited user set.

In a situation like that, greylisting can be more manageable, as there are a limited set of servers that you receive from, and therefore a limited subset of servers to whitelist.

White listing does reduce the effectiveness of greylisting though.

What is your answer for those mail servers who do not retry for 2 or 3 days, I assume you are happy to wait that long for the email to arrive ?
Title: activating greylisting
Post by: bjoyce on December 06, 2006, 12:19:35 PM: Quote from: "RayMitchell"
bjoyce

I think you need to learn more about greylisting.

Where do you suggest I go to do that?

Brad
Title: activating greylisting
Post by: raem on December 06, 2006, 01:05:43 PM: bjoyce

Search

http://forums.contribs.org/index.php?topic=34498.0

You could try white-listing the sending email host. This will selectively change the behavior of some of the subsequent qpsmtpd plugins, and might get your messages through...

To add a white-listed host:
config setprop qpsmtpd RequireResolvableFromHost yes
pico /var/service/qpsmtpd/config/whitelisthosts
(add the IP addresses you want to whitelist, one per line)
signal-event email-update

(Turning on "RequireResolvableFromHost enables the "whitelist_soft" plugin. Also, there's no default template for "whitelisthosts" so your changes shouldn't disappear by themselves. Ultimately, this should probably be templated and should use the configuration database...)
Title: activating greylisting
Post by: raem on December 06, 2006, 01:33:01 PM: bjoyce

Read this file

/usr/share/qpsmtpd/plugins/greylisting
Title: activating greylisting
Post by: william_syd on December 06, 2006, 02:22:01 PM: http://projects.puremagic.com/greylisting/

http://www.hjp.at/projekte/qpsmtpd/denysoft_greylist/

In particular this bit..

Quote
The version on this page contains two improvements over the version it is based on:

* There is a whitelisting mechanism for IP addresses. Gavin has since added a similar, but not identical mechanism (his "whitelist_soft" module). Whitelist_soft is a generic whitelisting solution which affects all modules which check for the notes it adds. My mechanism is specific to this module (so you can exempt a server from greylist checking, but still subject it to other checks.
* A new option per_recipient has been added. If it is used, greylisting is enabled only for recipients which want it. Again, whitelist_soft offers similar functionality, but on a global basis.
Title: activating greylisting
Post by: bjoyce on December 06, 2006, 11:23:14 PM: Thanks for all these comments, I am going to persist with greylisting despite the shortfalls becuase it does reduce the SPAM to all but 0 and email is still coming through in similar volume, See this graph.
(http://abel.suncoastcc.qld.edu.au/b.JPG)

Staff at the school have been made aware of the possible denial of incoming mail and agree that the reduction is spam is worth it.

I will investigate the whitelisting further.

Regards brad
Title: activating greylisting
Post by: mmccarn on December 07, 2006, 02:46:24 PM: Blackout Time
Quote from: "burnat"
Agreed, how about the intial delay, first post? Bringing it down fro soem 36 min as repported in this thread to say 5 minutes?
The initial time is specified in the very first post in this thread. The "black_timeout 60" part of the custom template fragment tells your server to reject it for 60 seconds - if you're seeing a 36 minute delay it's due to the sending server and there's nothing you can do.

White List
The greylisting plugin *does* pay attention to "whitelisthosts" but does *not* pay attention to the spamassassin white lists, as far as I can tell from /usr/share/qpsmtpd/plugins/greylisting. You may be able to seed your greylisting database by specifying "mode testonly" in the custom template fragment, like this:
echo greylisting black_timeout 60 mode testonly > /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/09greylisting
Title: activating greylisting
Post by: hanscees on January 07, 2007, 09:03:49 PM: I would certainly like to setup greylisting. But only when one can whitelist ip-adress-email tuples or ip-adresses.
Better yet, domains, because some providers have many different smtp servers active.

If you cannot whitelist, greylisting eats up some good email, and I dont want that.

Hans-Cees
Title: activating greylisting
Post by: duncan on January 07, 2007, 10:17:21 PM: It seems that the 7.1 update changes the way the plugins are handled. The above greylisting method needs to be modified to suit.
Title: 7.1 greylisting plugin enablement change?
Post by: jahlewis on January 08, 2007, 03:29:15 PM: I see that in 7.1 things have changed somewhat.

in 7.1 /var/service/qpsmtpd/config/plugins is no longer being used, but

/var/service/qpsmtpd/config/peers/ has two files which load the plugins in /usr/share/qpsmtpd/plugins

what is the best way to enable the greylisting plugin in 7.1 for the /var/service/qpsmtpd/config/peers/0 file?

db configuration show qpsmtpd doesn't seem to be the right place, and I've now pushed my knowledge of db far enough. I'd rather do it the right way than create a custom template of the above 0 file to add greylisting.

Thanks.
Title: Re: 7.1 greylisting plugin enablement change?
Post by: gordonr on January 08, 2007, 09:25:55 PM: Quote from: "jahlewis"
I see that in 7.1 things have changed somewhat.

http://bugs.contribs.org/show_bug.cgi?id=1893

Quote from: "jahlewis"

what is the best way to enable the greylisting plugin in 7.1 for the /var/service/qpsmtpd/config/peers/0 file?

As you may know, I'm not a fan of greylisting:

http://lists.contribs.org/mailman/public/devinfo/msg08292.html
http://lists.contribs.org/mailman/public/devinfo/msg06819.html

However, to make a change to the "non-local" connection definition (which is what the peers/0 file is), create the directory /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/ and put your fragment there.

Quote from: "jahlewis"

db configuration show qpsmtpd doesn't seem to be the right place, and I've now pushed my knowledge of db far enough. I'd rather do it the right way than create a custom template of the above 0 file to add greylisting.

You don't want a file custom-template, but you do want a directory custom template. Make sure you read the developer's guide section on directory templates and custom templates. I think it's clear :-) It's over here:

http://wiki.contribs.org/development/
Title: activating greylisting
Post by: jahlewis on January 08, 2007, 09:55:39 PM: Thanks Gordon, so here's the modified instructions that I used to enable greylisting. (I agree with your concerns, but enabling it significantly reduces my spam, and for me it is worth it the risk of missing something)

Code: [Select]
# enable greylisting mkdir -p /usr/bin/config chmod 777 /usr/bin/config mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0 echo greylisting black_timeout 60 > /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/09greylisting /sbin/e-smith/expand-template /var/service/qpsmtpd/config/peers/0

Could someone explain the purpose of the necessity for full access to /usr/bin/config? Surely it can be locked down somewhat? To root? What user/group does qpsmtpd run as?
Title: activating greylisting
Post by: gordonr on January 08, 2007, 10:07:26 PM: Quote from: "jahlewis"

Could someone explain the purpose of the necessity for full access to /usr/bin/config?

I didn't even look at that. It's certainly wrong. Nobody but root should be allowed to write to anything in /usr/bin. And any instruction which says "chmod 0777" is almost certainly wrong.

Quote from: "jahlewis"

Surely it can be locked down somewhat? To root?

If a temporary directory is required it should be something like /var/state/qpsmtpd/greylist, which should be qpsmtpd:qpsmtpd

Quote from: "jahlewis"

What user/group does qpsmtpd run as?

qpsmtpd:qpsmtpd
Title: activating greylisting
Post by: jahlewis on January 08, 2007, 10:32:52 PM: Here's what I see in /usr/bin/config (This date is just before I upgraded to 7.1):

Code: [Select]
[root@gluon var]# ll /usr/bin/config total 1580 -rw------- 1 qpsmtpd qpsmtpd 2519040 Jan 2 22:13 denysoft_greylist.dbm -rw------- 1 qpsmtpd qpsmtpd 0 Jan 2 22:13 denysoft_greylist.dbm.lock

So I changed ownership of the directory to qpsmtpd:qpsmtpd (and removed the lock file...)

I agree it is unusual, hence the question. I'm not geeky enough to interpret /usr/share/qpsmtpd/plugins/greylisting to determine where it stores its dbm files.

-JL
Title: activating greylisting
Post by: william_syd on January 09, 2007, 01:19:32 AM: Code: [Select]
# enable greylisting mkdir -p /usr/bin/config chown qpsmtpd:qpsmtpd /usr/bin/config mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/ echo greylisting black_timeout 60 > /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/09greylisting /sbin/e-smith/expand-template /var/service/qpsmtpd/config/plugins
The above is what I had in 7.0

Below is what I added for 7.1 (in addition to the above)
Code: [Select]
mkdir -p /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0 ln -s /etc/e-smith/templates-custom/var/service/qpsmtpd/config/plugins/09greylisting /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/09greylisting /sbin/e-smith/expand-template /var/service/qpsmtpd/config/peers/0 signal-event email-update

Appears to work.
Title: greylisting
Post by: mark on February 14, 2007, 06:48:35 AM: Hi

I live in New Zealand

99% of spam my clients recieve comes from non nz addresses and about 5% of legitimate email comes from non nz addresses. Can anyone think of how I could limit the calling to the greylisting module to only when an email has a non nz address. In other words: if not address contains *.nz then greylist?

thanks

Mark Signal
Title: greylisting
Post by: mark on February 15, 2007, 08:13:13 PM: I activated greylisting as the last activity in peers/0 and activated whitelist_soft immediately before it. I then added *.nz to /var/service/qpsmtpd/config/whitelistsenders.

greylisting is now working but not ignoring email addresses ending in .nz as I would have expected

do I need to tell greylisting to use the results of the whitelist_soft check or should the fact that it appears after whitelist_soft in the peers/0 file be enough?

cheers

Mark Signal
Title: Re: activating greylisting
Post by: jahlewis on November 13, 2007, 06:50:24 PM: how did you activate whitelistsoft?
Title: Re: activating greylisting
Post by: mark on November 21, 2007, 12:08:22 AM: Quote from: jahlewis on November 13, 2007, 06:50:24 PM
how did you activate whitelistsoft?

be lazy like me and install this

http://ftp.surfnet.nl/ftp/pub/os/Linux/distr/smeserver/contribs/dmay/smeserver/7.x/testing/smeserver-wbl/smeserver-wbl-0.0.1-a8.dmay.noarch.rpm

thanks Darrel May for yet another excellent contrib