Koozali.org: home of the SME Server

GEOIP plugin for qpsmtpd -find email by country code

Offline kruhm

  • *
  • 680
  • +0/-0
GEOIP plugin for qpsmtpd -find email by country code
« on: August 20, 2006, 07:07:57 AM »
# GEOIP QPSMTPD PLUGIN
# The GEOIP plugin lets us know where our mail server is receiving mail from.
# If we're receiving too much spam from a particular place, this will help track it down.
# We can then use that info to reject connections from that place taking the load off our server.

# MODIFY THE GEOIP PLUGIN
# Edit the GEOIP plugin.
vi /usr/share/qpsmtpd/plugins/ident/geoip
# Modify the plugin to make it look like the following:
use Geo::IP;
my $geoip = Geo::IP->new(GEOIP_STANDARD);
 
sub hook_connect {
        my ($self) = @_;
        my $country = $geoip->country_code_by_addr
                ($self->qp->connection->remote_ip);
        $self->qp->connection->notes('geoip_country', $country);
        $self->log(LOGNOTICE, "GeoIP Country: $country");
        if ( $self->qp->config("badcountries") ) {
                my @badcountries = $self->qp->config("badcountries");
 
                for (@badcountries) {
                        my ($pattern, $response) = split /\s+/, $_, 2;
                        return (DENY, $response) if ($country eq $pattern);
                }
        }
 
        return (DECLINED);
}

# INSTALL THE GEOIP
# We need the GEOIP package and the perl interface to the program but this isn't installed on SME.
# We'll have to grab the packages from yum.
# Yum has access to different public repositories where packages are available. GEOIP is in the EXTRAS repo.
# We'll enable the repo and install them.
yum --enablerepo=extras install perl-Geo-IP
# Yum does the magic and knows to install both the program and the interface.

# INSTALL THE GEOIP DATABASE
# We also need the GEOIP DATABASE. This database is updated monthly by a company called MaxMind.
# We'll have to download it every month or pay for their subscription service to be accurate.
# The database needs to be in a specific location or it won't work. We'll change to that location.
cd /
cd /var/lib/GeoIP
# Now we'll get the latest database. The database is also in the repositories but it's outdated.
# We'll grab the most recent directly from MaxMind.
wget http://www.maxmind.com/download/geoip/database/GeoIP.dat.gz
# The database is zipped. We'll have to unzip it.
gunzip GeoIP.dat.gz

# TEST THE GEOIP
# Now that the package and database are installed, we can test it.
geoiplookup 216.17.211.37
# It should return: GeoIP Country Edition: US, United States
# It gives us the country code (US) and the long name (United States)
# Let's test it again with a domain name.
geoiplookup contribs.org
# Same result. So we know it works with IP ADDRESSES or DOMAIN NAMES.
# Let's test it again around the world.
geoiplookup gormand.com.au
# It should return: GeoIP Country Edition: AU, Australia
geoiplookup e-smith.com
# It should return: GeoIP Country Edition: CA, Canada
geoiplookup swerts-knudsen.dk
# It should return: GeoIP Country Edition: DK, Denmark

# ENABLE THE GEOIP QPSMTPD PLUGIN
# The email receiving component of SME is called QPSMTPD.
# It's great because it allows us to turn plugins on or off or create our own when we need.
# The GEOIP plugin is already in SME but it's turned off. We'll create a template to turn it on. First create the directory.
mkdir -p /etc/e-smith/templates/var/service/qpsmtpd/config/peers/0
# Now let's add the template file.
vi /etc/e-smith/templates/var/service/qpsmtpd/config/peers/0/18check_geoip
# Insert the following:
Code: [Select]

{
    return "# geoip disabled" unless (${qpsmtpd}{GeoIP} eq "enabled");

    "ident/geoip";
}

# Add the value to the SME CADNHO db.
config setprop qpsmtpd GeoIP enabled
# Expand the template.
expand-template /var/service/qpsmtpd/config/peers/0
# Restart the qpsmtpd service.
service qpsmtpd restart
# GEOIP plugin should now do its work.
# Check the qpsmtpd logs and you'll see the countries from where mail is sent.
cat /var/log/qpsmtpd/current

# GREP THE LOGS FOR COUNTRIES
# We'll use a simple shell script to do the work then we'll run it.
# First create the the script.
vi geoipstats.sh
# Insert the following:
Code: [Select]

#!/bin/sh

# Read the qpsmtpd log file.
cat /var/log/qpsmtpd/* | \

# Read all of the countries and count them.
grep 'GeoIP Country:' |
sed -e 's/^.*\(..\)$/\1/' |
sort | uniq -c | sort -n

# Now run the script. It will show the number of messages sent by country code.
sh geoipstats.sh

# BLOCK BAD COUNTRIES
# Now we need to add a template to control the BADCOUNTRIES.
# First create the dir
mkdir -p /etc/e-smith/templates/var/service/qpsmtpd/config/badcountries
# Now let's add the template file.
vi /etc/e-smith/templates/var/service/qpsmtpd/config/badcountries/10badcountries
# Insert the following:
{
    my @badcountries = split /[,:]/, ${qpsmtpd}{BadCountries} || '';

    return "# No BadCountries are defined" unless (scalar @badcountries);

    return join "\n", @badcountries;
}
# Add the value to the SME CADNHO db.
config setprop qpsmtpd BadCountries <country code>
# Expand the template.
expand-template /var/service/qpsmtpd/config/badcountries
# Restart the service
service qpsmtpd restart

# TODO
-add templates into the email-update event
-roll into rpm
-possibly map connections
-possibly grep mail results and push to graph3d
-possibly automatically choose website language depending on country code

Offline byte

  • *
  • 2,183
  • +2/-0
GEOIP plugin for qpsmtpd -find email by country code
« Reply #1 on: August 20, 2006, 12:19:07 PM »
Kruhm,

Have you raised a NFR? I'm sure the Dev's would like to know what/where you are going with this.
--[byte]--

Have you filled in a Bug Report over @ http://bugs.contribs.org ? Please don't wait to be told this way you help us to help you/others - Thanks!

Offline kruhm

  • *
  • 680
  • +0/-0
GEOIP plugin for qpsmtpd -find email by country code
« Reply #2 on: August 20, 2006, 02:24:27 PM »
added.

Offline Alex Schaft

  • ****
  • 97
  • +0/-0
Re: GEOIP plugin for qpsmtpd -find email by country code
« Reply #3 on: October 03, 2006, 02:47:46 PM »
Quote from: "kruhm"

# Add the value to the SME CADNHO db.
config setprop qpsmtpd BadCountries <country code>
# Expand the template.
expand-template /var/service/qpsmtpd/config/plugins
# Restart the service
service qpsmtpd restart


Hi,

Thanks for this. One more barrier against spam :)

Just one comment. Should the above expansion not be:

Code: [Select]
expand-template /var/service/qpsmtpd/config/badcountries

Regards,
Alex
......

Offline kruhm

  • *
  • 680
  • +0/-0
GEOIP plugin for qpsmtpd -find email by country code
« Reply #4 on: October 04, 2006, 02:01:15 PM »
changed

Offline kruhm

  • *
  • 680
  • +0/-0
GEOIP plugin for qpsmtpd -find email by country code
« Reply #5 on: January 11, 2007, 05:35:06 AM »
updated the above to reflect the new home for the plugins

Offline stephen noble

  • *
  • 607
  • +1/-0
    • Dungog
GEOIP plugin for qpsmtpd -find email by country code
« Reply #6 on: January 11, 2007, 07:05:46 AM »
Good Work

unless you have a very good reason don't use templates-custom
see the first sticky post

don't make a new rpm, just attach the fragments to the bug
clarify what links to make (for the devs)
and what db settings (for users)
and it can go straight into the base

maybe GeoIP.dat.gz  could be an rpm
maybe a script to d/l and extract would be better, we need one anyway to update

Offline Alex Schaft

  • ****
  • 97
  • +0/-0
Re: GEOIP plugin for qpsmtpd -find email by country code
« Reply #7 on: April 29, 2009, 04:34:43 PM »
Just a suggestion, but could the Wiki page be updated with the modified qpsmtp plugin?

Alex
......

Offline Knuddi

  • *
  • 540
  • +0/-0
    • http://www.scanmailx.com
Re: GEOIP plugin for qpsmtpd -find email by country code
« Reply #8 on: April 30, 2009, 04:53:11 PM »
Why not just use SpamAssassin for this? It has it build-in:

# RelayCountry - add metadata for Bayes learning, marking the countries
# a message was relayed through
#
# Note: This requires the IP::Country::Fast Perl module
#
# loadplugin Mail::SpamAssassin::Plugin::RelayCountry


See:
http://wiki.apache.org/spamassassin/RelayCountryPlugin

Offline Stefano

  • *
  • 10,894
  • +3/-0
Re: GEOIP plugin for qpsmtpd -find email by country code
« Reply #9 on: April 30, 2009, 06:32:03 PM »
I think that the proper/only place to discuss such a thing is bugzilla and the dev's ML

my 2c

Ciao
Stefano