Topic: How can I test if GeoIP is really working?

[joost]

Hi, hope you can help.

I installed the contrip GeoIP according to the HOW TO. After installation I added the following countries, this is the output of:

Code: [Select]

config show qpsmtpd

Code: [Select]

qpsmtpd=service
    Authentication=disabled
    BadCountries=CO,AR,RU,BR,BG,KR,TW,CN,IN,TR,NE
    Bcc=disabled
    BccMode=cc
    BccUser=maillog
    DNSBL=enabled
    GeoIP=enabled
    Instances=40
    InstancesPerIP=5
    LogLevel=6
    MaxScannerSize=25000000
    MaximumDateOffset=0
    PatternsScan=enabled
    Proxy=blocked
    RBLList=b.barracudacentral.org,bl.spamcop.net,dnsbl-1.uceprotect.net,dnsbl-2.uceprotect.net,psbl.surriel.com,zen.spamhaus.org
    RHSBL=enabled
    RelayRequiresAuth=enabled
    SBLList=multi.surbl.org,black.uribl.com,rhsbl.sorbs.net
    TCPPort=25
    TCPProxyPort=25
    TlsBeforeAuth=1
    UBLList=multi.surbl.org:8-16-64-128,black.uribl.com,rhsbl.sorbs.net
    URIBL=disabled
    VirusScan=enabled
    access=public
    qplogsumm=disabled
    status=enabled
    tnef2mime=enabled

After running the script: geoipstats.sh I still get the following countries:  RU, CN.

And I am not able to send mail, unless I restart sqpsmtpd with:

Code: [Select]

service sqpsmtpd restart

De command:

Code: [Select]

netstat -an |grep :465.*EST

still shows IP-adresses form BR, CN and RU.

For example:

Code: [Select]

geoiplookup 191.36.149.53

Output:

Code: [Select]

GeoIP Country Edition: BR, Brazil
GeoIP City Edition, Rev 1: BR, 23, Rio Grande do Sul, Sao Gabriel, 97300, -30.333700, -54.330299, 0, 0
GeoIP ASNum Edition: AS263333 VIPTURBO COMÉRCIO & SERVIÇOS DE INFORMÁTICA LTDA

So it looks like its not working, but I'm not really sure. Please help.

[ReetP]

Please read this on how to report things effectively and how to check things yourself.

Especially the sections on logging.

https://forums.koozali.org/index.php/topic,54724.0.html

Did you set up the Maxmind v2 DBs?

Code: [Select]

config show geoip
(Remove the real accountID and key)

Code: [Select]

geoiplookup 191.36.149.53
That gave an answer so it looks like it is fundamentally working.

Check your qpsmptd logs for geoip entries.

Look for these:

Code: [Select]

grep -i check_badcountries /var/log/qpsmtpd/current
You may be better off using this which is more powerful as it can block multiple services, not just mail.

https://wiki.koozali.org/Xt_geoip

[Jean-Philippe Pialasse]

geoiplookup 191.36.149.53

That gave an answer so it looks like it is fundamentally working.

not true, as the default geoip install for looking ip will use db from a rpm. while xt_geoip will need to download raw list to convert them to its format and hence need a full free registration to Maxmind.
so as pointed by Reetp follow the wiki page to set up correctly. 

this will tell you if the xt_geoip is indeed loaded:
lsmod|grep xt



this will tell you if the firewall rules are effective:
iptables -L |grep -i geoip

and log will show there:
tail -f /var/log/iptables/denylog.log|grep -i geoip

[ReetP]

not true, as the default geoip install for looking ip will use db from a rpm.

AFAIA it is basically working, but results may be wrong/out of date if it is using the old Maxmind v1 DBs?

We should see the output of this to see which version of Geoip is installed:

Code: [Select]

/sbin/e-smith/audittools/newrpms
Also:

Code: [Select]

config show geoip
(Obfuscate the ID & Key)

xt_geoip is only relevant if it is installed but the OP did say

contrip GeoIP

so assuming right now it is not.

[bunkobugsy]

Check spamfilter-stats after a day or two.

or doing a manual update in verbose mode

or check date on downloaded v2 database file

[joost]

Thanks. I solved it with your help.

[ReetP]

Thanks. I solved it with your help.

Please tell us your solution for others.

[joost]

Please tell us your solution for others.

My real problem was that It was impossible to send mails. Only after a restart of qpsmtpd I could send mail. Receiving mail wasn't any problem. And sending mail worked before.

In the logfiles I saw a lot of unkonwn IP-adresses so I installed GeoIP. This didn't solve the problem, so I thought GeoIP wasn't working.

The command:

Code: [Select]

geoiplookup 191.36.149.53
worked.

Code: [Select]

grep -i check_badcountries /var/log/qpsmtpd/current
showed me check_badcountries in the logfile.

So I concluded that it probably worked but it wasn't enough to block only smtp-service. I decided to follow the tip of installing

https://wiki.koozali.org/Xt_geoip

This seems t work. I will give it a couple of days before I jump to conclusions. Contacting the smtp-server still takes a long time but the mail gets send I I don't have to restart the sqsmtpd service every time.

Code: [Select]

lsmod|grep xts
shows xt_geoip

Code: [Select]

iptables -L |grep -i geoip
shows GeoIP bans

and

Code: [Select]

tail -f /var/log/iptables/denylog.log|grep -i geoip
also shows bans.

So I think/hope my problem is solved.

[ReetP]

My real problem was that It was impossible to send mails. Only after a restart of qpsmtpd I could send mail. Receiving mail wasn't any problem. And sending mail worked before.

That is odd.

What was happening here (prior to install xt_geoip?) Were you blocking your own clients?

Code: [Select]

grep -i check_badcountries /var/log/sqpsmtpd/current 

So I think/hope my problem is solved.

I would check that properly rather than hope. Just jumping to a new system doesn't necessarily fix your issues (see xyinfo problems) and you should try to understand what had gone wrong first.

as I requested earlier please paste the output as requested above. This helps us enormously with debugging:

Code: [Select]

/sbin/e-smith/audittools/newrpms
/sbin/e-smith/audittools/templates

{Edited the title to fix the typo}

[Peasant]

Just to put in my 2p's worth, I was having similar problems to the OP. For some time I have noticed that it was taking longer to send a message from an email client that it had done, and it got to the point yesterday when I was getting timeout messages on port 465 from Thunderbird. On examining the /var/log/sqpsmtpd/current log file there were loads of 'too many connections' messages. I installed XT-Geoip and configured it, and it appears to be working. I still get a delay in sending emails, but the 'too many connections' messages have all but disappeared. I've probably still got some fine tuning to do with the countries to be blocked.

Oh, and Happy New Year to you all!

[ReetP]

I still get a delay in sending emails, but the 'too many connections' messages have all but disappeared.

Likely related to these settings in qpsmtpd and sqpsmtpd.

Code: [Select]

config show qpsmtpd InstancesPerIP
config show sqpsmtpd Instances

eg

Code: [Select]

    Instances=40
    InstancesPerIP=8

Defaults:

See

Code: [Select]

cat /var/service/qpsmtpd/runenv
cat /var/service/sqpsmtpd/runenv

sqpsmtpd
INSTANCES=10
INSTANCES_PER_IP=5

qpsmtpd
INSTANCES=40
INSTANCES_PER_IP=5


I think you get hammered by scammers trying to authenticate to send mail - max 5 connections from one IP - and they use up your connection limit so you are then blocked from sending - effectively a denial of service.

xt_geoip will limit the actual number of connections and free up the mailserver/authentication for genuine connections.

I'm not sure why this is different to authent with the old cvm-unix.

JP can probably clarify all this and will know some answers.

Oh, and Happy New Year to you all!

And to you!!

[joost]

Sorry for the late reply. Hereby the requested output.

as I requested earlier please paste the output as requested above. This helps us enormously with debugging:

Code: [Select]
/sbin/e-smith/audittools/newrpms
/sbin/e-smith/audittools/templates

Code: [Select]

/sbin/e-smith/audittools/newrpms
/sbin/e-smith/audittools/templates

Output:

Code: [Select]

/sbin/e-smith/audittools/newrpms
Geladen plugins: fastestmirror, post-transaction-actions, priorities, smeserver
Loading mirror speeds from cached hostfile
 * base: mirror.proserve.nl
 * updates: mirror.proserve.nl
Extra pakketten
GeoIP.x86_64                               1.6.12-9.el7.sme         @smecontribs
GeoIP-GeoLite-data.noarch                  2018.06-7.el7.sme        @smecontribs
GeoIP-GeoLite-data-extra.noarch            2018.06-7.el7.sme        @smecontribs
fail2ban-sendmail.noarch                   0.11.2-3.el7             @smecontribs
fail2ban-server.noarch                     0.11.2-3.el7             @smecontribs
openvpn.x86_64                             2.4.12-1.el7             @smecontribs
perl-B-Hooks-EndOfScope.noarch             0.24-1.of.el7            @smecontribs
perl-B-Hooks-OP-Check.x86_64               0.22-1.of.el7            @smecontribs
perl-Class-Load-XS.x86_64                  0.10-1.of.el7            @smecontribs
perl-Class-Method-Modifiers.noarch         2.13-1.of.el7            @smecontribs
perl-Class-XSAccessor.x86_64               1.19-2.el7               @smecontribs
perl-Clone-PP.noarch                       1.06-1.of.el7            @smecontribs
perl-Data-Dumper-Concise.noarch            2.023-1.of.el7           @smecontribs
perl-Data-IEEE754.noarch                   0.01-1.of.el7            @smecontribs
perl-Data-Printer.noarch                   0.35-1.of.el7            @smecontribs
perl-Data-Validate-IP.noarch               0.27-13.el7              @smecontribs
perl-DateTime.x86_64                       2:1.55-1.of.el7          @smecontribs
perl-DateTime-Locale.noarch                1.33-1.of.el7            @smecontribs
perl-DateTime-TimeZone.noarch              2.51-1.of.el7            @smecontribs
perl-Devel-GlobalDestruction.noarch        0.14-1.of.el7            @smecontribs
perl-Devel-OverloadInfo.noarch             0.007-1.of.el7           @smecontribs
perl-Devel-StackTrace.noarch               1:2.04-1.of.el7          @smecontribs
perl-Eval-Closure.noarch                   0.14-1.of.el7            @smecontribs
perl-File-HomeDir.noarch                   1.002-1.of.el7           @smecontribs
perl-Geo-IP.x86_64                         1.45-1.of.el7            @smecontribs
perl-GeoIP2.noarch                         2.001002-1.of.el7        @smecontribs
perl-Hash-FieldHash.x86_64                 0.14-1.of.el7            @smecontribs
perl-Lexical-SealRequireHints.x86_64       0.011-1.of.el7           @smecontribs
perl-List-AllUtils.noarch                  0.08-1.of.el7            @smecontribs
perl-MRO-Compat.noarch                     0.12-2.el7               @smecontribs
perl-Math-Int128.x86_64                    0.18-1.of.el7            @smecontribs
perl-Math-Int64.x86_64                     0.52-1.el7               @smecontribs
perl-MaxMind-DB-Common.noarch              0.040000-1.of.el7        @smecontribs
perl-MaxMind-DB-Reader.noarch              1.000004-1.of.el7        @smecontribs
perl-Module-Implementation.noarch          0.09-1.of.el7            @smecontribs
perl-Module-Runtime.noarch                 0.016-1.of.el7           @smecontribs
perl-Module-Runtime-Conflicts.noarch       0.003-1.of.el7           @smecontribs
perl-Moo.noarch                            2.004004-2.of.el7        @smecontribs
perl-MooX-StrictConstructor.noarch         0.006-1.of.el7           @smecontribs
perl-Moose.x86_64                          2.2015-1.of.el7          @smecontribs
perl-Net-OpenVPN-Manage.noarch             0.02-2.el7.sme           @smecontribs
perl-Net-Works.noarch                      0.21-1.of.el7            @smecontribs
perl-Params-Classify.x86_64                0.013-7.el7              @smecontribs
perl-Params-Validate.x86_64                1.30-1.of.el7            @smecontribs
perl-Params-ValidationCompiler.noarch      0.30-1.of.el7            @smecontribs
perl-Role-Tiny.noarch                      2.001004-1.of.el7        @smecontribs
perl-Sort-Naturally.noarch                 1.03-8.el7               @smecontribs
perl-Specio.noarch                         0.47-1.of.el7            @smecontribs
perl-Sub-Exporter-Progressive.noarch       0.001013-1.of.el7        @smecontribs
perl-Sub-Identify.x86_64                   0.14-1.of.el7            @smecontribs
perl-Sub-Install.noarch                    0.928-1.of.el7           @smecontribs
perl-Sub-Name.x86_64                       0.26-1.of.el7            @smecontribs
perl-Sub-Quote.noarch                      2.006006-1.of.el7        @smecontribs
perl-Test-Warnings.noarch                  0.031-1.of.el7           @smecontribs
perl-Throwable.noarch                      1.000-1.of.el7           @smecontribs
perl-Try-Tiny.noarch                       0.22-1.of.el7            @smecontribs
perl-Variable-Magic.x86_64                 0.62-1.of.el7            @smecontribs
perl-bareword-filehandles.x86_64           0.007-1.of.el7           @smecontribs
perl-indirect.x86_64                       0.39-1.of.el7            @smecontribs
perl-multidimensional.x86_64               0.014-1.of.el7           @smecontribs
perl-namespace-autoclean.noarch            0.29-1.of.el7            @smecontribs
perl-namespace-clean.noarch                0.27-1.of.el7            @smecontribs
perl-strictures.noarch                     2.000006-1.of.el7        @smecontribs
phpMyAdmin.noarch                          5.1.0-1.el7.sme          @smecontribs
phpki-ng.noarch                            0.84-16.el7.sme          @smecontribs
phpldapadmin.noarch                        1.2.6-4.el7.sme          @smecontribs
pkcs11-helper.x86_64                       1.11-3.el7               @smecontribs
smeserver-bridge-interface.noarch          0.2-9.el7.sme            @smecontribs
smeserver-dhcpmanager.noarch               2.0.4-12.el7.sme         @smecontribs
smeserver-fail2ban.noarch                  9:0.1.18-30.el7.sme      @smecontribs
smeserver-openvpn-bridge.noarch            2.1-22.el7.sme           @smecontribs
smeserver-phpki-ng.noarch                  0.3-22.el7.sme           @smecontribs
smeserver-phpldapadmin.noarch              1.6.0-9.el7.sme          @smecontribs
smeserver-phpmyadmin.noarch                4.0.10.2-13.el7.sme      @smecontribs
smeserver-vacation.noarch                  1.1-34.el7.sme           @smecontribs
smeserver-wbl.noarch                       0.5.0-7.el7.sme          @smecontribs
smeserver-xt_geoip.noarch                  1.3.1-19.el7.sme         @smecontribs
xtables-addons.x86_64                      2.14-01.el7.sme          @smecontribs
xtables-addons-kmod.x86_64                 2.14-05.el7.sme          @smecontribs


Code: [Select]

/sbin/e-smith/audittools/templates
/etc/e-smith/templates-custom/etc/rc.d/init.d/masq/40DenyIPAdresses: MANUALLY_ADDED, ADDITION
/etc/e-smith/templates-custom/etc/sysconfig/rsyslog/90AllowRemoteSyslog: MANUALLY_ADDED, ADDITION


I think you get hammered by scammers trying to authenticate to send mail - max 5 connections from one IP - and they use up your connection limit so you are then blocked from sending - effectively a denial of service.

I think this is the case. After installing xt_geoip everything works better but it still takes longer to send a message from an email client then before and it still times out sometimes. So the problem is not totally solved.

The instances are still set to default. Is it wise to change these to a larger number?

Code: [Select]

config show qpsmtpd InstancesPerIP
config show sqpsmtpd Instances

Defaults:

sqpsmtpd
INSTANCES=10
INSTANCES_PER_IP=5

qpsmtpd
INSTANCES=40
INSTANCES_PER_IP=5

Thanks

[ReetP]

Sorry for the late reply. Hereby the requested output.

We all have busy lives!

I think this is the case. After installing xt_geoip everything works better but it still takes longer to send a message from an email client then before and it still times out sometimes. So the problem is not totally solved.

Better to understand the issue than guess at a fix.

Check your logs:

Code: [Select]

/var/log/qpsmtpd/current
/var/log/sqpsmtpd/current

sqpsmtd handles outgoing mail (qmail too but the issue is likely the client connecting to the server to send)

Check the earlier link for ways to look at the logs for errors.

Also here:

https://wiki.koozali.org/User_talk:Stephdl#Deep_parse_Log_to_find_errors

Look for a transaction number eg

Code: [Select]

@40000000659ed831188e3f4c 6076 Accepted connection 0/10 from
 192.168.99.1 myserver.com

Then look at the entire transaction eg;

Code: [Select]

grep 6076 /var/log/sqpsmtpd/current |tai64nlocal

Any, delays or errors?

If you can nail the cause we can look at the right fix.