Topic: Block IP list

[Mouse]

Hi

SME 10 now has attack on IMAP from hundreds of IP around world.
I put Fail2ban to 4 days block, but it doesn’t help much.
Are some possibility block IP lists (lots find on internet - like https://github.com/stamparm/ipsum)

GeoIP is maybe ok, but I can’t block all world - in attack IP is almost from all world.

I want somehow block possibility to log in server/mail from all world except some 3-4 countries. But in same time receive mail from anywhere.

[Gary Douglas]

xt-geoip is your friend; https://wiki.koozali.org/Xt_geoip

[mmccarn]

[here be dragons]

When SMTP & IMAP were still accessible on my SME server I put a lot of work into finding curated block lists.

One option I used for a while was the blocklists maintained by the firehol firewall project - my notes on that are in this forum post: https://forums.koozali.org/index.php?topic=53302.0. Sadly, the firehol project appears to have no updates since 2018...

I switched from firehol to "emerging threats" at some point (maybe the firehol blocks mostly came from the emerging threat list?) -  https://rules.emergingthreats.net. 

Emerging Threats used to provide a bash script to download and update ipset block lists, but I can't find this any more...

I made some modifications to the old bash script -- you can see my version on github: https://raw.githubusercontent.com/mmccarn/smeserver/master/bin/emerging-ipset-update.sh

My script appears to be working on a non-sme system (Ubuntu 22 hosting nextcloud).

The script does this:
- get the latest rule version from https://rules.emergingthreats.net/fwrules/FWrev
- compare the "Rev" to the last downloaded version
- if Rev is different:
  - download the emerging threat IP block list from https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
  - create new ipset lists using the new data
  - swap the new ipset lists with the old lists

This first script does nothing to the firewall All it does is create and update two ipset lists (et-blacklist and et-blacklistnet)

Blocking the IPs included in the new lists requires a custom template fragment -
/etc/e-smith/templates-custom/etc/rc.d/init.d/masq/40emerging-threats-blocklist

My version can be seen here:
https://raw.githubusercontent.com/mmccarn/smeserver/master/templates-custom/etc/rc.d/init.d/masq/40emerging-threats-blocklist

[mmccarn]

This bash script
* gets the names of any ipset lists on your system
* scans x days of log files in /var/log/iptables
* outputs the count of entries in the list, the count of connections blocked by the list, and the count of "uniquips" (I have no recollection at all of what this means...)

#!/bin/bash
DAYS=${DAYS:-1}
lists=$*
if [ -z "$lists" ]
  then
    lists=$(ipset list | awk ' /Name/ {print $2}')
fi
#echo lists: \"$lists\"
for l in $lists;
  do
    listcount=$(ipset list $l |egrep -c "^[0-9]")
    blockcount=$(cat $(find /var/log/iptables/ -type f -ctime -$DAYS) | grep -ch "\-$l" )
    uniquips=$(cat $(find /var/log/iptables/ -type f -ctime -$DAYS) | grep -h  "\-$l" |sed s/^.*SRC\=// |sed s/\ .*$// |sort -u |wc -l)
    echo -e "$listcount\t$blockcount\t$uniquips\t$l"
  done

Here's the output on my system after running "emerging-ipset-update.sh" from the last post (my server has no access, so the 2nd and 3rd numbers are zero...)

Code: [Select]

# /root/bin/ipsets.sh
4 0 0 et-blacklist
1412 0 0 et-blacklistnet


[mmccarn]

This might be easier...

I found this script online that takes a list of IPs and adds it to the fail2ban jail of your choice:
https://github.com/cherdt/fail2ban-blocklist/blob/master/etc/fail2ban/blocklist.sh

That script is using the blocklist.de list for the ssh jail (https://lists.blocklist.de/lists/ssh.txt).

You could edit the script to use the imap jail and the imap list (https://lists.blocklist.de/lists/imap.txt)

[Mouse]

Thank You.

I try to do something from Your sugestions

[bunkobugsy]

Xt-geoip and allow IMAPS (993) only from your home country, when users are roaming they need to use webmail or they might get home IPs from their mobile carrier.

[ReetP]

when users are roaming they need to use webmail or they might get home IPs from their mobile carrier.

Or VPN to the server or a local VPS.

[Gary Douglas]

another one i do is change imaps port;
svr:~# config show imaps
imaps=configuration
    BadCountries=GB
    TCPPort=33333
    XTGeoipRev=enabled
    access=public
    status=enabled

[Jean-Philippe Pialasse]

if you go the block list way,
i would suggest the approach with the template rather than the fail2ban add of huge list of ip.

fail2ban will end up adding each ip one at a time blocking firewall and system for hours trying to update both iptable and a flat file db and his own db.
this would reproduce this kind of freezing on every event restarting fail2ban


this could be a nice simple contrib

[Stefano]

I'm using this approach on my personal server
https://github.com/trick77/ipset-blacklist
When I'll get back home from vacation Will share my setup

[Jean-Philippe Pialasse]

another one i do is change imaps port;
svr:~# config show imaps
imaps=configuration
    BadCountries=GB
    TCPPort=33333
    XTGeoipRev=enabled
    access=public
    status=enabled

because port scanner does not exist.

you are probably safer from kiddies only by blocking non GB ips.
Also pay attention that some firewalls, like hospitals, will block all port except some legitimate known port so you might prevent legitimate traffic only because you do not use standard port.

[Stefano]

here's my setup with ipset
1) download and setup as per instructions https://github.com/trick77/ipset-blacklist
2) my /etc/ipset-blacklist/ipset-blacklist.conf is this:

Code: [Select]

IPSET_BLACKLIST_NAME=blacklist # change it if it collides with a pre-existing ipset list
IPSET_TMP_BLACKLIST_NAME=${IPSET_BLACKLIST_NAME}-tmp

# ensure the directory for IP_BLACKLIST/IP_BLACKLIST_RESTORE exists (it won't be created automatically)
IP_BLACKLIST_RESTORE=/etc/ipset-blacklist/ip-blacklist.restore
IP_BLACKLIST=/etc/ipset-blacklist/ip-blacklist.list

VERBOSE=yes # probably set to "no" for cron jobs, default to yes
FORCE=yes # will create the ipset-iptable binding if it does not already exist
let IPTABLES_IPSET_RULE_NUMBER=1 # if FORCE is yes, the number at which place insert the ipset-match rule (default to 1)

# Sample (!) list of URLs for IP blacklists. Currently, only IPv4 is supported in this script, everything else will be filtered.
BLACKLISTS=(
    # "file:///etc/ipset-blacklist/ip-blacklist-custom.list" # optional, for your personal nemeses (no typo, plural)
    "https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1" # Project Honey Pot Directory of Dictionary Attacker IPs
    "https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=1.1.1.1"  # TOR Exit Nodes
    "http://danger.rulez.sk/projects/bruteforceblocker/blist.php" # BruteForceBlocker IP List
    "https://www.spamhaus.org/drop/drop.lasso" # Spamhaus Don't Route Or Peer List (DROP)
    "https://cinsscore.com/list/ci-badguys.txt" # C.I. Army Malicious IP List
    "https://lists.blocklist.de/lists/all.txt" # blocklist.de attackers
    "https://blocklist.greensnow.co/greensnow.txt" # GreenSnow
    "https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level1.netset" # Firehol Level 1
    "https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/stopforumspam_7d.ipset" # Stopforumspam via Firehol
    # "https://raw.githubusercontent.com/ipverse/rir-ip/master/country/zz/ipv4-aggregated.txt" # Ban an entire country(-code), see https://github.com/ipverse/rir-ip
    # "https://raw.githubusercontent.com/ipverse/asn-ip/master/as/1234/ipv4-aggregated.txt" # Ban a specific autonomous system (ISP), see https://github.com/ipverse/asn-ip
)
MAXELEM=196608
be aware that default MAXELEM value can be too small, and be aware too that this file is not templated (and not included in standard backup set)

3) create the following fragment in /etc/e-smith/templates-custom/etc/rc.d/init.d/masq path

Code: [Select]

nano 40DenyRiffRaff
/sbin/iptables -I INPUT 1 -m set --match-set blacklist src -j DROP

4) if you're using fail2ban, you must create 2 custom fragments in the same folder:

Code: [Select]

nano 40Fail2Ban

    # A blacklist chain for fail2ban
    /sbin/iptables --new-chain Fail2Ban
    /sbin/iptables --new-chain Fail2Ban_1
    /sbin/iptables --append Fail2Ban -j Fail2Ban_1
    /sbin/iptables --insert INPUT 2 \
        -j Fail2Ban
(we're forcing fail2ban's iptables rule at position 2)

Code: [Select]

nano 90adjustFail2Ban

{
    my $f2bdb = esmith::ConfigDB->open_ro('fail2ban') ||
        esmith::ConfigDB->create('fail2ban');
    # Find the current Fail2Ban_$$ chain, and create a new one.
    $OUT .=<<'EOF';
    OLD_Fail2Ban=$(get_safe_id Fail2Ban filter find)
    NEW_Fail2Ban=$(get_safe_id Fail2Ban filter new)
    /sbin/iptables --new-chain $NEW_Fail2Ban
EOF

    if ( ($fail2ban{'status'} || 'disabled') eq 'enabled' ){
        foreach my $ban ( $f2bdb->get_all_by_prop(type=>('ban')) ){
            my $ip = $ban->prop('Host');
            my $proto = $ban->prop('Protocol') || '';
            my $port = $ban->prop('Port') || '';
            $OUT .= "    /sbin/iptables --append \$NEW_Fail2Ban -s $ip";
            $OUT .= " -p $proto" if ($proto =~ m/^tcp|udp|icmp$/);
            $OUT .= " -m multiport --dports $port" if ($proto =~ m/^tcp|udp$/ && $port =~ m/^\d+(,\d+)*$/);
            $OUT .= " -j denylog\n";
        }
        $OUT .= "    /sbin/iptables --append  \$NEW_Fail2Ban" .
                " -j RETURN\n";
    }

    # Having created a new Fail2Ban chain, activate it and destroy the old.
    $OUT .=<<'EOF';
    /sbin/iptables --replace Fail2Ban 2 \
            --jump $NEW_Fail2Ban
    /sbin/iptables --flush $OLD_Fail2Ban
    /sbin/iptables --delete-chain $OLD_Fail2Ban
EOF

}

5)  create 2 more custom fragments in /etc/e-smith/templates-custom/etc/crontab/ path:

Code: [Select]

nano 99_ipset

33 23 * * *      root /usr/local/sbin/update-blacklist.sh /etc/ipset-blacklist/ipset-blacklist.conf
for dailiy update
and

Code: [Select]

nano zz_boottime

@reboot /usr/sbin/ipset restore < /etc/ipset-blacklist/ip-blacklist.restore
to reload ipset at boot time

to be sure everything is correctly expanded, a post-upgrade event followed by a reboot will do the job

you can check the result with

Code: [Select]

iptables -L INPUT -v --line-numbers
command, which spits out something like this:

Code: [Select]

Chain INPUT (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     381K   19M DROP       all  --  any    any     anywhere             anywhere             match-set blacklist src
2      13M 4585M XTGeoIP    all  --  any    any     anywhere             anywhere
3      12M 4584M state_chk  all  --  any    any     anywhere             anywhere
4    1343K   93M Fail2Ban   all  --  any    any     anywhere             anywhere
(yes, ipset, geoip and fail2ban in place )

HTH

[Mouse]

HI

Thanks all

I use Xt-geoip and block ports 80,443,465,993.
I left open only country what I need !=LV,....
To receive mails don't block everything. (25 port must be opened)

now:
  Numbers of IPs banned (xt_geoip) by country during LAST MONTH
       ( XX means 'country not found' )

--------------------
XX | 172454 | 65.9%
US | 34930 | 13.4%
CN | 14709 | 5.6%
KR | 7865 | 3.0%
IN | 3904 | 1.5%
RU | 3131 | 1.2%
FR | 1549 | 0.6%
BR | 1515 | 0.6%
HK | 1368 | 0.5%
BG | 1368 | 0.5%

[ReetP]

Just a FYI but be aware I discovered that XT_Geoip seems to insert itself higher up the chain than deny RiffRaff.

So I had IPs apparently blocked by RiffRaff, but still saw them hitting Geoip.

The answer is in the geoip template code.

[Stefano]

Just a FYI but be aware I discovered that XT_Geoip seems to insert itself higher up the chain than deny RiffRaff.

So I had IPs apparently blocked by RiffRaff, but still saw them hitting Geoip.

The answer is in the geoip template code.

everything seems ok here

Code: [Select]

[root@sme9 ~]# rpm -qa | grep -i xt_geo
smeserver-xt_geoip-1.3.1-20.el7.sme.noarch
[root@sme9 ~]# iptables -L INPUT -v --line-numbers
Chain INPUT (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     265K   14M DROP       all  --  any    any     anywhere             anywhere             match-set blacklist src
2      11M 3886M XTGeoIP    all  --  any    any     anywhere             anywhere
3      11M 3885M state_chk  all  --  any    any     anywhere             anywhere
4    1132K   77M Fail2Ban   all  --  any    any     anywhere             anywhere
5    1132K   77M local_chk  all  --  any    any     anywhere             anywhere
6        0     0 denylog    all  --  any    any     base-address.mcast.net/4  anywhere
7        0     0 denylog    all  --  any    any     anywhere             base-address.mcast.net/4
8    86661 6781K InboundICMP  icmp --  any    any     anywhere             anywhere
9        0     0 denylog    icmp --  any    any     anywhere             anywhere
10    313K   16M InboundTCP  tcp  --  any    any     anywhere             anywhere             tcp flags:FIN,SYN,RST,ACK/SYN
11       0     0 denylog    tcp  --  any    any     anywhere             anywhere             tcp flags:FIN,SYN,RST,ACK/SYN
12   98523 9182K InboundUDP  udp  --  eth0   any     anywhere             anywhere
13       0     0 denylog    udp  --  eth0   any     anywhere             anywhere
14       0     0 ACCEPT     udp  --  dummy0 any     anywhere             anywhere             udp spts:bootps:bootpc
15   79176 4684K denylog    all  --  any    any     anywhere             anywhere


am I missing anything?

TIA

[ReetP]

everything seems ok here
am I missing anything?

Yes.

/etc/e-smith/templates/etc/rc.d/init.d/masq/40Xt_Geoip

Code: [Select]

{
  $OUT .=<<'EOF';
   # A blacklist chain for xtables-addons GEOIP
    /sbin/iptables --new-chain XTGeoIP
    /sbin/iptables --new-chain XTGeoIP_1
    /sbin/iptables --append XTGeoIP -j XTGeoIP_1
    /sbin/iptables --insert INPUT 1 \
       -j XTGeoIP
EOF
}


I think that this:

Code: [Select]

    /sbin/iptables --insert INPUT 1 \
Inserts the rule above every other rule - including RiffRaff.

So you only get to the Riffraff block AFTER GeoIP no matter where you add the RiffRaff template.

So effectively a RiffRaff block will be checked AFTER it is checked for GeoIP and will then ONLY block IPs that are permitted by GeoIP.

Try blocking a IP that you see blocked by GeoIP. You would think that GeoIP would never see it, but it is still there.

Took me ages to realise what was going on!

You could of course add say

Code: [Select]

--INSERT 1

to your RiffRaff rule to put it above GeoIP.

(all based on observation and my limited knoweledge of IPTables !!)

Proof if required.

Code: [Select]

[root@esmith masq]# iptables -L INPUT -v --line-numbers
Chain INPUT (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1     413M  164G XTGeoIP    all  --  any    any     anywhere             anywhere           
2     412M  164G Fail2Ban   all  --  any    any     anywhere             anywhere           
3     412M  164G state_chk  all  --  any    any     anywhere             anywhere           
4    4647K  313M local_chk  all  --  any    any     anywhere             anywhere           
5        0     0 denylog    all  --  any    any     base-address.mcast.net/4  anywhere           
6        0     0 denylog    all  --  any    any     anywhere             base-address.mcast.net/4
7        0     0 DROP       all  --  any    any     193.47.61.13         anywhere                           <<<< See this?
8    88733 3869K InboundICMP  icmp --  any    any     anywhere             anywhere           
9        0     0 denylog    icmp --  any    any     anywhere             anywhere
I have a config entry for my RiffRaff:

Code: [Select]

ipblock=configuration
    DenyHosts=193.47.61.13/32
So that DROP is after GeoIP.

[Stefano]

well, I have the same file but everything is working fine since more than 2 years

Code: [Select]

Chain INPUT (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     279K   14M DROP       all  --  any    any     anywhere             anywhere             match-set blacklist src
2      12M 4168M XTGeoIP    all  --  any    any     anywhere             anywhere
3      12M 4167M state_chk  all  --  any    any     anywhere             anywhere
4    1195K   82M Fail2Ban   all  --  any    any     anywhere             anywhere
5    1195K   82M local_chk  all  --  any    any     anywhere             anywhere
6        0     0 denylog    all  --  any    any     base-address.mcast.net/4  anywhere
7        0     0 denylog    all  --  any    any     anywhere             base-address.mcast.net/4
8    91804 7186K InboundICMP  icmp --  any    any     anywhere             anywhere
9        0     0 denylog    icmp --  any    any     anywhere             anywhere
10    323K   16M InboundTCP  tcp  --  any    any     anywhere             anywhere             tcp flags:FIN,SYN,RST,ACK/SYN
11       0     0 denylog    tcp  --  any    any     anywhere             anywhere             tcp flags:FIN,SYN,RST,ACK/SYN
12    108K   10M InboundUDP  udp  --  eth0   any     anywhere             anywhere
13       0     0 denylog    udp  --  eth0   any     anywhere             anywhere
14       0     0 ACCEPT     udp  --  dummy0 any     anywhere             anywhere             udp spts:bootps:bootpc
15   84130 4982K denylog    all  --  any    any     anywhere             anywhere


[ReetP]

well, I have the same file but everything is working fine since more than 2 years

My point was your IP block only operates AFTER processing by GeoIP.

I had imagined it would be first thereby saving GeoIP the overhead.

[Stefano]

My point was your IP block only operates AFTER processing by GeoIP.

I had imagined it would be first thereby saving GeoIP the overhead.

no, as you can see, the chain order is correct

Code: [Select]

[root@sme9 ~]# iptables -L INPUT -v --line-numbers
Chain INPUT (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     292K   15M DROP       all  --  any    any     anywhere             anywhere             match-set blacklist src
2      12M 4308M XTGeoIP    all  --  any    any     anywhere             anywhere
3      12M 4307M state_chk  all  --  any    any     anywhere             anywhere
4    1243K   85M Fail2Ban   all  --  any    any     anywhere             anywhere
5    1243K   85M local_chk  all  --  any    any     anywhere             anywhere
6        0     0 denylog    all  --  any    any     base-address.mcast.net/4  anywhere
7        0     0 denylog    all  --  any    any     anywhere             base-address.mcast.net/4
8    94934 7442K InboundICMP  icmp --  any    any     anywhere             anywhere
9        0     0 denylog    icmp --  any    any     anywhere             anywhere
10    332K   17M InboundTCP  tcp  --  any    any     anywhere             anywhere             tcp flags:FIN,SYN,RST,ACK/SYN
11       0     0 denylog    tcp  --  any    any     anywhere             anywhere             tcp flags:FIN,SYN,RST,ACK/SYN
12    114K   11M InboundUDP  udp  --  eth0   any     anywhere             anywhere
13       0     0 denylog    udp  --  eth0   any     anywhere             anywhere
14       0     0 ACCEPT     udp  --  dummy0 any     anywhere             anywhere             udp spts:bootps:bootpc
15   87915 5217K denylog    all  --  any    any     anywhere             anywhere

so that my blocklist is evaluated before XTGeoIP

[ReetP]

Damn I just saw:

Code: [Select]

let IPTABLES_IPSET_RULE_NUMBER=1 # if FORCE is yes, the number at which place insert the ipset-match rule (default to 1)
That'll do it

Ok thanks Stefano. Nice job.

[Stefano]

Damn I just saw:

Code: [Select]

let IPTABLES_IPSET_RULE_NUMBER=1 # if FORCE is yes, the number at which place insert the ipset-match rule (default to 1)
That'll do it

Ok thanks Stefano. Nice job.

you're more than welcome