Koozali.org: home of the SME Server

Obsolete Releases => SME Server 9.x => Topic started by: mmccarn on September 17, 2017, 01:24:03 AM

Title: Firehol 'update-ipsets' install notes
Post by: mmccarn on September 17, 2017, 01:24:03 AM

I ran across the firehol (https://firehol.org/) project which includes a tool for automatically updating ipset lists from various online services that provide lists of bad, compromised, or otherwise malicious IP addresses.

These notes are incomplete, but
- I still have internet access
- email still passes in and out
- I am seeing traffic that is getting blocked by the ipset lists
- I can verify that the block lists are being periodically updated

The install involves installing the entire 'firehol' firewall product, but I have been careful not to use it and it doesn't seem to have broken anything (yet).

Install Firehol update-ipsets (https://github.com/firehol/blocklist-ipsets/wiki/Installing-update-ipsets) and enable the default recommended block lists

1. Install dependencies
On my system, I needed to install 'autoconf' and 'automake' (I already had the other dependencies installed)

Code: [Select]

yum install autoconf automake

2. Download and install Firehol from git

Code: [Select]

mkdir -p /root/firehol
cd /root/firehol
git clone https://github.com/firehol/iprange.git iprange.git
git clone https://github.com/firehol/firehol.git firehol.git

cd iprange.git
./autogen.sh
./configure --prefix=/usr CFLAGS="-march=native -O3" --disable-man
make
make install

cd ../firehol.git
./autogen.sh
./configure --prefix=/usr --sysconfdir=/etc --disable-man --disable-doc

# create a folder that 'update-ipsets' uses for various things...
mkdir -p /usr/var/run

3. Create ipset lists and enable the block lists recommended by Firehol

Code: [Select]

# create and enable default ipsets
for x in fullbogons dshield spamhaus_drop spamhaus_edrop; do ipset create $x hash:net; done
for x in fullbogons spamhaus_drop spamhaus_edrop; do update-ipsets enable $x; done
#
for x in feodo palevo sslbl zeus openbl blocklist_de; do ipset create $x hash:ip; done
for x in feodo palevo sslbl zeus openbl blocklist_de; do update-ipsets enable $x; done
#
update-ipsets

# confirm update
ipset list

# (you should see lots of IP addresses and networks fly by...)


4. Create a custom template fragment that references the enabled ipset lists

Code: [Select]

mkdir -p /etc/e-smith/templates-custom/etc/rc.d/init.d/masq
cd  /etc/e-smith/templates-custom/etc/rc.d/init.d/masq
vi 41CreateIpsetDenyLog
Paste this content into the new template fragment

Code: [Select]

{
# deny/log fragment to match ipsets maintained by update-ipsets
#
# RETURN for traffic from the local network or from 127.0.0.0/12
# (otherwise this traffic gets blocked by the 'fullbogons' list...)
#
# add ULOG and DROP lines for each enabled ipset
#
    $OUT .="    /sbin/iptables --new-chain ipset-denylog\n";
    my @mylocals = @locals;
    my $local= shift @mylocals;
    $OUT .= "    /sbin/iptables --append ipset-denylog -s $local --jump RETURN\n";
    foreach my $network (@mylocals)
    {
        $OUT .= "    /sbin/iptables --append ipset-denylog -s $local --jump RETURN\n";
    }
}
    /sbin/iptables --append ipset-denylog -s 127.0.0.0/12 --jump RETURN
    /sbin/iptables --append ipset-denylog -m set --set blocklist_de src   --jump ULOG --ulog-nlgroup 1 --ulog-prefix "ipset-blocklist_de"
    /sbin/iptables --append ipset-denylog -m set --set blocklist_de src   --jump DROP
    /sbin/iptables --append ipset-denylog -m set --set dshield src        --jump ULOG --ulog-nlgroup 1 --ulog-prefix "ipset-dshield"
    /sbin/iptables --append ipset-denylog -m set --set dshield src        --jump DROP
    /sbin/iptables --append ipset-denylog -m set --set feodo src          --jump ULOG --ulog-nlgroup 1 --ulog-prefix "ipset-feodo"
    /sbin/iptables --append ipset-denylog -m set --set feodo src          --jump DROP
    /sbin/iptables --append ipset-denylog -m set --set fullbogons src     --jump ULOG --ulog-nlgroup 1 --ulog-prefix "ipset-fullbogons"
    /sbin/iptables --append ipset-denylog -m set --set fullbogons src     --jump DROP
    /sbin/iptables --append ipset-denylog -m set --set openbl src         --jump ULOG --ulog-nlgroup 1 --ulog-prefix "ipset-openbl"
    /sbin/iptables --append ipset-denylog -m set --set openbl src         --jump DROP
    /sbin/iptables --append ipset-denylog -m set --set palevo src         --jump ULOG --ulog-nlgroup 1 --ulog-prefix "ipset-palevo"
    /sbin/iptables --append ipset-denylog -m set --set palevo src         --jump DROP
    /sbin/iptables --append ipset-denylog -m set --set spamhaus_drop src  --jump ULOG --ulog-nlgroup 1 --ulog-prefix "ipset-spamhaus_drop"
    /sbin/iptables --append ipset-denylog -m set --set spamhaus_drop src  --jump DROP
    /sbin/iptables --append ipset-denylog -m set --set spamhaus_edrop src --jump ULOG --ulog-nlgroup 1 --ulog-prefix "ipset-spamhaus_edrop"
    /sbin/iptables --append ipset-denylog -m set --set spamhaus_edrop src --jump DROP
    /sbin/iptables --append ipset-denylog -m set --set sslbl src          --jump ULOG --ulog-nlgroup 1 --ulog-prefix "ipset-sslbl"
    /sbin/iptables --append ipset-denylog -m set --set sslbl src          --jump DROP
    /sbin/iptables --append ipset-denylog -m set --set zeus src           --jump ULOG --ulog-nlgroup 1 --ulog-prefix "ipset-zeus"
    /sbin/iptables --append ipset-denylog -m set --set zeus src           --jump DROP
    /sbin/iptables --append ipset-denylog -j RETURN
    /sbin/iptables --insert INPUT 1 -j ipset-denylog

5. Activate the changes.
masq must be restarted to activate these changes

Code: [Select]

expand-template /etc/rc.d/init.d/masq
/etc/rc.d/init.d/masq restart

6. Monitor activity
Blocked traffic will appear in /var/log/iptables/current with a prefix of "ipset-xxx", where "xxx" is the list that caused the traffic to be blocked.

7. Create a cron job to update the blocklists
Firehol recommends updating at an interval of between 5 and 15 minutes, avoiding 5, 10 and 15.  I chose 11 minutes

Code: [Select]

crontab -e
then add this line:

Code: [Select]

*/11 * * * * /usr/sbin/update-ipsets >/dev/null 2>&1


8. Verify that blocklists are being updated
The blocklists are downloaded to, and processed in, /etc/firehol/ipsets

If crontab is working correctly, the files in that folder should have very recent times.


The firewall changes can be reverted using:

Code: [Select]

'rm' /etc/e-smith/templates-custom/etc/rc.d/init.d/masq/41CreateIpsetDenyLog
expand-template /etc/rc.d/init.d/masq
/etc/rc.d/init.d/masq restart

A final note:
It's not 100% clear to me if this process updates the ipset lists on my server, of the part of 'update-ipsets' that does that is part of the 'firehol' product that I am not using.

Title: Re: Firehol 'update-ipsets' install notes
Post by: guest22 on September 17, 2017, 08:46:05 AM

Nice work, thanks! I'll wikinize it soonish.

Title: Re: Firehol 'update-ipsets' install notes
Post by: guest22 on September 17, 2017, 08:47:51 AM

FYI, in the addition of the above and on the topic of 'firewall', there is also http://www.voipbl.org/ (http://www.voipbl.org/) specifically to block VoIP offenders/attackers. I have it installed on SME 9.2 with FreePBX and Fail2ban. It works. I'll wikinize that too soonish.

Title: Re: Firehol 'update-ipsets' install notes
Post by: Stefano on September 17, 2017, 12:49:15 PM

My 2c:
Take a look at ipset utility (onboard on a vanilla SME)
Ti can be easily used without compiling anythin with BL and geoip
https://github.com/danradom/geoipblock
https://www.nova-labs.net/strong-firewall-configuration-for-linux-with-blacklist-blocklist-auto-update/

Title: Re: Firehol 'update-ipsets' install notes
Post by: mmccarn on September 17, 2017, 02:52:22 PM

Quote from: Stefano on September 17, 2017, 12:49:15 PM

My 2c:
Take a look at ipset utility (onboard on a vanilla SME)
Ti can be easily used without compiling anythin with BL and geoip
https://github.com/danradom/geoipblock
https://www.nova-labs.net/strong-firewall-configuration-for-linux-with-blacklist-blocklist-auto-update/

Yes, all the firewalling is being done using ipset.  My notes were aimed at creating a schedule to download available IP block-lists that ipset could then use.  I did not find another pre-rolled script online other than the 'update-ipsets' from fireHOL for downloading IP blocklists and rotating the ipsets.

I'll definitely check out the links provided.  I agree that anything requiring compiling on the server is less-than-optimal...

Quote from: guest22 on September 17, 2017, 08:46:05 AM

Nice work, thanks! I'll wikinize it soonish.

I wouldn't do that yet; I'm a noob with iptables, and I suspect that there are problems with my template fragment.

I've already changed the ipset traffic match lines from "src" to "src,dst" so that traffic from my network to command-and-control servers will also be blocked, and I think there are problems with local network exceptions at the top.

Title: Re: Firehol 'update-ipsets' install notes
Post by: mmccarn on September 18, 2017, 12:32:04 PM

Note:

I have now disabled fullbogons (https://www.team-cymru.org/bogon-reference.html) in 41CreateIpsetDenyLog to avoid having local network traffic blocked.

Also, I can't find any way to un-enable a list in update-ipsets after it has been enabled - so any list I have tested now gets checked for updates every 11 minutes...