Topic: are there mail settings to allow sending mail from local networks only

[wdepot]

Is there a way to set the server to only allow sending outgoing email from computers located on the local network?

We've suddenly started having troubles with sending emails from our computers as the connection to the mail server times out. As near as I can tell from the log files for qpsmtpd it looks like computers from IP addresses outside our system are using our server to send emails. To counter this after looking at the documentation for qpsmtpd I first checked the db configuration settings where I discovered that Authentication was set to disabled for some reason (a holdover from an earlier version of SME Server maybe since we've been using SME since about version 6). I immediately enabled that and did a signal-event email-update. I've also changed every user password on the server. However we are still having the problem. The Mail Log File Analysis of the Sender uids shows two that are sending many more messages than they should but I need to know how to determine the user name from the uid number so I will know which accounts are compromised.

Since the only computers allowed to send outgoing email from our system is the server itself and the computers that are on its local network I would like to be able to block the sending of outgoing email from any other IPs.

I would also like to reduce the number of tries in fail2ban that a computer has against qpsmtpd down from the default of 9 but I haven't been able to find where to set the configuration variables for the fail2ban jails.

One other thing to ask about, while looking at the log files in Server Manager I discovered that there are a bunch of files named under journal/2db9adc0dc814cc6bc8cb7b56e9ee03c/system.journal and journal/2db9adc0dc814cc6bc8cb7b56e9ee03c/user-1005.journal and they all just have gibberish in them, nothing human readable. Any ideas as to what could be creating these log files and are they something to be worried about?

[Jean-Philippe Pialasse]

first I assume you have no template-custom, If any move them out.

also have a look to server-manager Local Network for any non LAN Network wher you could have let stranger use your server as relay.

also check any website you host for malware.

look at your qmail log and qmail queue. if high I would recommend to sotp qmail and use qmailhandle to clean the queue and help identify the origine by reviewing the mails https://wiki.koozali.org/Qmhandle_mail_queue_manager

Is there a way to set the server to only allow sending outgoing email from computers located on the local network?

go to server manager => E-mail => change email reception settings and set SMTP authentication    Allow SSMTP (secure)  or better disabled if you do not want anyone to be able to authenticate to send email

if you want to keep Allow SSMTP (secure) but only on LAN then after that go to cli and do

Code: [Select]

config setprop sqpsmtpd access private
signal-event email-update

We've suddenly started having troubles with sending emails from our computers as the connection to the mail server times out. As near as I can tell from the log files for qpsmtpd it looks like computers from IP addresses outside our system are using our server to send emails. To counter this after looking at the documentation for qpsmtpd I first checked the db configuration settings where I discovered that Authentication was set to disabled for some reason (a holdover from an earlier version of SME Server maybe since we've been using SME since about version 6). I immediately enabled that and did a signal-event email-update.

well you just opened the closed door.
for qpsmtpd the property Authentication if disabled prevents any authentication and hence sending email as (unless proxy smtp is enabled and an external network is added as local network) nothing except scripts and webmail in the server can send mails outside.

that said I would check first any website hosted on the server (they might sendmail direcly and you should only see them on qmail log)

I've also changed every user password on the server. However we are still having the problem. The Mail Log File Analysis of the Sender uids shows two that are sending many more messages than they should but I need to know how to determine the user name from the uid number so I will know which accounts are compromised.

well, qpsmtpd/sqpsmtpd DO only use username and not uid.

Code: [Select]

235 LOGIN authentication successful for yourusername - auth_imap/login

Since the only computers allowed to send outgoing email from our system is the server itself and the computers that are on its local network I would like to be able to block the sending of outgoing email from any other IPs.

go to server-manager => proxy setting and set SMTP proxy status     to Blocked

I would also like to reduce the number of tries in fail2ban that a computer has against qpsmtpd down from the default of 9 but I haven't been able to find where to set the configuration variables for the fail2ban jails.

I would discourage this way , as it is hardcoded to use the generik fail2ban MaxRetry and multiply by 3.

I would rather suggest to use geoip blocking
https://wiki.koozali.org/Xt_geoip

One other thing to ask about, while looking at the log files in Server Manager I discovered that there are a bunch of files named under journal/2db9adc0dc814cc6bc8cb7b56e9ee03c/system.journal and journal/2db9adc0dc814cc6bc8cb7b56e9ee03c/user-1005.journal and they all just have gibberish in them, nothing human readable. Any ideas as to what could be creating these log files and are they something to be worried about?

welcome to systemd and journalctl
you can read them by just using journalctl in cli (See man journalctl)



to me you are more victim of a ddos (slow connection and lot of ip see in log) rather than massive email sending with a compromised account, but qmail queue woule help you see the difference.
if ddos type your best bet is xt_geoip


edit : change typo from config setprop qpsmtpd access private  to config setprop sqpsmtpd access private

[wdepot]

We have no custom templates related to email on the server.

I checked the Local Network setting and found one IP that didn't need to be there so I removed it. It was actually one of the static IP addresses assigned to us by Comcast and was the previous external IP address for our server before we changed it during a DDOS attack on our web site. For anyone else to have used it they would have had to be connected to our side of the Comcast cable modem that connects us to the internet so I doubt that was the problem but it didn't need to be there anyway.

Under the E-Mail settings for Server Manager I am seeing only settings for POP3, IMAP and Webmail and these are all set to only local networks. I probably should mention that we are using Thunderbird (latest version for Windows) for email and the settings for the SMTP server are set to SSL/TLS with normal passwords. We aren't having trouble reading incoming emails at all. I just started being unable to send emails about three days ago because "The message could not be sent because the connection to Outgoing server (SMTP)  timed out."

I did change the SMTP Proxy to Blocked like you suggested.

The qpsmtpd access private setting sounds like what I was looking for since I need to allow Thunderbird to have access on the server for both incoming and outgoing emails. I had noticed the setting and suspected it might be the key but I was unable to find anything in the SME documentation about it other than that the key existed and that the default value is public. Nothing said what the setting did or what the allowable values were.

I just checked the setting for qpsmtpd Authentication and noticed it is already set back to disabled which is apparently what it should be. The SME documentation says that the default value is enabled which is the main reason I had changed it. I thought enabled meant require authentication.

After further thought on the Mail Log File Analysis report it occurred to me that the two uids showing a high volumes of sent messages are probably the system itself since I get a lot of messages from fail2ban about blocked actions (usually qpsmtpd, Recidive and Apache) and the PHP sendmail since that is normally sending several messages daily and then close to 5000 each time we send our weekly email. I'm not sure how long of a time period the Mail Log File Analysis covers but if long enough it would certainly account for the large numbers of sent messages I saw.

I thought maybe the fail2ban attempts were controlled by db settings but since they are obviously not I won't worry about it. Getting banned for 30 minutes after 9 failed attempts will still make it hard for anyone trying to guess a password to accomplish much.

Out of curiosity I checked journalctl and noticed that it listed back to the beginning of December so I suspect logrotate isn't pruning it enough since there are well over 30 files each listed in Server Manager messages for both journal/system and journal/user. Good to know that they are a valid part of the system. I'm not sure why the Server Manager Log Files Messages bothers listing them since they require journalctl to read.

I would think a DDOS would cause the entire server to slow down and the only thing we are having a problem with is sending outgoing email from Thunderbird. I'm wondering if a recent update to Thunderbird might be causing the issue.

I may check into the xt_geoip if it seems to be needed. How do I check the qmail queue?

I'll also check for unwanted files in the ibays though both SSH and FTP are set to local network only and users of the web site aren't allowed to upload files. I've have noticed some things in the Logwatch emails that show possible exploits though most are ones I know would just be translated as unused $_GET variables by the website and be ignored. A list from yesterday is more puzzling:

Code: [Select]

    /index.action?method:%23_memberAccess%3d@ognl.OgnlContext@DEFAULT_MEMBER_ACCESS,%23res%3d%40org.apache.struts2.ServletActionContext%40getResponse(),%23res.setCharacterEncoding(%23parameters.encoding%5B0%5D),%23w%3d%23res.getWriter(),%23s%3dnew+java.util.Scanner(@java.lang.Runtime@getRuntime().exec(%23parameters.cmd%5B0%5D).getInputStream()).useDelimiter(%23parameters.pp%5B0%5D),%23str%3d%23s.hasNext()%3f%23s.next()%3a%23parameters.ppp%5B0%5D,%23w.print(%23str),%23w.close(),1?%23xx:%23request.toString&pp=%5C%5CA&ppp=%20&encoding=UTF-8&cmd=cat%20/etc/passwd HTTP Response 302
    /jnoj/web/polygon/problem/viewfile?id=1&name=../../../../../../../etc/passwd HTTP Response 302
    /..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\..\\windows\\win.ini HTTP Response 302
    /maint/modules/home/index.php?lang=english|cat%20/etc/passwd HTTP Response 302
I'm not sure why the server would return 302 for requests like these and not 404.

Edit : removed double quoted text without direct reference to it

[wdepot]

Just a quick update. After changing the qpsmtpd access setting to private as suggested we discovered that we were no longer receiving any incoming email. When I changed it back to public the incoming emails arrived again. So this setting obviously controls incoming emails (and possibly outgoing as well) which was not my intent.

[wdepot]

Second update, after figuring out how to check the qmail queue I found it to be empty so I think the problem must lie with Thunderbird and how it connects to the server for outgoing email. I don't know if it could suddenly be something to do with ssl though as I recall SME 10 received an update that added TLS 1.3 before it went end-of-life. I am going to try and see if Outlook will work. If not I guess I may get stuck using webmail to send and reply to messages which isn't ideal.

[Jean-Philippe Pialasse]

ok sorry for the typo in my first answer I hitted refresh while typying and had to start from scratch my answer, so this is what happen when you try to be fast on the second attempt.

Code: [Select]

config setprop sqpsmtpd access private  Authentication enabled
config setprop qpsmtpd access private Authentication disabled
signal-event email-updateis what you need

you need Authentication=enabled for sqpsmtpd  and Authentication=disabled for qpsmtpd
qpsmtpd , port 25 should listen to public

I checked the Local Network setting and found one IP that didn't need to be there so I removed it. It was actually one of the static IP addresses assigned to us by Comcast and was the previous external IP address for our server before we changed it during a DDOS attack on our web site. For anyone else to have used it they would have had to be connected to our side of the Comcast cable modem that connects us to the internet so I doubt that was the problem but it didn't need to be there anyway.

Under the E-Mail settings for Server Manager I am seeing only settings for POP3, IMAP and Webmail and these are all set to only local networks. I probably should mention that we are using Thunderbird (latest version for Windows) for email and the settings for the SMTP server are set to SSL/TLS with normal passwords. We aren't having trouble reading incoming emails at all. I just started being unable to send emails about three days ago because "The message could not be sent because the connection to Outgoing server (SMTP)  timed out."

that is because you need to let enable Authentication on SSL (sqpsmtpd, port 465)
qpsmtpd (port 25) should not allow authentication but should be access public, or you will stop receive external mail

I did change the SMTP Proxy to Blocked like you suggested.

good

I thought maybe the fail2ban attempts were controlled by db settings but since they are obviously not I won't worry about it. Getting banned for 30 minutes after 9 failed attempts will still make it hard for anyone trying to guess a password to accomplish much.

it has some setting but not that granular, however you then have the recidive jail and they are gone for a while.
fail2ban is really ressource craving, more than the mail server, so again you should first filter out via geoip part of the world you do not want to sneak in .

Out of curiosity I checked journalctl and noticed that it listed back to the beginning of December so I suspect logrotate isn't pruning it enough since there are well over 30 files each listed in Server Manager messages for both journal/system and journal/user. Good to know that they are a valid part of the system. I'm not sure why the Server Manager Log Files Messages bothers listing them since they require journalctl to read.

again man journalctl, and check the internet about journald, you will have to compose with it in max 2-3 versions of your server as rsyslog/logrotate will be gone  at one point.

I would think a DDOS would cause the entire server to slow down and the only thing we are having a problem with is sending outgoing email from Thunderbird. I'm wondering if a recent update to Thunderbird might be causing the issue.

depends on the intensity and how the services are implemented, our mail service are implemented with some softlimit so they won't slow down the whole server in case of attack.

I may check into the xt_geoip if it seems to be needed.

always a good choice not to leave access to people who should not have access.

How do I check the qmail queue?

check the server manager has a menu for that, or you can use qmhandle contrib, which is easier

I'll also check for unwanted files in the ibays though both SSH and FTP are set to local network only and users of the web site aren't allowed to upload files. I've have noticed some things in the Logwatch emails that show possible exploits though most are ones I know would just be translated as unused $_GET variables by the website and be ignored. A list from yesterday is more puzzling:

it is always worth investigating.

[mmccarn]

These resources might help with analyzing your email/smtp performance:

• Mail log file analysis
• Email Statistics
    > Contribs & Addons
    > Useful Commands
  

[wdepot]

ok sorry for the typo in my first answer I hitted refresh while typying and had to start from scratch my answer, so this is what happen when you try to be fast on the second attempt.

Code: [Select]

config setprop sqpsmtpd access private  Authentication enabled
config setprop qpsmtpd access private Authentication disabled
signal-event email-updateis what you need

you need Authentication=enabled for sqpsmtpd  and Authentication=disabled for qpsmtpd
qpsmtpd , port 25 should listen to public


that is because you need to let enable Authentication on SSL (sqpsmtpd, port 465)
qpsmtpd (port 25) should not allow authentication but should be access public, or you will stop receive external mail

fail2ban is really resource craving, more than the mail server, so again you should first filter out via geoip part of the world you do not want to sneak in .
again man journalctl, and check the internet about journald, you will have to compose with it in max 2-3 versions of your server as rsyslog/logrotate will be gone  at one point.

depends on the intensity and how the services are implemented, our mail service are implemented with some softlimit so they won't slow down the whole server in case of attack.
always a good choice not to leave access to people who should not have access.
check the server manager has a menu for that, or you can use qmhandle contrib, which is easier

it is always worth investigating.

It looks like it was the sqpsmtpd access setting that was causing the problem. For some reason it was set to public rather than private. Now that it is corrected email sending is working as expected.

Yesterday I looked into systemctl and listed the services and noticed both qpsmtpd and sqpsmtpd showed "loaded active exited". The exited part surprised me but since we were receiving emails without a problem I figured that the status was normal.

I then tried systemctl restart sqpsmtpd.service and discovered that immediately afterword I could send an email. After a couple of minutes it was back to time outs. At least it was much better and less disruptive than the shutdown -r now that I was originally doing to temporarily allow email sending. Thankfully correcting the access setting has proved to be a permanent fix so I can avoid the work arounds.

I looked at the documentation for xt_geoip and immediately realized that it wouldn't work for us since it only bans by country and we actually do some business internationally.

As for fail2ban we haven't noticed it causing any issues with slowing the server or using excessive resources so we'll keep using it to block to bad guys.

[Jean-Philippe Pialasse]

on sme 10 systemd only used to start qpsmtpd and then it is monitored by runit.
hence the  exit status.


geoip might be usefull. i guess you might have few countries where you do not have contact. just check from where you get the most connection and see if any legitimate could come from there. 

[ReetP]

F2B is OK, but as JP said, it is a real resource hog.

XT Tables/geoip is far more efficient and effective.

Here's the top countries/offenders I have blocked - there are more a lot more but smaller numbers.

Note I have some services like imaps, ssh, sqpsmtpd set as != ie I ONLY permit from a couple of countries.

Everything else gets blocked - so you can see the worst offenders.

I then add any of those bad boys to my qpsmtpd list unless we specifically get mail from them (yes I'd love to entirely block the USA.....) And I periodically have a peek and update as required.

Code: [Select]

  Numbers of IPs banned (xt_geoip) by country during LAST MONTH
       ( XX means 'country not found' )

--------------------
CN | 89913 | 21.8%
RU | 37209 | 9.0%
KR | 36845 | 8.9%
BG | 35777 | 8.7%
US | 23310 | 5.7%
HK | 23059 | 5.6%
IN | 22646 | 5.5%
SG | 16710 | 4.1%
JP | 16033 | 3.9%
BR | 12498 | 3.0%
TR | 7875 | 1.9%
UA | 5938 | 1.4%
PL | 5294 | 1.3%
TW | 5100 | 1.2%
VN | 4302 | 1.0%
CZ | 3929 | 1.0%
PH | 3799 | 0.9%
MD | 3627 | 0.9%
ID | 3546 | 0.9%
AU | 3625 | 0.9%
IR | 2765 | 0.7%
SE | 2435 | 0.6%
RO | 2476 | 0.6%
MY | 2537 | 0.6%
IL | 2515 | 0.6%
CA | 2472 | 0.6%
AE | 2569 | 0.6%
IT | 2023 | 0.5%
TH | 1619 | 0.4%
NL | 1791 | 0.4%
IQ | 1631 | 0.4%
DE | 1646 | 0.4%
PK | 1257 | 0.3%
MX | 1200 | 0.3%
MA | 1333 | 0.3%
ET | 1164 | 0.3%
BD | 1223 | 0.3%
AR | 1278 | 0.3%


Here's the bans - note that I only permit access to a couple of services from 3 countries - our UK office, me here in Spain, and the server location in France.

If I travel I will add any specific countries temporarily.

Code: [Select]


Global
Current list of banished country codes : == CN,IN,RU,BR,KR,RO,LT,AR,TW,VN,JP,HK,ID,PH,BD,CZ,BO,TH,MX,MD,IL,CO,BG,SG,RS,PL,IR,UA,EE,AE,BY,NG,TR,AW,AM

Per service
Name PORT STATUS Access Blacklist
ftp 21 disabled private != GB,ES,FR
httpd-e-smith 80 enabled public ==
imap 143 enabled private != GB,ES,FR
imaps 993 enabled public != GB,ES,FR
modSSL 443 enabled public ==
pop3 110 disabled private != GB,ES,FR
pop3s 995 disabled private != GB,ES,FR
qpsmtpd 25 enabled public == CN,IN,RU,BR,KR,RO,LT,AR,TW,VN,JP,HK,ID,PH,BD,CZ,BO,TH,MX,MD,IL,CO,BG,SG,RS,PL,IR,UA,EE,AE,BY,NG,TR,AW,AM
sqpsmtpd 465 enabled public != GB,ES,FR
sshd 2222 enabled public != GB,ES,FR

I have F2B ibstalled on that server but it is it disabled - geoip does all the heavy lifting these days.

HTH.

[wdepot]

F2B is OK, but as JP said, it is a real resource hog.

XT Tables/geoip is far more efficient and effective.

Here's the top countries/offenders I have blocked - there are more a lot more but smaller numbers.

Note I have some services like imaps, ssh, sqpsmtpd set as != ie I ONLY permit from a couple of countries.

Everything else gets blocked - so you can see the worst offenders.

I then add any of those bad boys to my qpsmtpd list unless we specifically get mail from them (yes I'd love to entirely block the USA.....) And I periodically have a peek and update as required.

Here's the bans - note that I only permit access to a couple of services from 3 countries - our UK office, me here in Spain, and the server location in France.

If I travel I will add any specific countries temporarily.

Code: [Select]


Global
Current list of banished country codes : == CN,IN,RU,BR,KR,RO,LT,AR,TW,VN,JP,HK,ID,PH,BD,CZ,BO,TH,MX,MD,IL,CO,BG,SG,RS,PL,IR,UA,EE,AE,BY,NG,TR,AW,AM

Per service
Name PORT STATUS Access Blacklist
ftp 21 disabled private != GB,ES,FR
httpd-e-smith 80 enabled public ==
imap 143 enabled private != GB,ES,FR
imaps 993 enabled public != GB,ES,FR
modSSL 443 enabled public ==
pop3 110 disabled private != GB,ES,FR
pop3s 995 disabled private != GB,ES,FR
qpsmtpd 25 enabled public == CN,IN,RU,BR,KR,RO,LT,AR,TW,VN,JP,HK,ID,PH,BD,CZ,BO,TH,MX,MD,IL,CO,BG,SG,RS,PL,IR,UA,EE,AE,BY,NG,TR,AW,AM
sqpsmtpd 465 enabled public != GB,ES,FR
sshd 2222 enabled public != GB,ES,FR

I have F2B ibstalled on that server but it is it disabled - geoip does all the heavy lifting these days.

HTH.

It looks like xt_geoip may be more useful than I first thought if it can ban countries for specific services. I already have the major ones (imap, pop, ssh, ftp, sqpsmtpd) set to local network only but it wouldn't hurt to add a second layer of protection and ban them from anywhere that is not in the US.

Spamassassin works well enough on the incoming emails that I've only ever had to white list one domain. Otherwise a quick daily glance through the Junk email folder at the subjects is sufficient to tell that the ones sent to Junk are indeed junk so I don't think I'd worry about using xt_geoip against qpsmtpd.

[Stefano]

I have F2B, xt_geo and also this on my personal server

https://forums.koozali.org/index.php/topic,55193.msg291088.html#msg291088

HTH