Koozali.org: home of the SME Server

DOS-like behavior on the SMTP - or is it?

seefood

DOS-like behavior on the SMTP - or is it?
« on: June 01, 2005, 08:39:16 PM »
Hello people. I hope this is the right place. I searched the forums and mailing lists but found no record of this...

some 8 months ago I installed for a client 6.0.1 with a few extra bits from dungog, clamav and SA were a must, and so were LDAP, virtual domains and a common phonebook. I did quite a few improvisations to get them all working neatly and left it there. other than the LDAP schema which I never managed to fit their needs for some reason, the machine has been working without a hitch for several long months.

some 2-3 weeks ago things started to get stuck. It looked as if the full quota of 40 incoming SMTP sessions were open, with 40 instances of smtpfront the swap was thrashing and the machine reached load averages high in the double digits. I immediately canceled this perl script and went back to qmail-smtpd (like god^H^H^HDJB intended) and instaled simscan instead, like I do for all my other clients. oddly enough it didn't work, I discovered that SME's qmail was compiled without the qmailqueue patch, recompiled that, got the thing to work and 18 hours later it was stuck again!

only this time I had no crazy load averages, netstat showed me that all 40 sessions were just hung there on SYN_RECIEVED. I killed all the qmail-smtpd processes, but I dread the moment this will happen once again.

The problem is this: my first thought was this was a DOS attempt, but the IP addresses were from all over the place, almost a quarter of them from INSIDE the LAN (the machine is in the DMZ, with NAT), and the connections were of verious ages, which suggested they got stuck little by little over time. naturally this time I started suspecting the kernel, which is 2.4.20-30.7.legacy. I found -37.7 on fedoralegacy.org but a remote kernel upgrade failed so for now I'm sticking to this older one.

Has anyone bumped into these problems? Any suggestions? will an upgrade help and how painful will it be after all these customizations? :-(

TIA,
Ira.

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: DOS-like behavior on the SMTP - or is it?
« Reply #1 on: June 01, 2005, 10:17:44 PM »
Quote from: "seefood"

I immediately canceled this perl script and went back to qmail-smtpd (like god^H^H^HDJB intended) and instaled simscan instead, like I do for all my other clients. oddly enough it didn't work, I discovered that SME's qmail was compiled without the qmailqueue patch, recompiled that, got the thing to work and 18 hours later it was stuck again!


You could have safely used mailfront's smtpfront-qmail rather than qmail-smtpd. It's a drop in replacement, and has the qmailqueue patch functionality out of the box.

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: DOS-like behavior on the SMTP - or is it?
« Reply #2 on: June 01, 2005, 10:18:42 PM »
Quote from: "seefood"

I immediately canceled this perl script and went back to qmail-smtpd (like god^H^H^HDJB intended) and instaled simscan instead, like I do for all my other clients.


A HOWTO for simscan installation and configuration would be a great contrib!

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: DOS-like behavior on the SMTP - or is it?
« Reply #3 on: June 01, 2005, 10:21:26 PM »
Quote from: "seefood"

only this time I had no crazy load averages, netstat showed me that all 40 sessions were just hung there on SYN_RECIEVED. I killed all the qmail-smtpd processes, but I dread the moment this will happen once again.


This doesn't make sense to me. When a connection is in SYN_RECIEVED, only the kernel should know about it. The waiting tcpserver should be waiting in connect, and there should be no qmail-smtpd processes. The connect should not succeed until the TCP handshake is completed.

seefood

Re: DOS-like behavior on the SMTP - or is it?
« Reply #4 on: June 01, 2005, 10:35:26 PM »
Quote from: "CharlieBrady"

You could have safely used mailfront's smtpfront-qmail rather than qmail-smtpd. It's a drop in replacement, and has the qmailqueue patch functionality out of the box.


no I couldn't. as I mentioned earlier it had some bug where it got stuck in a loop and got the load average to 70-90. I could not risk keeping it there for another minute.

and another thing, why use it if I have that functionality in the much more commonly used and security-tested qmail-smtpd original?

in other news, I see that 6 procs are stuck on the server now. smtp is flowing fine, only there are 6 (of 40) processes of qmail-smtpd, each forking to simscan and in turn stuck on clamdscan, sleeping, listening on a socket. I took a bet and restarted clamd and lo+behold, they ere released... I have a new suspect to the list :-(

it's clamav-es-0.80-01dungog
I guess I had it coming... I'll go check for updates and see if things start flowing again.

Offline CharlieBrady

  • *
  • 6,918
  • +3/-0
Re: DOS-like behavior on the SMTP - or is it?
« Reply #5 on: June 02, 2005, 12:04:21 AM »
Quote from: "seefood"
Quote from: "CharlieBrady"

You could have safely used mailfront's smtpfront-qmail rather than qmail-smtpd. It's a drop in replacement, and has the qmailqueue patch functionality out of the box.


no I couldn't. as I mentioned earlier it had some bug where it got stuck in a loop and got the load average to 70-90.


I'd be very, very suprised if your problem was in mailfront, and not in your qmail-queue wrapper.

Quote

and another thing, why use it if I have that functionality in the much more commonly used and security-tested qmail-smtpd original?


Because it was already there and configured to be used.

Quote

in other news, I see that 6 procs are stuck on the server now. smtp is flowing fine, only there are 6 (of 40) processes of qmail-smtpd, each forking to simscan and in turn stuck on clamdscan, sleeping, listening on a socket. I took a bet and restarted clamd and lo+behold, they ere released...


No surprises there.

Offline raem

  • *
  • 3,972
  • +4/-0
Re: DOS-like behavior on the SMTP - or is it?
« Reply #6 on: June 02, 2005, 06:16:13 AM »
seefood

If you get your system back to "typical standard" some of the suggestions here may be of use.

http://mirror.contribs.org/smeserver/contribs/rmitchell/smeserver/howto/Mail%20system%20tweaks%20HOWTO%20for%20sme%20server.htm
...