Topic: [7.1.3] Broken qmail

[dtech]

Hi All;

I seem to have broken my qmail installation. A user's machine was compromised by one of the email ecard trojans going around, and started sending out large quantities of spam. I have SME set up to use the SMTP proxy (set in Proxy settings) and I use the ISP's smtp server (set up in E-mail).

The offending machine is off the network now, but in order to clean up the mess I had to remove thousands of emails from the qmail queue. Browsing the forums I found that I could stop qmail services and then delete the emails, then restart qmail. By the way, this is the first time I deal with a qmail problem. So, I did the following:

Code: [Select]

service qmail stop

This command failed after a few minutes. So I moved on to

Code: [Select]

service qpsmtpd stop
service sqpsmtpd stop

These two commands worked. Then I deleted all of the contents (including the numbered directories, 0, 1, 2 through 22) of /var/qmail/queue/info, mess, and remote, and then the contents of intd.

Then I did a

Code: [Select]

service qmail start
service qpsmtpd start
service sqpsmtpd start

Then I went back and added the directories I had deleted, and changed the ownership and permissions to match a clean SME install that I use for reference.

Unfortunately, what I now have is this; email sent by clients arrives at the SME server, lands in the queue, and just stays there. No mail is sent, either local or outside. Here is the output from qmail-qstat:

Code: [Select]

# ./qmail-qstat
messages in queue: 53
messages in queue but not yet preprocessed: 53

Here is a snip from the current qpsmtpd log:

Code: [Select]

2007-07-26 22:51:18.495883500 5502 Accepted connection 0/40 from 192.168.0.111 / pc-00111.example.org
2007-07-26 22:51:18.496900500 5502 Connection from pc-00111.example.org [192.168.0.111]
2007-07-26 22:51:18.498662500 5502 running plugin (set_hooks): peers
2007-07-26 22:51:18.500331500 5502 trying to get config for peers/192.168.0
2007-07-26 22:51:18.512669500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.513632500 5502 trying to get config for peers/192.168.0
2007-07-26 22:51:18.514191500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.516214500 5502 peers hooking valid_auth
2007-07-26 22:51:18.516924500 5502 peers hooking set_hooks
2007-07-26 22:51:18.517755500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.518900500 5502 logging::logterse hooking queue
2007-07-26 22:51:18.519572500 5502 logging::logterse hooking deny
2007-07-26 22:51:18.520213500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.521128500 5502 check_relay hooking connect
2007-07-26 22:51:18.522118500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.539585500 5502 check_norelay hooking connect
2007-07-26 22:51:18.539590500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.539593500 5502 check_basicheaders hooking data_post
2007-07-26 22:51:18.539596500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.539598500 5502 check_badmailfrom hooking rcpt
2007-07-26 22:51:18.539601500 5502 check_badmailfrom hooking mail
2007-07-26 22:51:18.539603500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.539606500 5502 check_badrcptto_patterns hooking rcpt
2007-07-26 22:51:18.539625500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.539628500 5502 check_badrcptto hooking rcpt
2007-07-26 22:51:18.539631500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.539633500 5502 check_spamhelo hooking ehlo
2007-07-26 22:51:18.539636500 5502 check_spamhelo hooking helo
2007-07-26 22:51:18.539638500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.539641500 5502 check_goodrcptto hooking rcpt
2007-07-26 22:51:18.539644500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.539652500 5502 rcpt_ok hooking rcpt
2007-07-26 22:51:18.539655500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.539657500 5502 tnef2mime hooking data_post
2007-07-26 22:51:18.539660500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.539905500 5502 virus::clamav hooking data_post
2007-07-26 22:51:18.540995500 5502 trying to get config for plugin_dirs
2007-07-26 22:51:18.542171500 5502 queue::qmail_2dqueue hooking queue
2007-07-26 22:51:18.543712500 5502 Plugin peers, hook set_hooks returned DECLINED,
2007-07-26 22:51:18.544534500 5502 running plugin (connect): check_relay
2007-07-26 22:51:18.545103500 5502 trying to get config for relayclients
2007-07-26 22:51:18.557684500 5502 trying to get config for morerelayclients
2007-07-26 22:51:18.558834500 5502 Plugin check_relay, hook connect returned DECLINED,
2007-07-26 22:51:18.559744500 5502 running plugin (connect): check_norelay
2007-07-26 22:51:18.560381500 5502 trying to get config for norelayclients
2007-07-26 22:51:18.579384500 5502 Plugin check_norelay, hook connect returned DECLINED,
2007-07-26 22:51:18.580112500 5502 trying to get config for smtpgreeting
2007-07-26 22:51:18.598839500 5502 220 smeserver.example.org ESMTP
2007-07-26 22:51:18.600109500 5502 trying to get config for timeoutsmtpd
2007-07-26 22:51:18.600792500 5502 trying to get config for timeout
2007-07-26 22:51:18.601521500 5502 dispatching EHLO [192.168.0.111]
2007-07-26 22:51:18.602697500 5502 running plugin (ehlo): check_spamhelo
2007-07-26 22:51:18.603560500 5502 trying to get config for badhelo
2007-07-26 22:51:18.604506500 5502 Plugin check_spamhelo, hook ehlo returned DECLINED,
2007-07-26 22:51:18.605406500 5502 trying to get config for tls_before_auth
2007-07-26 22:51:18.605942500 5502 trying to get config for me
2007-07-26 22:51:18.606598500 5502 trying to get config for databytes
2007-07-26 22:51:18.607231500 5502 trying to get config for databytes
2007-07-26 22:51:18.608134500 5502 250-example.org Hi pc-00111.example.org [192.168.0.111]
2007-07-26 22:51:18.608651500 5502 250-PIPELINING
2007-07-26 22:51:18.609024500 5502 250-8BITMIME
2007-07-26 22:51:18.609397500 5502 250 SIZE 15000000
2007-07-26 22:51:18.610451500 5502 dispatching MAIL FROM:<person@example.org> SIZE=357
2007-07-26 22:51:18.611305500 5502 full from_parameter: FROM:<person@example.org> SIZE=357
2007-07-26 22:51:18.612651500 5502 from email address : [<person@example.org>]
2007-07-26 22:51:18.614142500 5502 running plugin (mail): check_badmailfrom
2007-07-26 22:51:18.614815500 5502 trying to get config for badmailfrom
2007-07-26 22:51:18.639302500 5502 Plugin check_badmailfrom, hook mail returned DECLINED,
2007-07-26 22:51:18.640454500 5502 getting mail from <person@example.org>
2007-07-26 22:51:18.641027500 5502 250 <person@example.org>, sender OK - how exciting to get mail from you!
2007-07-26 22:51:18.642066500 5502 dispatching RCPT TO:<anotherperson@gmail.com>
2007-07-26 22:51:18.643117500 5502 to email address : [<anotherperson@gmail.com>]
2007-07-26 22:51:18.644188500 5502 running plugin (rcpt): check_badmailfrom
2007-07-26 22:51:18.644839500 5502 Plugin check_badmailfrom, hook rcpt returned DECLINED,
2007-07-26 22:51:18.645364500 5502 running plugin (rcpt): check_badrcptto_patterns
2007-07-26 22:51:18.645935500 5502 Plugin check_badrcptto_patterns, hook rcpt returned DECLINED,
2007-07-26 22:51:18.646366500 5502 running plugin (rcpt): check_badrcptto
2007-07-26 22:51:18.646973500 5502 Plugin check_badrcptto, hook rcpt returned DECLINED,
2007-07-26 22:51:18.647418500 5502 running plugin (rcpt): check_goodrcptto
2007-07-26 22:51:18.648289500 5502 Plugin check_goodrcptto, hook rcpt returned DECLINED,
2007-07-26 22:51:18.648951500 5502 running plugin (rcpt): rcpt_ok
2007-07-26 22:51:18.649608500 5502 trying to get config for me
2007-07-26 22:51:18.650060500 5502 trying to get config for rcpthosts
2007-07-26 22:51:18.650962500 5502 trying to get config for morercpthosts
2007-07-26 22:51:18.651740500 5502 Plugin rcpt_ok, hook rcpt returned OK,
2007-07-26 22:51:18.652906500 5502 250 <anotherperson@gmail.com>, recipient ok
2007-07-26 22:51:18.654748500 5502 dispatching DATA
2007-07-26 22:51:18.655949500 5502 354 go ahead
2007-07-26 22:51:18.656938500 5502 trying to get config for databytes
2007-07-26 22:51:18.657651500 5502 max_size: 15000000 / size: 0
2007-07-26 22:51:18.658534500 5502 trying to get config for timeout
2007-07-26 22:51:18.669596500 5502 spooling message to disk
2007-07-26 22:51:18.716976500 5502 max_size: 15000000 / size: 346
2007-07-26 22:51:18.718143500 5502 trying to get config for me
2007-07-26 22:51:18.719479500 5502 running plugin (data_post): check_basicheaders
2007-07-26 22:51:18.720525500 5502 Plugin check_basicheaders, hook data_post returned DECLINED,
2007-07-26 22:51:18.721238500 5502 running plugin (data_post): tnef2mime
2007-07-26 22:51:18.772092500 5502 Plugin tnef2mime, hook data_post returned DECLINED,
2007-07-26 22:51:18.772587500 5502 running plugin (data_post): virus::clamav
2007-07-26 22:51:18.773334500 5502 virus::clamav plugin: Changing permissions on file to permit scanner access
2007-07-26 22:51:18.773857500 5502 virus::clamav plugin: Running: /usr/bin/clamdscan --stdout  --config-file=/etc/clamd.conf --disable-summary /var/spool/qpsmtpd/1185504678:5502:0 2>&1
2007-07-26 22:51:18.811408500 5502 virus::clamav plugin: clamscan results: /var/spool/qpsmtpd/1185504678:5502:0: OK
2007-07-26 22:51:18.812065500 5502 trying to get config for me
2007-07-26 22:51:18.813229500 5502 Plugin virus::clamav, hook data_post returned DECLINED,
2007-07-26 22:51:18.814092500 5502 running plugin (queue): logging::logterse
2007-07-26 22:51:18.815705500 5502 logging::logterse plugin: ` 192.168.0.111 pc-00111.example.org [192.168.0.111] <person@example.org> <anotherperson@gmail.com> queued <46A95E14.60206@example.org>
2007-07-26 22:51:18.816319500 5502 Plugin logging::logterse, hook queue returned DECLINED,
2007-07-26 22:51:18.816727500 5502 running plugin (queue): queue::qmail_2dqueue
2007-07-26 22:51:18.821682500 5507 queue::qmail_2dqueue plugin: (for 5502 ) Queuing qp 5507 to /var/qmail/bin/qmail-queue
2007-07-26 22:51:18.840788500 5502 Plugin queue::qmail_2dqueue, hook queue returned OK, Queued! 1185504678 qp 5507 <46A95E14.60206@example.org>
2007-07-26 22:51:18.841491500 5502 250 Queued! 1185504678 qp 5507 <46A95E14.60206@example.org>
2007-07-26 22:51:18.845161500 5502 dispatching QUIT
2007-07-26 22:51:18.845916500 5502 trying to get config for me
2007-07-26 22:51:18.846398500 5502 221 example.org closing connection. Have a wonderful day.
2007-07-26 22:51:18.846972500 5502 click, disconnecting
2007-07-26 22:51:19.422782500 4095 cleaning up after 5502

Any help with this would be appreciated.

-dtech

[mmccarn]

I've had 'service qmail stop' take up to 45 minutes to complete -- but it *always* does complete, eventually...

So, (if you're lucky) you may just need to do 'service qmail start' (again)

Otherwise, (since you've deleted all the queue folders), you may want to do

 signal-event post-upgrade; signal-event reboot

(Perhaps that will re-initialize the qmail queues...)

[CharlieBrady]

(Perhaps that will re-initialize the qmail queues...)

No it won't. OP will need to --force reinstall qmail RPM.

I seem to have broken my qmail installation.

Well, that's what happens when you go carelessly deleting crucial directories without knowing what you are doing. Pleae don't do it again.

[mmccarn]

OP will need to --force reinstall qmail RPM

Is that going to look like

Code: [Select]

yum remove qmail
yum install qmail
signal-event post-upgrade; signal-event reboot
or is it going to look like

Code: [Select]

rpm -Uvh --force /var/cache/yum/smeupdates/packages/e-smith-qmail-1.10.0-11.el4.sme.noarch.rpm(Assuming he's lucky enough to have a cached copy of e-smith-qmail available)

Or something else entirely?

[CharlieBrady]

or is it going to look like

Code: [Select]

rpm -Uvh --force /var/cache/yum/smeupdates/packages/e-smith-qmail-1.10.0-11.el4.sme.noarch.rpm(Assuming he's lucky enough to have a cached copy of e-smith-qmail available)

That. Never use "yum remove ..." because it can do some very surprising things. Use "rpm --e xxx" instead if you want to remove something.

[dtech]

I downloaded the qmail rpm from the SME 7 ibiblio repository here

http://mirror.contribs.org/smeserver/releases/7/smeos/i386/SME/RPMS/

and then did

Code: [Select]

rpm -ivh --force qmail-1.03-13.el4.sme.i386.rpm

and then did a reconfigure/reboot. And now things work like they used to. I will point out that upon reboot all the emails that were in the queue (in my case about 50) were all delivered. Keep that in mind if you have a queue full of spam... So thank you to Charlie for the helpful part of your post.

Regarding the last part of your post Charlie, let me just point out that your immature and condescending tone in numerous replies that I've read over the past couple of years only serves to damage the community. Berating or insulting users with questions discourages learning, participation, and cooperation. And in some cases it discourages financial investment.

I'll take your recommendation to not go "carelessly deleting crucial directories without knowing what you are doing". My recommendation to you would be to consider the real effect of your comments on the community, and try to eliminate your apparent need to demonstrate your superiority by stepping on others.

-dtech

[idp_qbn]

Dtech,
Be thankful you got sound advice and good help.

You had a problem and, by the sounds of your own description, you did something that exacerbated it.(We all do that - sometimes several times!)

Charlie's comment looks very fair from where I stand.

A wise monkey told me years ago - "Never write emails when you are angry."

I am glad you got your problem sorted. It is good to know how to do it for the future.

Cheers

Ian

[frond]

dtech

Gee wiz, that's a good case of biting the hand that feeds you !
You received very helpful advice from the worlds leading expert on sme server and you turn around & criticise Charlie in return.
I think you need to learn to be somewhat more grateful than that.

...your immature and condescending tone in numerous replies...
Berating or insulting users....

That's your interpretation & certainly not factual. I have identified none of that in posts by Charlie, and certainly not in his reply to you, if you read everything he has posted in these forums you will find his answers factual & accurate, & only occasionally does he display his annoyance with other peoples poor attitudes. Everyone here is lucky he answers questions for them, I'm sure he is very busy programming the free server software you are using.

Comments like yours dtech, would only discourage Charlie to answer peoples questions, so I suggest you apologise to him immediately, if your ego is not too big to prevent you from doing so.

[dehacked]

I recently had a similar problem. What I found was that in all instances the from address did not match the sender address. I there a way to reject messages if they do not match? I'm assuming it's done with qpsmtpd but my knowledge of linux is very limited atm.

[cactus]

I recently had a similar problem. What I found was that in all instances the from address did not match the sender address. I there a way to reject messages if they do not match? I'm assuming it's done with qpsmtpd but my knowledge of linux is very limited atm.

IIRC correclty you do not want that as it is perfectly valid to have a send from address that differs from the reply to address. If you would not allow that you would also block possibly legit email.

[dehacked]

IIRC correclty you do not want that as it is perfectly valid to have a send from address that differs from the reply to address. If you would not allow that you would also block possibly legit email.

Thanks for the fast reply Cactus. I hear what you are saying but let me elaborate... A virus compromises a trusted machine. It starts spamming, logging into the SME server with valid credentials of the compromised PC but the FROM address in the message is not the local domain. What I want to do is have SME do spam filtering on the local domain by making sure the FROM address is in the local domain.

[janet]

dehacked

The approach you are asking for would also stop legitimate mail, where senders have different return addresses for a variety of reasons.

Rather than use that approach, which is really an "after the event" fix, you would be better to stop the virus from being able to use the sme server.
See these
http://wiki.contribs.org/SME_Server:Documentation:FAQ#How_do_I_enable_smtp_authentication_for_users_on_the_internal_network
and
http://wiki.contribs.org/SME_Server:Documentation:FAQ#How_do_I_disable_SMTP_relay_for_unauthenticated_LAN_clients

[dehacked]

In our environment we will never have different from addresses. The problem is, 1, You don't just go and by a new AV licence for 150 pc's cause 1 virus defeats it(temporarily) and 2. the pc's are trusted machines, logging in with valid credentials. What I found so far is something called check_badaddressfrom for qpsmtpd. I think something exactly the opposite, where you have a friendly list will help in cases like mine. Seems there was a plugin called check_goodmailfrom but I haven't been able to get it anywhere. Has someone here maybe got it?

[janet]

dehacked

1, You don't just go and by a new AV licence for 150 pc's cause 1 virus defeats it(temporarily)

I did not suggest that, what I suggested was to enable and force smtp (mail server) authentication from your workstations. That way if a virus engine tries to access the sme smtp server, it will not be able to, as the virus does not know the authentication username and password.

This approach will require you to reconfigure all attached workstations email clients, but is a "one off " job, and you then have a more robust email system for good.

and 2. the pc's are trusted machines, logging in with valid credentials.

The issue is not whether the workstations are trusted or not, the issue is that a rogue virus can create an email engine and is then capable of accessing your sme servers smtp server directly (without using your email client). By enabling and forcing authentication to the smtp server, you defeat the virus email engine.

This problem (rogue virus email engines) is the very reason that smtp authentication was implemented on sme, and has been proven to work.

Personally, this fix should be the default setting on sme server.

A virus compromises a trusted machine. It starts spamming, logging into the SME server with valid credentials of the compromised PC but the FROM address in the message is not the local domain.

This is a incorrect conclusion I believe.
The virus email engine does not know the valid credentials (of the logged in user).
That is why smtp server authentication will block the virus email engine from sending messages.

[dehacked]

Alright, I've got people on the changes now. I hope that you are right else this is going to be a long weekend for me. The reason I did not suspect a virus mail engine is because the machine I was looking at myself showed no scan boxes for the AV scanning outgoing mail until Outlook was opened. Once that happens the entire screen is filled up with scan boxes. Changes should be done by the end of the day. We've also started installing an AV which is not defeated by the virus. I'll give a battle report within a day or so.