Topic: E-mail return to sender option is misnamed (bug 2291)

[mej]

Scenario: New install of SME 7 on HP/Compaq server (was previously SME 6.00). Patched up to date. Data migrated from old system, all contribs removed in process.

This company deals with China and the far east extensively, has been doing so for 15 years, and so has (predictably) a large body of spam arriving, and a huge quantity of forged undeliverable plus mail to random users (a few thousand a day when at its worst).

Under 6.00 I used the contrib from Dungog to block mail to unknown recipients. Together with double bounce deletion that cleared things up - or kept it manageable.

SME 7 is supposed to block email to unknown recipients by default. However this does not seem to be working - see test script following (sanitised) - this is repeatable:

Code: [Select]

01/11/07 10:26:45 SMTP Verify tripe@client.co.uk, at mail.client.co.uk
Contacting 217.154.x.x
220 targetserver1.client.co.uk ESMTP

HELO example.com
250 client.co.uk Hi mail.testingbox.co.uk [213.254.x.x]; I am so happy to meet you.

VRFY tripe@client.co.uk
252 Just try sending a mail and we'll see how it turns out ...

EXPN tripe@client.co.uk
500 Unrecognized command

Doesn't want to talk to us
RSET
250 OK

MAIL FROM:<spade@example.com>
250 <spade@example.com>, sender OK - how exciting to get mail from you!

RCPT TO:<tripe@client.co.uk>
250 <tripe@client.co.uk>, recipient ok

RCPT TO:<bogus11205@client.co.uk>
250 <bogus11205@client.co.uk>, recipient ok

RSET
250 OK

QUIT
221 client.co.uk closing connection. Have a wonderful day.

This is troubling and the server responses do not look at all like the examples shown here by others.

First, where did all the wibble ('Nice to meet you' etc) come from? Something has turned it on - can I turn it off again?

But this is not important - what is important is the acceptance of email to tripe@ and bogus-n-@ (which I assure you do not exist).

So - have I missed a switch somewhere in the interface? (duh moment?). Can't see one.

And why the wibble? Have I done something else stupid? Any suggestions welcome...

JR

[gordonr]

SME 7 is supposed to block email to unknown recipients by default.

It does.

And why the wibble? Have I done something else stupid? Any suggestions welcome...

All issues should be reported in the bug tracker, and only there. Thanks.

And when you do, please do not obfuscate logs or domain names. Doing so makes it nearly impossible for us to actually diagnose the issue. We need to be able to do things like look at the DNS entries and we can't do that if we don't know which domain is being talked about.

Please note that Bugzilla now allows you to specify that a bug should be limited to the 'security team' (it's a checkbox when you report the bug). That can be useful in these cases. However, we remove that restriction once patches are made available (if required).

If your request is more sensitive, please contact security at lists dot contribs dot org as this is only read by the security team.

[dmay]

Make sure you look at the logs in /var/log/qpsmtpd.

Darrell

[mej]

mej wrote:

SME 7 is supposed to block email to unknown recipients by default.

gordonr:
It does.

mej wrote:

And why the wibble? Have I done something else stupid? Any suggestions welcome...

gordonr:
All issues should be reported in the bug tracker, and only there. Thanks.

And when you do, please do not obfuscate logs or domain names. Doing so makes it nearly impossible for us to actually diagnose the issue.

GordonR, thanks very much for all the work you put in on the SME Server.

That said, no thanks at all for that extremely unhelpful response.

I have not posted it as a bug because I am asking *IS* this a bug? Or is it me? Until I know which - I don't know if it is a (putative) bug. So I am posting this transcript to find out if it is a recognised consequence of mis-configuration on my part. Perhaps I did not make that sufficiently clear - I hope I have now.

You have not addressed any of my questions. That log is a genuine log taken straight from the SMTP conversation on the machine as stated. Any comments? Can I have misconfigured so as to cause this?

Now as to data obfuscation: this is a public board. In my country we have a thing called the Data Protection Act. I will NOT post details that will identify a client to a public board in possible contravention of the Data Protection Act, and I feel no need to apologise for not doing so, as a Registered Data Controller.

*If* this is identified as a (putative) bug then I will post it to the bug tracker unobfuscated for diagnosis - but that point has not yet been reached.

Some help to find out if this *is* a putative bug would be useful - thank you Darrell, I will check that log tomorrow.

JR[/quote]

[slords]

That said, no thanks at all for that extremely unhelpful response.

I have not posted it as a bug because I am asking *IS* this a bug? Or is it me? Until I know which - I don't know if it is a (putative) bug. So I am posting this transcript to find out if it is a recognised consequence of mis-configuration on my part. Perhaps I did not make that sufficiently clear - I hope I have now.

You can't expect a helpful answer unless you post in the right place.  You (everyone) have been told that ANY issue with the system should be posted to the bugtracker.  We don't mind closing bugs.  If it isn't a bug then we will tell you so.  But asking here isn't going to get the attention of the people that can fix it if it is a real bug.

I hope I'm making myself (and the other developers) clear that if you are having problems first search the bugtracker and if you don't find a bug describing your issue then post one.  Can I be any clearer??

[mej]

Can I be any clearer??

Yes. You could answer my question - Is this something I could have caused through mis-configuration?

I suppose from all this that the answer is "No" and that is why you are suggesting I post it to the bug tracker, so I will do that.

Why can't I get a straight answer to a simple question? Because it's Linux

J

[bpivk]

Ok here is the answer from linux/windows user.....
NO... Please post your problem to the bugtracker. Would this be ok?
You're asking for help and thay say to post your problem to the bugtracker.

[gordonr]

GordonR, thanks very much for all the work you put in on the SME Server.

That said, no thanks at all for that extremely unhelpful response.

I'm not quite sure what you want me to say. My answer was complete, and correct. I answered your question (SME Server 7 does block mail to unknown recipients). I did not go on to say that there is an option to tweak, because there isn't one. I then  told you how to report the issue, in detail, so that we could help you to solve the problem. I asked it to be reported in the bug tracker and gave you two options to maintain privacy of potentially sensitive information.

I have spent hundreds (more likely thousands) of hours on the SME Server and have now managed a large number of releases (4.0, 4.1, 5, 5.5, 5.6, 6, 7) and sub-releases. My time is valuable, and it would be polite of you to appreciate that. I also think that as the elected Development Manager I have every right to set the policy of how issues are reported.

I say in every announcement and to every post I take time to respond to 'Please report issues to the bug tracker, and only there'. I do this as it is the most efficient way to fix any issues with the SME Server. We (the dev team) ask again and again for bugs, and potential bugs, to be raised in the bug tracker. We do this for a reason.

If we answer the question here, we answer it again in the bug tracker. Or, more likely, someone's individual problem gets fixed in the forums, and that's all that happens. But the issues don't get reported in the bug tracker and so they don't get fixed for us all to benefit. Or they get reported with a reference to "see forums" without the logs or other details we need to diagnose the problem. This means the dev team has to wander through the forums to collect the information and we don't have the time or resources to do that.

The place to report issues, and potential issues, is the bug tracker. Period.

[piran]

no thanks at all for that extremely unhelpful response.

Actually mej---- it's you that is the one that's being extremely unhelpful.
The UK Data Protection Act has got absolutely NOTHING to do with it.
It's YOUR job (fine), the people here (trying) to help you do it for free.
Perhaps you'd do well to remember that in your next response.

The Developers don't know what you might of done (config/plugins/install).
They have asked you for what they need from you.

IMHO it seems probable that you're too impatient to have spotted the
obvious in the qpsmtpd/current log. Namely that a later part of the
response algorithm drops emails for which there are no recipients.
Just like what the nice (extremely knowledgeable) people said earlier.

The bit you've apparently taken amiss is likely to be a part of the process
checking that the email address vaguely 'looks' like an email ie got the
rough bits enclosed (beginning, middle and end) NOT a specific email
address existence check. As for all the 'wibble'... well there's some here
who are probably asking themselves the very same thing about you;~)

I'm not an expert and I rely on the helpful responses of all those here.
Perhaps you might consider offering them a sincere apology?

[mej]

GordonR, thanks very much for all the work you put in on the SME Server.

That said, no thanks at all for that extremely unhelpful response.

I'm not quite sure what you want me to say. My answer was complete, and correct. I answered your question (SME Server 7 does block mail to unknown recipients). I did not go on to say that there is an option to tweak, because there isn't one. I then  told you how to report the issue, in detail, so that we could help you to solve the problem. I asked it to be reported in the bug tracker and gave you two options to maintain privacy of potentially sensitive information.

I have spent hundreds (more likely thousands) of hours on the SME Server and have now managed a large number of releases (4.0, 4.1, 5, 5.5, 5.6, 6, 7) and sub-releases. My time is valuable, and it would be polite of you to appreciate that. I also think that as the elected Development Manager I have every right to set the policy of how issues are reported.

First, I do respect the work you put in and appreciate it - I thanked you for it and it was not weasel words.

Next, my experience of reporting issues to the bug tracker has been unfortunate. This is almost certainly my fault, but I have not resolved any of the ongoing problems that I have seen with the 6->7.xx upgrade by reporting them to the bug tracker. That failure did not make the problems go away. I was completely unable to upgrade from 6 to 7 using tape because tapes written with 6 (on various specific hardware that I am supporting) were not readable on 7 (on the same hardware). There is nothing wrong with the hardware - backup and restore works both before and after the upgrade. Yes, I know, report it to the bug tracker. Well I did and got nowhere - probably because I had misidentified the problem.

I resolved the upgrade data issue using the restore from disk script due, I think, to Charlie Brady (thank you), and a secondary IDE disk copy of the hardware RAID arrays. Yes I know this is not the place to report that and I am not reporting it.  Yes I know that I should have raised the new issue with the bug tracker and worked through that - but the clients needed their servers back and I had no time to do so.

However, given that prior experience I simply asked if what I was seeing could be a result of my own mis-configuration. The answer to that is a single word, "no" and I don't think it would take that long to type it

So I am sorry if I have rattled any cages, or ruffled any feathers, I *do* know how much work goes into this and how much you all get paid for it and again, I say thanks. But would it really have been so hard to just say, no, looks like you should go to the bug tracker?

I say in every announcement and to every post I take time to respond to 'Please report issues to the bug tracker, and only there'. I do this as it is the most efficient way to fix any issues with the SME Server. We (the dev team) ask again and again for bugs, and potential bugs, to be raised in the bug tracker. We do this for a reason.

If we answer the question here, we answer it again in the bug tracker. Or, more likely, someone's individual problem gets fixed in the forums, and that's all that happens. But the issues don't get reported in the bug tracker and so they don't get fixed for us all to benefit. Or they get reported with a reference to "see forums" without the logs or other details we need to diagnose the problem. This means the dev team has to wander through the forums to collect the information and we don't have the time or resources to do that.

The place to report issues, and potential issues, is the bug tracker. Period.

I accept the logic. But I was merely asking - is this a real issue? That is all. I don't understand why this is so hard to answer. The reason I started looking at the SMTP response was simply this: I was receiving up to 500 emails for unknown recipients per day in the admin mailbox. If that is 'dropping mail for unknown recipients' then its workings are a mystery to me. On 6 with the Dungog addins they were blocked at the front door.

Anyway, as you have requested and made clear, I will cease posting here and move to the bug tracker. With trepidation: I think it should have a warning: 'Abandon hope all ye who enter here'

With genuine great respect,

apologies if necessary,

and a polite, indeed obsequious bow,

(have I demeaned myself sufficiently yet?)



James Roberts

[piran]

Noted;~)
You have my sympathy.
I too find it daunting in 'there' (bugzilla) - VERY technical & unforgiving.
But that's the sharp end and its where the guts of an issue get fixed.
They'll still want your log files, so brace yourself;~)
Curious symptoms... have you been 'Joe Jobbed' do you think
ie your site been 'quoted' as the source of a spam wave?

[mej]

Noted;~)
You have my sympathy.
I too find it daunting in 'there' (bugzilla) - VERY technical & unforgiving.
But that's the sharp end and its where the guts of an issue get fixed.
They'll still want your log files, so brace yourself;~)
Curious symptoms... have you been 'Joe Jobbed' do you think
ie your site been 'quoted' as the source of a spam wave?

No its not a joejob, just a client with mostly East Asian contacts. I have had a joejob - that's worse

Thanks for the sympathy And I did not intend to offend anyone. It's been a difficult Friday, admittedly, with BT pulling out 3 of 5 T1s to our backbone providers

 

I have been using e-smith since early version 4. I know it pretty well,- for a user. I've been using Linux on and off since 1993 and qualified as a programmer/analyst in 1979 (IBM assembler:) so I am an old f**t. I claim no expertise whatever in the SME Server except as a user. I know Linux and BSD reasonably well but am still most definitely an amateur.

I have been using and supporting email on bulletin board systems since before the Internet became generally available and have had a (commercial) Internet email account since 1987.

My point in saying so is only, I have some slight experience of email and how it is supposed to work.

Large parts of the issues with e-smith/SME Server email handling are because the developers have chosen to stick with QMail. They have very good and cogent reasons for doing so (I am inclined to disagree with them - but I respect their choice), but because of its design it is an issue for it to drop mail to unknown senders, and this issue requires various workarounds (basically a front end).

I stated that I was receiving up to 1000 emails a day for unknown users to admin, and therefore checked the SMTP conversation. I specified the condition - clean install with no contribs. I asked if I had done something stupid (wouldn't be the first, or last time). This was to filter out a spurious bug report from myself. It does not seem unreasonable to me.

The bit you've apparently taken amiss is likely to be a part of the process
checking that the email address vaguely 'looks' like an email ie got the
rough bits enclosed (beginning, middle and end) NOT a specific email
address existence check. As for all the 'wibble'... well there's some here
who are probably asking themselves the very same thing about you ;~)

Quite so

I'm not an expert and I rely on the helpful responses of all those here.
Perhaps you might consider offering them a sincere apology?

I have bowed and scraped, hope that will suit. I have no pride when faced with a problem I can't solve, I will ask for help anywhere I can get and take it gratefully when offered

Now as to the UK Data Protection Act. That is my responsibility and I repeat I will not post identifying information in a public forum. We have to negotiate various legislation, including the UK DPA, RIPA, EHCR and others, and I am not going to take any risks with legislation that can see you banged up (even if it is very unlikely). I have to show due diligence and I offer no apologies for doing so

However let me demonstrate a couple of SMTP conversations (client data sanitised).

Here is one with a Barracuda in front of an e-smith:

Code: [Select]


01/12/07 17:20:50 SMTP Verify trash@example.co.uk, at mail.example.co.uk
Contacting 217.x.x.210
220 barracuda.example.co.uk ESMTP (13dd60aee979e619b83ae9387882d649)

HELO stabilys.com
250 barracuda.example.co.uk

VRFY trash@example.co.uk
502 VRFY command is disabled

Doesn't want to talk to us
RSET
250 Ok

EXPN trash@example.co.uk
502 Error: command not implemented

Doesn't want to talk to us
RSET
250 Ok

MAIL FROM:<me@stabilys.com>
250 Ok

RCPT TO:<trash@example.co.uk>
553 <trash@example.co.uk>: Recipient address rejected: Sorry, that is an invalid e-mail address

421 Error: too many errors

Doesn't want to talk to us
RSET
QUIT

The Barracuda is there because the client unwittingly bought a domain name that had been used by a spammer. We were getting tens of thousands of bounced and forged emails a day at first

*That* is rejecting mail to an unknown recipient. And note - no wibble.

Here is the dialogue for our own mail server running exim:

Code: [Select]


01/12/07 17:57:55 SMTP Verify trash@stabilys.com, at mx1.balanced.thingy.mail.example.com
Contacting 208.97.132.51
220 thingymail-mx2.example.com ESMTP

HELO stabilys.com
250 thingymail-mx2.example.com

VRFY trash@stabilys.com
502 VRFY command is disabled

Doesn't want to talk to us
RSET
250 Ok

EXPN trash@stabilys.com
502 Error: command not implemented

Doesn't want to talk to us
RSET
250 Ok

MAIL FROM:<me@stabilys.com>
250 Ok

RCPT TO:<trash@stabilys.com>
550 <trash@stabilys.com>: Recipient address rejected: User unknown in virtual alias table

Doesn't want to talk to us
RSET
250 Ok

QUIT
221 Bye


Note unknown recipient rejected. And no wibble.

Now the same thing for SME 6 without the Dungog addins:

Code: [Select]


01/12/07 17:29:03 SMTP Verify trash@client.example.com, at client.example.com
Contacting client.example.com
220 serv1.example.com mailfront ESMTP

HELO stabilys.com
250 serv1.example.com

VRFY trash@client.example.com
252 Send some mail, I'll try my best.

EXPN trash@client.example.com
500 Not implemented.

Doesn't want to talk to us
RSET
250 OK

MAIL FROM:<spade@stabilys.com>
250 Sender accepted.

RCPT TO:<trash@client.example.com>
250 Recipient accepted.

RCPT TO:<bogus22943@client.example.com>
250 Recipient accepted.

RSET
250 OK

QUIT
221 Good bye.


Note that the junk addresses are accepted. (But no wibble).

Here is the SME 7 dialog posted earlier:

Code: [Select]


01/11/07 10:26:45 SMTP Verify tripe@client.co.uk, at mail.client.co.uk
Contacting 217.154.x.x
220 targetserver1.client.co.uk ESMTP
HELO example.com
250 client.co.uk Hi mail.testingbox.co.uk [213.254.x.x]; I am so happy to meet you.
VRFY tripe@client.co.uk
252 Just try sending a mail and we'll see how it turns out ...
EXPN tripe@client.co.uk
500 Unrecognized command
Doesn't want to talk to us
RSET
250 OK
MAIL FROM:<spade@example.com>
250 <spade@example.com>, sender OK - how exciting to get mail from you!
RCPT TO:<tripe@client.co.uk>
250 <tripe@client.co.uk>, recipient ok
RCPT TO:<bogus11205@client.co.uk>
250 <bogus11205@client.co.uk>, recipient ok
RSET
250 OK
QUIT
221 client.co.uk closing connection. Have a wonderful day.

Note that the junk addresses are accepted. And loadsa wibble. I can't demonstrate the result due to the Dungog addins, as the machine is upgraded now to 7 - but it was like the first one above.

This is not rejecting email addresses for unknown recipients. It is ACCEPTING email for unknown recipients. If they are dropped later then that is something else, but it is not rejecting. This means that client bandwidth is used up - though probably not much. Rejecting it uses no bandwidth. BLOCK =/= ACCEPT and DROP.

As far as the wibble is concerned, Sendmail used to provide similar chatty responses but they do not look very professional and are only funny the first few times, so they were disabled years ago; I'm wondering how they crept back in here! I know the messages are coming from qpsmtpd - but I don't want them. I want to turn them *off*, and want to know if there is a canonical e-smith way to do so. Ialso find qpsmtpd occasionally slow - there are only 86400 secs in 24 hours and if it takes 2 secs to receive an email that limits us to (much less than) 43200 transactions a day: some of my clients may push that (because of the spam load).

For one (of thousands) of pages listing reasons for not using QMail see: http://www.pgregg.com/forums/viewtopic.php?id=24&qid=155

Anyway I think that is more than enough on this and I have moved the issue to the bug tracker. As requested

Have a nice weekend, all. And more apologies to anyone who I have inadvertently contrived to offend...

MeaJamjaR spinning quietly...  

[piran]

...awaiting the explanation of, and any resolution to,
the apparent issue you've raised - with much interest;~)

[gordonr]

Large parts of the issues with e-smith/SME Server email handling are because the developers have chosen to stick with QMail.

http://lists.contribs.org/mailman/public/devinfo/msg08093.html

However let me demonstrate a couple of SMTP conversations (client data sanitised).

This is precisely the information we need in the bug tracker if we're to resolve your issue. Why write it twice?

Let me be clear - if a standard SME7 box is allowing mail to invalid recipients, I want to know about it. It would be a serious issue which would need to be fixed.

I presume you do have "E-mail to unknown users    Return to sender" configured on the E-mail panel?

Apologies, I misspoke before, there is an option to not block mail for invalid recipients. This can be useful for testing during conversion (which is the only reason we left it in), but should normally be disabled. The default is disabled/reject.

As far as the wibble is concerned,

http://bugs.contribs.org/show_bug.cgi?id=1325

For one (of thousands) of pages listing reasons for not using QMail see: http://www.pgregg.com/forums/viewtopic.php?id=24&qid=155

So, does it matter enough to you to fund the development and testing?

http://lists.contribs.org/mailman/public/devinfo/msg09015.html
http://lists.contribs.org/mailman/public/devinfo/msg09053.html
http://lists.contribs.org/mailman/public/devinfo/msg09058.html
http://lists.contribs.org/mailman/public/devinfo/msg09039.html

Anyway I think that is more than enough on this and I have moved the issue to the bug tracker. As requested

What is the bug reference?

[mej]

However let me demonstrate a couple of SMTP conversations (client data sanitised).

This is precisely the information we need in the bug tracker if we're to resolve your issue. Why write it twice?

Because these were examples on other systems illustrating the rejection of email for piran. They would not constitute a valid bug report.

Let me be clear - if a standard SME7 box is allowing mail to invalid recipients, I want to know about it. It would be a serious issue which would need to be fixed.

I presume you do have "E-mail to unknown users    Return to sender" configured on the E-mail panel?

I do not. I see no point whatsoever in returning mail to someone who did not send it (1). I do not want to be part of an NDN attack (2). Nor do I want to accept badly-addressed mail in the first place! I thought SME7 was supposed to reject it by default? There is no option for this in the long list of accounts it can be delivered to. Are we saying that email for unknown recipients is still sent to admin (if that is selected) and not rejected then? "Return to sender" is not = rejected in my book.


This is what my original query was concerned with. (!!)

I will check further to see if anything else is wrong with the set up before I post a bug.

Apologies, I misspoke before, there is an option to not block mail for invalid recipients. This can be useful for testing during conversion (which is the only reason we left it in), but should normally be disabled. The default is disabled/reject.

Not a problem. I have been known to mispeak myself

For one (of thousands) of pages listing reasons for not using QMail see: http://www.pgregg.com/forums/viewtopic.php?id=24&qid=155

So, does it matter enough to you to fund the development and testing?

http://lists.contribs.org/mailman/public/devinfo/msg09015.html
http://lists.contribs.org/mailman/public/devinfo/msg09053.html
http://lists.contribs.org/mailman/public/devinfo/msg09058.html
http://lists.contribs.org/mailman/public/devinfo/msg09039.html

Perhaps (at least in part!). As I stated, I understand why you (all) have not changed. In Debian I just have to do an apt-get to change mail server. Because of the intricate clockwork (alright templates) holding e-smith together, it is no trivial task to change a component (which is both its merit and a pain in the a***), and qmail is pretty solid as a mailserver, it's the one big merit of its design. You may prefer sponsorship of other, more pressing, missing features. And so might I. Ideally, though, a mail server that simplified dealing with today's problems could be a good move.

Oh. You did not pickup on the speed issue. Sometimes the response is so slow that the next mail goes to the backup MX - which spoils the whole idea and lets anything in. This was less of a prob with 6.

What is the bug reference?

I was hoping for a day off (having worked right through Xmas and last weeked- aah)! But will post this tomorrow. I have just had a half bottle of wine and now is not a good moment

It's very late. G'Night.

(1) http://www.city-fan.org/mailwasher-bounce-bad.html (and many others)
(2) http://www.techzoom.net/paper-mailbomb-7.asp

[piran]

Good to see dialogue, I'm sure the community will benefit;~)

gordonr---- I read your response to mej's 'wibble' concern...
http://bugs.contribs.org/show_bug.cgi?id=1325
...but it's not fully appropriate.
FWIW I too have been uncomfortable with all such rubbish.
Is there a more straightforward and laconic mode of response?
My site shows...
220 myservername.mydomain
...but the wibble is the inane...
250 <admin@contribs.org>, sender OK - how exciting to get mail from you!
...stuff. I looked through the remaining files in...
/var/service/qpsmtpd/config
...but none contained the ' -how exciting to get mail from you!' drivel.
Is there somewhere else I must look to cull the inappropriateness?

mej---- Just a small point... my site finds the response lag quite useful.
In SME6 days the RBL lookup frequently beat the latter's ability to post
new culprits, my log post mortems clearly demonstrated this slightly ironic
situation. With a brace of dozen or so RBLs slowing things down further
I find that SME7's spam control procedures extremely effective overall.
Yes, I grant that my site doesn't bear the sheer load that you quoted.

FWIW the MX was a spam nightmare, I've switched it off. 100% of any
delivered spam came from that route. Indeed most of the log post
mortems showed that the secondary MX was deliberately used.

[gordonr]

I do not. I see no point whatsoever in returning mail to someone who did not send it (1). I do not want to be part of an NDN attack (2). Nor do I want to accept badly-addressed mail in the first place! I thought SME7 was supposed to reject it by default?

It does and this explains the behaviour on your system. The wording on the panel is now inaccurate with the changes in SME7 (which has previously been raised, but has not yet been corrected). It should say "rejected" rather than "returned". And I have ensured that a bug is raised for that wording change:

http://bugs.contribs.org/show_bug.cgi?id=2291

Mail for unknown users in SME7 is rejected by default during the SMTP transaction. SME6 returned the mail (qmail backscatter). This change was a specific design goal of SME7 and works as advertised and is in line with the recommendations in your links. It was one of my primary goals for the SME7 mail system changes.

However, you have asked it to not reject mail for unknown users, so you get everything addressed to local domains. That mail defaults to delivery to admin, but you could, for example, send it to a 'deadletter' user which you can check from time to time (if you like wading through spam for misdirected mail).

If you ask for recipient filtering, you get it (and that is the default). If you want everything for a domain, you can have that too (the option you have chosen).

Some people find this useful, particularly in new installs. Most people switch back to the dedault of full recipient filtering/rejection once they've checked that all looks o.k.

In Debian I just have to do an apt-get to change mail server.

And then write the configuration. Installing the new MTA is the easy part. The hard part is the configuration changes which need to be made to support all of the features of the SME Server (e.g. pseudonyms, group aliases, mailing list support addons, etc). We do a lot of qmail configuration, and would have to do a similar amount of configuration for any other MTA. Templates actually ease that work, but it still has to be done, and has to be tested.

As I stated in the threads I linked to from devinfo, most of the work has been done for Postfix support, but there are some difficult upgrade issues and a lot of testing time required. That "last 10%" is the expensive part of most development tasks.

My estimate is that there is at least two weeks fulltime for a senior developer to complete the conversion and deal with the upgrade issues. That's not a trivial amount of work, which is why we haven't done it. And I expect a number of contribs to break after the conversion.

Oh. You did not pickup on the speed issue. Sometimes the response is so slow that the next mail goes to the backup MX - which spoils the whole idea and lets anything in. This was less of a prob with 6.

Then raise it in the bug tracker so we can deal with it.

Similarly with the response messages - raise them in the bug tracker so that we can deal with them.

[gordonr]

Is there somewhere else I must look to cull the inappropriateness?

No, unfortunately. The strings come from the qpsmtpd code and should be configurable. I put in patches for this some time back, but they have not been applied.

Raise it in the bug tracker, please.

[gordonr]

I've taken the liberty of renaming the thread as there's a lot of information in here, but the fundamental issue is captured in this bug report:

http://bugs.contribs.org/show_bug.cgi?id=2291

[mej]

I do not. I see no point whatsoever in returning mail to someone who did not send it (1). I do not want to be part of an NDN attack (2). Nor do I want to accept badly-addressed mail in the first place! I thought SME7 was supposed to reject it by default?

It does and this explains the behaviour on your system. The wording on the panel is now inaccurate with the changes in SME7 (which has previously been raised, but has not yet been corrected). It should say "rejected" rather than "returned". And I have ensured that a bug is raised for that wording change:

http://bugs.contribs.org/show_bug.cgi?id=2291

OK, you have convinced me. I will raise any future points that may be even slightly doubtful in the Bug Tracker.

But just as you asked for courtesy, I'd ask that it be applied there too. I feel that as soon as I raise a bug, there is an implied confrontational situation (= "you got to be an idjit"). That is why I avoid the bug tracker and have done ever since my first experiences with previous versions of it.

I do understand that most issues are in fact PEBCAK/EBSAC (1)- I have some hundreds of users at different companies to remind me of this - and this is why I raised the issue here instead. Because I was correct in my supposition - I had misconfigured.  As far as:

> omg, it took forever to find his problem
> i'll draft a patch

is concerned - this could have been established immediately after my first message.

But there wasn't a bug in the system software - it was an inconsistency in the interface - it lied to me about what the effect of a setting was.

Incidentally: should previously-raised unresolved interface issues not be added to the release notes?

FWIW, I immediately changed the settings and another test shows it operating as stated:

Code: [Select]


01/13/07 12:35:37 SMTP Verify trash@client.co.uk, at mail.client.co.uk
Contacting 217.154.x.x
220 mailserver1.client.co.uk ESMTP

HELO stabilys.com
250 client.co.uk Hi 85-210-x-x.dsl.x.com [85.210.x.x]; I am so happy to meet you.

VRFY trash@client.co.uk
252 Just try sending a mail and we'll see how it turns out ...

EXPN trash@client.co.uk
500 Unrecognized command

Doesn't want to talk to us
RSET
250 OK

MAIL FROM:<me@stabilys.com>
250 <me@stabilys.com>, sender OK - how exciting to get mail from you!

RCPT TO:<trash@client.co.uk>
550 invalid recipient trash@client.co.uk

Doesn't want to talk to us
RSET
250 OK

QUIT
221 client.co.uk closing connection. Have a wonderful day.



Thank you. Just the wibble to go. YES I will raise a bug.

In Debian I just have to do an apt-get to change mail server.

And then write the configuration. Installing the new MTA is the easy part. The hard part is the configuration changes which need to be made to support all of the features of the SME Server (e.g. pseudonyms, group aliases, mailing list support addons, etc). We do a lot of qmail configuration, and would have to do a similar amount of configuration for any other MTA. Templates actually ease that work, but it still has to be done, and has to be tested.

As I stated in the threads I linked to from devinfo, most of the work has been done for Postfix support, but there are some difficult upgrade issues and a lot of testing time required. That "last 10%" is the expensive part of most development tasks.

My estimate is that there is at least two weeks fulltime for a senior developer to complete the conversion and deal with the upgrade issues. That's not a trivial amount of work, which is why we haven't done it. And I expect a number of contribs to break after the conversion.

I know all this (* though note Debian does much of the re-configuration too). There has been loadsa discussion elsewhere about what priorities to adopt in SME Server. Personally I'd prefer to have it based on Debian Not holding my breath.

I don't know what the most pressing issues are for changes to the system.

What has troubled me most has been difficulties in migration which YES I will raise again as a new bug.

If you want sponsorship for a change to Postfix then some could be arranged. I'll PM you (or you me).

JR

(1) http://www.johnrostron.co.uk/fun/nonsense/pebcak.htm

[piran]

Is there somewhere else I must look to cull the inappropriateness?

Raise it in the bug tracker, please.

http://bugs.contribs.org/show_bug.cgi?id=2293

[mej]

FWIW the MX was a spam nightmare, I've switched it off. 100% of any
delivered spam came from that route. Indeed most of the log post
mortems showed that the secondary MX was deliberately used.

Yeah too right

However it is not my site I am talking about. Clients are IME prepared to accept loadsa spam rather than lose an email. When an email can represent 50K to them I can understand this

Since they are generally on ADSL/SDSL sometimes it goes down or has maximum contention so the backup MX is needed.

I'm looking into other ways of solving the problem optimally (probably involving us sticking a box in a rack somewhere to host the backup MX with filtering ourselves...)

MeJR

[piran]

What a busy chap, would that any of my emails came carrying 50K;~)
FWIW have filed a New Feature Request against the inane text strings
(aka wibble) but it looks highly probable we're all stuck with the stuff.

[mej]

would that any of my emails came carrying 50K;~)

Me2

Many of our clients are solicitors (lawyers). + We have a major corporate headhunter. Sadly our fees are at a different level...  

There is a *lot* of competition around.

80% of our clients are on Windows Server 2003 and more and more on Citrix (with yet another Windows 2003 server) but I still have 4 on SME-Server. Trying to keep it alive...

MeJ

[CharlieBrady]

Since they are generally on ADSL/SDSL sometimes it goes down or has maximum contention so the backup MX is needed.

No, it is not needed. If there is no backup MX, and the server or connection is down, the sending mail server will keep the mail and retry later.

[mej]

Since they are generally on ADSL/SDSL sometimes it goes down or has maximum contention so the backup MX is needed.

No, it is not needed. If there is no backup MX, and the server or connection is down, the sending mail server will keep the mail and retry later.

I absolutely agree that this is RFC and how it is supposed to be.

Unfortunately no-one has told the sysops of a large number of mailservers - which return mail as undeliverable if not delivered first attempt. Mostly they are not running Free or Open mailservers.



J

[gordonr]

Unfortunately no-one has told the sysops of a large number of mailservers - which return mail as undeliverable if not delivered first attempt. Mostly they are not running Free or Open mailservers.

I have not seen such behaviour for many years. Your experiences may vary, but I think there's a bit of net folklore here. Any mailer which doesn't handle retries isn't speaking SMTP - it's not a matter of strict compliance, it's a basic part of SMTP.

IMO the best thing to do is to force compliance in this case. If a mailer doesn't retry, that sender will lose mail and papering over the cracks doesn't help anyone in the longer term.

What I have seen is broken mailers which treat 4xx temporary error codes as 5xx permanent failure codes. A backup MX probably won't help all that much in those cases as there are plenty of reasons why you might need to return a 4xx code from time to time. Those mailers are just as broken and we (the recipient) can't fix them. The kindest thing to do is to make it plain that the sender's mailer is broken and needs to be fixed.

I understand how valuable these mails might be, but you fundamentally can't make any sort of guarantee if the client doesn't retry, or ignores temporary errors. A backup might reduce the chance of losing the mail, but probably not by much.

Gordon

[mej]

...mailservers - which return mail as undeliverable if not delivered first attempt

I have not seen such behaviour for many years. Your experiences may vary, but I think there's a bit of net folklore here. Any mailer which doesn't handle retries isn't speaking SMTP - it's not a matter of strict compliance, it's a basic part of SMTP.

IMO the best thing to do is to force compliance in this case. If a mailer doesn't retry, that sender will lose mail and papering over the cracks doesn't help anyone in the longer term.

What I have seen is broken mailers which treat 4xx temporary error codes as 5xx permanent failure codes. A backup MX probably won't help all that much in those cases as there are plenty of reasons why you might need to return a 4xx code from time to time. Those mailers are just as broken and we (the recipient) can't fix them. The kindest thing to do is to make it plain that the sender's mailer is broken and needs to be fixed.

I understand how valuable these mails might be, but you fundamentally can't make any sort of guarantee if the client doesn't retry, or ignores temporary errors. A backup might reduce the chance of losing the mail, but probably not by much.

Gordon

It's a tricky call. We cannot afford to lose email for clients. We give them the option - they prefer to have the backup. It's probably because "backup" sounds good to them.

However I have seen mail go missing in exactly this way - correction, I have received *reports* from clients whose contacts are complaining that mail is undeliverable. I've seen the complaints too, but can't read them - they're in Chinese!

It was the experience of large numbers of such reports that caused us to arrange the backup MX.

As far as our own system is concerned, I only want mail from compliant sources.

The 4xxx/5xxx issue you mention also breaks greylisting

We've tried.

Is it a coincidence that many of these broken mailers seem to be in the Far East?

JR

[gordonr]

It's a tricky call. We cannot afford to lose email for clients. We give them the option - they prefer to have the backup. It's probably because "backup" sounds good to them.

The tricky part with any backup MX is keeping the config in sync with the master (if you want to do complete recipient filtering on the backup, which I pretty much bet that you do). I've seen lots of misconfigured backup MX servers which are out of sync with the master. This means that they bounce mail precisely when you need them

An rsync of /home/e-smith/db/ (at least domains and accounts) to the backup MX, followed by "signal-event email-update" would be very close to what you want.

Note that the backup would want to be in "Delegate Mail Server" mode, queuing for the primary, or an internal mail server, so the sketch above isn't quite enough.

Also, in delegate mail server mode we do an SMTP connection to the internal mail server and return 4xx if we can't connect to it.