Topic: ON SME 9.2, trouble when receiving some mails

[schirrms]

Hi,

(Long time not been here )

I migrated my SME server to a new server running SME 9.2 this summer. I recently discovered that some incoming mails aren't received anymore.

As far as I understande, the trouble is with the SPF check : in some case, this test take 2 minutes. Then SME is happy to continue the process, but some server have closed the call in beetween.

Here is a sample transaction log :
2017-09-24 15:36:43.371677500 5078 Accepted connection 0/40 from 37.148.183.96 / webgridb096.emsecure.net
2017-09-24 15:36:43.371861500 5078 Connection from webgridb096.emsecure.net [37.148.183.96]
2017-09-24 15:36:44.579465500 5078 (connect) earlytalker: pass, not spontaneous
2017-09-24 15:36:44.580338500 5078 (connect) relay: skip, no match
2017-09-24 15:36:44.606449500 5078 (connect) dnsbl: combined.njabl.org query failed:  SERVFAIL
2017-09-24 15:36:44.606494500 5078 (connect) dnsbl: pass
2017-09-24 15:36:44.606829500 5078 220 schirrms.net ESMTP
2017-09-24 15:36:44.649780500 5078 dispatching EHLO webgridb096.emsecure.net
2017-09-24 15:36:44.650403500 5078 (ehlo) helo: pass
2017-09-24 15:36:44.650932500 5078 250-schirrms.net Hi webgridb096.emsecure.net [37.148.183.96]
2017-09-24 15:36:44.650964500 5078 250-PIPELINING
2017-09-24 15:36:44.650987500 5078 250-8BITMIME
2017-09-24 15:36:44.651012500 5078 250-SIZE 15000000
2017-09-24 15:36:44.651036500 5078 250 STARTTLS
2017-09-24 15:36:44.693232500 5078 dispatching MAIL FROM:<info@mailing.action.com>
2017-09-24 15:36:44.747407500 5078 (mail) resolvable_fromhost: pass, mailing.action.com has MX at mx1.slgnt.eu
2017-09-24 15:36:44.771056500 5078 (mail) rhsbl: pass
************* Here is the 2 minutes wait time ********************************
2017-09-24 15:38:45.053209500 5078 (mail) sender_permitted_from: fail, tolerated, permerror, mailing.action.com: Included domain 'spf.slgnt.eu' has no applicable sender policy
2017-09-24 15:38:45.053452500 5078 (mail) naughty: pass
2017-09-24 15:38:45.053971500 5078 250 <info@mailing.action.com>, sender OK - how exciting to get mail from you!
2017-09-24 15:38:45.054242500 5078 dispatching QUIT
2017-09-24 15:38:45.054480500 5078 221 schirrms.net closing connection. Have a wonderful day.
2017-09-24 15:38:45.054527500 5078 click, disconnecting

It's not related to one remote domain, as soon as I have the sentence has no applicable sender policy I have the two mintues wait time.

I think that this answer come from onof these two files :
/usr/share/perl5/vendor_perl/Mail/SPF/Mech/Include.pm
/usr/share/perl5/vendor_perl/Mail/SPF/Mod/Redirect.pm
but then, it overcomes my Perl skilness (and I also can be totally wrong).

So, could some of you see if you have also this wait time of 2 minutes in that situation
And if the case is common, any idea how to shorten this time ?
Even is the RFC say that the remote sender should wait 2 minutes, I find that wery log !

Thanks,

Pascal

[schirrms]

Hi,

It seems that I am alone... OK, after hour trying to find where qpsmtpd can lost 2 minuts, I cannot do more. Is there a way to stop using QPSMTP ? I cannot accept that many mails are lost on incomming... I think I'll have to migrate all messagebox on a new place, but in between, can you help me to connect QMAIL directly on Internet (that will be a mess, I know...)

Thanks,

Pascal

[ReetP]

Pascal,

It will be better to get to the root of the issue.

Have you read this?

https://wiki.contribs.org/Email#Qpsmtpd_for_SME_versions_9.2_and_Later

I could be wrong but I seem to remember someone else having a similar issue. Have you searched the forums and bugs?

[Daniel B.]

cannot accept that many mails are lost on incomming

Are email rejected ? Or simply "deferred" for 2 minutes ? Looks like a DNS issue to me. Are you using a DNS forwarder ?

[mmccarn]

From your log extract it looks like "sender_permitted_from" may be introducing a delay.

I've created a wiki page with the perldoc info for sender_permitted_from:
https://wiki.contribs.org/Qpsmtpd:sender_permitted_from

There is a setting for 'sender_permitted_from' that says that talks about temporarily deferring the connection; I don't know if this would cause your observed behavior:

Most sites should start at level 3. It temporarily defers connections (4xx) that have soft SFP failures and only rejects (5xx) messages when the sending domains policy suggests it.

In any case, you should be able to completely disable sender_permitted_from by creating a custom template fragment, copying the existing fragment, then putting a "#" in front of the line...

[schirrms]

Hi, everybody,

Thanks for yours answers, I wasn't a lot at home today.

I did some search in the forums, and didn'd find any answers. That is not a proof that nothing is written ! It's only the result of my research.

For instance, I didn't find the spécific part about SMLE 9.2 and qpsmtpd0.96, thanks ReetP

Daniel, yes I use a DNS forwarder, as I did on SME 8 before (My SME Server is behind a PFsense fierewall, witch acts also as a DNS forwarder/server).

I'm not completely sure about a DNS Problem : so far, I identify problems on site with a wrong SPF record (or at least, an SPF record not complete.) If I receive a mail from a domain without SPF at all, no 2 minutes wait time. If I receive a GMail mail (with a very complex and cascaded SPF records, at least 5 spf records to query), everything is fine also.
But If I receive a mail from a host that is not recorded in the SPF record, there is the trouble (I think, I'm not totally sure for now, I have to check furter)

mmccarn, I'll try to comment the spf plugin to see if the trouble is here.

I'll let you know.

Thanks again,

Pascal

[schirrms]

Hi, everybody,

with a custom template for the fragment /etc/e-smith/templates-custom/var/service/qpsmtpd/config/peers/0/221spf, with that (very original !) content :
# PS TEST (2017/11/05)
#sender_permitted_from reject 1 no_dmarc_policy { $qpsmtpd{SPFRejectPolicy} || '0' }

it works !

Daniel, as you are in France, can you please confirm that you have (or not) the trouble when asking to subscribe to the newletter on this page ?
https://www.action.com/fr-fr/Newsletter_footer/

This service send within a minute a confirmation mail.

I don't know if there is any control to use it outside France. At least, it would write in French

For information, I had also the very same trouble for amazon mails, who seems to be send fron a lu server not in the amazon SPF record. Very annoying in that "pre Xmas" time !

Thanks again, I'll really would know if this problem is general or (more probably) specific on my side.

Pascal,

[Daniel B.]

Have you tried without the DNS forwarder ? (using SME's own resolver)

[schirrms]

Hi Daniel,

That vould be a little complicated

I can probably try this WE.

I'll let you know.

Thanks,

Pascal