Topic: Suddenly getting lots of theses reports

[SchulzStefan]

Yes, your server must be able to make outbound connections on ports 80 and 443, and the entire Internet needs to be able to reach your server on port 80.

At this point, letsyencrypt is not for me. That does not meet my security concept for a server, which is not hosting a webpage for the entire world and is "only" fetching and distributing email from ISP in the internal net. This server is meant as a fileserver for internal use, nothing else. Of course with the ability of sending email.

You should not have needed to manually edit domains.txt.  Simply set letsencryptSSLcert to enabled for those hostnames and/or domains you want to be named on the cert, and only for those hostnames and/or domains.  Then, a signal-event console-save will generate the files properly.

In the how-to is said: https://wiki.contribs.org/Letsencrypt#Prerequisites:

The Letsencrypt client and server interact to confirm that the person requesting a certificate for a hostname actually controls that host. For this reason, there are some prerequisites for your configuration. For example, if you're trying to obtain a certificate for www.example.com, the following conditions must be met:

    www.example.com is a valid domain name--the domain has been registered, and DNS records are published for it.
    www.example.com resolves to your SME Server--published DNS records give the external IP address of your SME Server when queried for www.example.com.
    Your SME Server is connected to the Internet.
    Port 80 on your SME Server is open to the Internet--you aren't behind a firewall, or some ISP filtering, that would block it.

Letsencrypt will issue certificates that include multiple hostnames (for example, www.example.com, example.com, and mail.example.com), all of which would be part of the request. All of the conditions above must be true for all of the hostnames you want to include in the certificate.

Make sure you've got this all set up correctly before continuing.

On this server there are a few virtual domains which are not registered. Therefore not valid. Further I understand, that in the case of valid domains every cname should bei altered to point/resolve the IP of the SME. Am I wrong?

As I stated before, for me (maybe only for me) at the first place the SME is for internal use. There is too much around to tweak, what I don't like to do out of several reasons.

regards,
stefan

[SchulzStefan]

FYI

https://community.letsencrypt.org/

Seems to me, that it's not working out-of-the-box.

regards,
stefan

[Stefano]

unfortunately for you, you're wrong..

99% the issues reported by user on forums are about misconfiguration or sort of.. IOW, it's not the tool that doesn't work, it's how it's used.

in any case, if it doesn't work out of the box for you, you 'd know that you'd open a bug giving all the needed info to help us to debug it; "it doesn't work" means nothing and it's useless.

[Jean-Philippe Pialasse]

Having a domain existing and resolvable on the internet, does not mean it should also resolve to the same box on your lan. You can still use your internal dns to modify locally resolution. Also you have few alternatives if a server 80 port is not accessibe from outside. Either using dns to prove you control the donain or use another server to get the certificates and deploy it to your internal server.

While you can also have a proxypass configuration to only redirect the letsencrypt verification to your internal server as a third solution.

[DanB35]

Seems to me, that it's not working out-of-the-box.

Nonsense, and I think you know better than that.  Some people will have problems with any software; the fact that there are people reporting problems in no way shows that the software or system is defective.  I'm reasonably active on the Let's Encrypt forum, and can confirm what Stefano is saying--the large majority of problem posts there (which themselves are a very small fraction of the millions of people who are using Let's Encrypt) are a matter of either (1) people configuring something wrong, or (2) Let's Encrypt itself working just fine, but then they don't know what to do with the cert once they receive it.

But none of that has anything to do with your case.  If you're unwilling to open port 80 of your SME server, things get more complicated for you, but you still have options.

• You can use the DNS challenge; I posted links on this up-thread
• You can use the built-in hook script functionality to copy the challenge files to an Internet-accessible location--this isn't documented very well at this time;
   I'll try to get something added to the wiki page shortly
• You may be able to use some sort of reverse proxy arrangement as JPP mentions
• You can obtain the cert on your firewall and deploy it from there to your SME box

Any of these are going to require some scripting and/or custom template fragments, but they're all options.

[guest22]

Maybe it's all a bit complicated to understand what is under the hood. Maybe the contrib should be more verbose and give advise. Especially as it is targeted as a core part for SME 10.

[DanB35]

Maybe it's all a bit complicated to understand what is under the hood.

That's true of most of SME; it's designed so that users don't need to understand what's under the hood.  What about this contrib, from a user's perspective, do you think is too complicated?

Maybe the contrib should be more verbose and give advise.

What advice do you think the contrib should give that it doesn't?

[guest22]

Well, the reports indicate that the contrib is not verbose at all. So fot he 'I installed SME Server and need a certificate' user, there nothing much to report. Let alone one understands the importance or what it does.

[guest22]

To add to that, the the smeserver-letsencrypt contrib does not work. period. Evidence has been presented by not too stupid guys.


The manual install works perfectly tho and provides feedback.

[guest22]

I have provided enough detailed feedback from a production server, and I will stick to the manual install.

[Drifting]

Well I am confused..

From an end user, with a moderate amount of knowledge, is there any documentation in plain english that I could understand, so that that I may make an informed guess about Dmark etc and if I should implement it. So far all I have seen it do is reject genuine email. Please do not send me to some random FAQ that assumes you understand every part of the mail server system, I don't!

I thank whoever did the document, as I would be in an ever worse state without it, but really need to understand the part about delegate mail server, as this is what I use. I forward external mail to my ISP, they in turn relay mail back to my SME. Could not get my head round the docs on this sequence.

Can I also clarify that it is best to change the domains from .local to the genuine? from what I have read, the answer is yes?

Is this Dmarc going to be a standard all servers will implement? Seems to me it is going to generate a massive amount of of email to postmaster mailboxes around the world? I must point out this is only from my very brief understand of what I have seen leaving my own servers. (Got a stinking cold so not the most coherent at the moment)

Paul.

[DanB35]

I have provided enough detailed feedback from a production server,

You have provided very little feedback, and the most important part of it was wrong--it's simply incorrect that "dehydrated -c" returns you to the shell prompt with no output, and your reporting that sent me on a wild goose chase.  If you (or @ShulzStefan) had indicated that the command did return some output (and better yet, what that output was), that would have saved some time in narrowing down the issue.

To add to that, the the smeserver-letsencrypt contrib does not work. period.

...except for all the servers for which it does work.  period.

There's something about the three curl commands that the contrib adds to the config file that is breaking things on at least some systems.  Why that should be the case is puzzling to me.  My production server is a SME 9.2 VM, on a Proxmox host, in server-only mode, behind a pfSense firewall.  It has the contrib, and the curl commands don't cause any problem at all.  My test box is a SME 9.2 VM, on a Proxmox host, in server-only mode, behind a pfSense firewall, and the curl commands cause it to die.  The curl commands are identical, both machines are running the same version of curl, and they're in identical network configurations.  But one works and the other doesn't.  And when I remove the redirects from the one that doesn't work (so I can see the error it's returning), it starts working.

Yes, there's clearly a problem, and yes, it needs to be fixed.  But it's wrong to say "it does not work. period."

Well, the reports indicate that the contrib is not verbose at all.

You realize that the contrib uses the exact same dehydrated script as the manual installation, right?

[Stefano]

I agree with Dan

most of our contribs are done to add features hiding all the complications.

If something isn't working (but it used to do) the only right approach is to open a bug and start digging/debugging
just coming here saying "it doesn't work" is useless and doesn't help devs (and, above all, other users) to make a better product.

[SchulzStefan]

I did install it on clean VMs several times throughout the development process for testing, and it is currently working on my main machine.  I just built a clean test VM with 9.2 and installed it, though, and I'm seeing similar (though not identical) results to what you're reporting:

Code: [Select]

[root@sme92-test ~]# dehydrated -c
# INFO: Using main config file /etc/dehydrated/config
[root@sme92-test ~]#

My apologies.  As penance, I've reported the bug: https://bugs.contribs.org/show_bug.cgi?id=10300

If you (or @ShulzStefan) had indicated that the command did return some output (and better yet, what that output was), that would have saved some time in narrowing down the issue.

Dan Brown 2017-05-15 14:59:53 CEST

Thought I'd added this last night, but I don't see it.

Obviously the issue has something to do with the curl commands added to config.  So, rolled the VM back to where the certs hadn't been successfully issued yet.  Ran dehydrated -c, and it behaved as I reported--gave a message that it was using the default config file, then returned to the shell prompt. Changed the curl commands to remove the redirect (the "2>&1 > /dev/null" at the end of each line) so I could see what was happening.  Ran "dehydrated -c" again, and (unsurprisingly) the curl commands generated some ugly output, but issuing the certs succeeded without any further issues.

Rolled back to a pre-issuance state again.  Ran dehydrated -c, and it again behaved as reported.  Edited config to only remove the 2>&1 part of the redirect--this should send stdout to /dev/null, but stderr still to the console.  Ran "dehydrated -c" again, and once again a cert was issued without errors.

This isn't helping--no error output is being generated.

Rolled back again.  Ran dehydrated -c.  It behaved as reported.  Changed nothing, and ran dehydrated -c again.  It completed without issues.  This didn't work last night--repeated invocations of dehydrated -c had the same results as the first invocation.

Rolled back again.  Edited config (without running dehydrated -c first) to remove the 2>&1 from each of the curl commands.  Ran dehydrated -c, and it behaved as reported, with no further errors from curl.  Ran dehydrated -c again, and it completed without issues.

Opening port 80 was not enough, after opening 443, I reported in the bugtracker:

Here is what I tried so far:

I opened my firewall. @stefano: BTW it's not only port 80, for curl the https port also needs to be opened. And, another point - I have to allow in my LAN *any* to *any* which I do not really understand... Usually my firewall is configured with the last rule to deny everything what is not allowed. (Default deny LAN to any rule).

First of all I followed the advice from janet and changed the primary doamin to a registered domain. In my case from *.local to *.de. The *.de domain is a registered domain. Altering the cname I am able to reach over dyndns my server.

Secondly I re-installed the contrib after clearing all pre-installed fragments (of this contrib - smeserver-letsencrypt) and reboot with signal-event post.....

Then I altered the domain.txt to the *one* registered domain, I'd like to have a cert for email. There are a few more, but I don't want certs for them. "ftp.xxx.de mail.xxx.de www.xxx.de xxx.de"

Running dehydrated -c results in:

# INFO: Using main config file /etc/dehydrated/config
+ Generating account key...
+ Registering account key with ACME server...
Processing ftp.xxx.de with alternative names: mail.xxx.de www.xxx.de xxx.de
 + Signing domains...
 + Creating new directory /etc/dehydrated/certs/ftp.xxx.de ...
 + Generating private key...
 + Generating signing request...
 + Requesting challenge for ftp.xxx.de...
 + Requesting challenge for mail.xxx.de...
 + Requesting challenge for www.xxx.de...
 + Requesting challenge for xxx.de...
 + Responding to challenge for ftp.xxx.de...
 + Responding to challenge for mail.xxx.de...
 + Responding to challenge for www.xxx.de...
 + Challenge is valid!
 + Responding to challenge for xxx.de...
ERROR: Challenge is invalid! (returned: invalid) (result: {
  "type": "http-01",
  "status": "invalid",
  "error": {
    "type": "urn:acme:error:unauthorized",
    "detail": "Invalid response from http://xxx.de/.well-known/acme-challenge/BRZHc8cpXjpj7_gjYaiJmDnKJ_-QOPxv_sNic0SMXEw: \"\u003c!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\"\u003e\n\u003chtml\u003e\u003chead\u003e\n\u003ctitle\u003e404 Not Found\u003c/title\u003e\n\u003c/head\u003e\u003cbody\u003e\n\u003ch1\u003eNot Found\u003c/h1\u003e\n\u003cp\"",
    "status": 403
  },
  "uri": "https://acme-staging.api.letsencrypt.org/acme/challenge/-xYqKSs8_ndFiuQdpCaAR6OP3MfcZ_xD_pnKwL-rUj4/38745745",
  "token": "BRZHc8cpXjpj7_gjYaiJmDnKJ_-QOPxv_sNic0SMXEw",
  "keyAuthorization": "BRZHc8cpXjpj7_gjYaiJmDnKJ_-QOPxv_sNic0SMXEw.ajQV71Epgz6e1zvfNQd7npQs17GuYvYWjQmuqNxcBCc",
  "validationRecord": [
    {
      "url": "http://xxx.de/.well-known/acme-challenge/BRZHc8cpXjpj7_gjYaiJmDnKJ_-QOPxv_sNic0SMXEw",
      "hostname": "xxx.de",
      "port": "80",
      "addressesResolved": [
        "UUU.169.145.68",
        "UUUU:238:20a:202:1068::"
      ],
      "addressUsed": "UUUU:238:20a:202:1068::",
      "addressesTried": []
    }
  ]
})

Per default the firewall is blocking IP6.

Don't know why there's an invalid response?

In /etc/dehydrated the directory "accounts" and "certs" have been created. In "certs" is the directory "ftp.xxx.de", and in this dir are the files cert-1494796583.csr  cert-1494796583.pem  privkey-1494796583.pem.

That's it. No luck so far.

Server is a production server - no virtual box.

Before there was nothing to report, I'm sorry.

Nonsense, and I think you know better than that.  Some people will have problems with any software; the fact that there are people reporting problems in no way shows that the software or system is defective.  I'm reasonably active on the Let's Encrypt forum, and can confirm what Stefano is saying--the large majority of problem posts there (which themselves are a very small fraction of the millions of people who are using Let's Encrypt) are a matter of either (1) people configuring something wrong, or (2) Let's Encrypt itself working just fine, but then they don't know what to do with the cert once they receive it.

But none of that has anything to do with your case.  If you're unwilling to open port 80 of your SME server, things get more complicated for you, but you still have options.

• You can use the DNS challenge; I posted links on this up-thread
• You can use the built-in hook script functionality to copy the challenge files to an Internet-accessible location--this isn't documented very well at this time;
   I'll try to get something added to the wiki page shortly
• You may be able to use some sort of reverse proxy arrangement as JPP mentions
• You can obtain the cert on your firewall and deploy it from there to your SME box

Any of these are going to require some scripting and/or custom template fragments, but they're all options.

I'm sorry. I installed a contrib following the how-to. It didn't work as expected. As a matter of fact, it's not only the port 80 to open. If I knew that also port 443 has to be opened, I could have reported earlier the output which I gave in the bugtracker.

After opening my firewall (besides a modification of an internal deny rule) it still didn't work as expected. I got the hint to alter the domain.txt file. I didn't try this yet.

I reported that I changed my primary domain from *.local to a verfied one (*.de). I understood altering the cname of the hosted domain to dyndns (pointing as a result on the SME), will show letsencrypt that I have fully control over the server. Doing this brings a few other problems on the table. Of course depending on your firewall/network configuration. But for me, not so easy to manage. At least there's some work and re-configuration to do.

If millions of people are using letsencrypt without having problems, that's great for them. I didn't judge letsencrypt, dehydrated nor the contrib. And I am not going to do this.

Personally I think everybody should cool down. Every contrib is a great work and more then appreciated. It helps in any way.

My 2c

regards,
stefan

[DanB35]

You can use the built-in hook script functionality to copy the challenge files to an Internet-accessible location--this isn't documented very well at this time;
 I'll try to get something added to the wiki page shortly

I've added something--could probably use some clean-up, but it may help:
https://wiki.contribs.org/Letsencrypt#Obtaining_certificates_for_a_private_SME_Server