Koozali.org: home of the SME Server

Letsencrypt Dehydrated fails with "Challenge is invalid!"

Offline smeghead

  • *
  • 552
  • +0/-0
Letsencrypt Dehydrated fails with "Challenge is invalid!"
« on: December 22, 2021, 02:31:27 PM »
Howdy brainstrust

I have a server where the primary website is external to the SME & the host entry is configures as such in the server manager.

When I run the commands to implement Letsencrypt I get the "Challenge is invalid!" error.  When I check the external access to the SME Server using http (which is how dehydrated does it) all paths fail including the root path.

Server manager works fine using http or https from the LAN or WAN (https only using a nominated specific IP)

So, some detail:

The script I run based on the howto page

#! /bin/bash

set -x

clear

# Base settings

Internet_Domain=<Domain1>

config setprop letsencrypt ACCEPT_TERMS yes status test API 2
config setprop letsencrypt configure none

# Foreach of your domains you want SSL do the following
db domains setprop $Internet_Domain letsencryptSSLcert disabled

# Foreach of your hosts (subdomains) you want SSL do the following
db hosts setprop www.$Internet_Domain letsencryptSSLcert disabled
db hosts setprop wpad.$Internet_Domain letsencryptSSLcert disabled
db hosts setprop proxy.$Internet_Domain letsencryptSSLcert disabled
db hosts setprop ftp.$Internet_Domain letsencryptSSLcert disabled
db hosts setprop mail.$Internet_Domain letsencryptSSLcert enabled
db hosts setprop gateway.$Internet_Domain letsencryptSSLcert enabled

signal-event console-save

# Make sure Apache subfolder perms are correct for Dehydrated check
namei --modes /home/e-smith/files/ibays/Primary/html/.well-known/acme-challenge

chown root:root /home/e-smith/files/ibays/Primary
chmod 0755 /home/e-smith/files/ibays/Primary
chown admin:shared /home/e-smith/files/ibays/Primary/html
chmod 2750 /home/e-smith/files/ibays/Primary/html

dehydrated -c

The output I get when it runs

[root@gateway ~]# ./lets_encrypt_setup.sh
+ clear
+ Internet_Domain=<domain1>
+ config setprop letsencrypt ACCEPT_TERMS yes status test API 2
+ config setprop letsencrypt configure none

+ db domains setprop $Internet_Domain letsencryptSSLcert disabled

+ db hosts setprop www.$Internet_Domain letsencryptSSLcert disabled
+ db hosts setprop wpad.$Internet_Domain letsencryptSSLcert disabled
+ db hosts setprop proxy.$Internet_Domain letsencryptSSLcert disabled
+ db hosts setprop ftp.$Internet_Domain letsencryptSSLcert disabled
+ db hosts setprop mail.$Internet_Domain letsencryptSSLcert enabled
+ db hosts setprop gateway.$Internet_Domain letsencryptSSLcert enabled
+ signal-event console-save
+ namei --modes /home/e-smith/files/ibays/Primary/html/.well-known/acme-challenge
f: /home/e-smith/files/ibays/Primary/html/.well-known/acme-challenge
 dr-xr-xr-x /
 drwxr-xr-x home
 drwxr-xr-x e-smith
 drwxr-xr-x files
 drwxr-xr-x ibays
 drwxr-xr-x Primary
 drwxr-s--- html
 drwxrwsr-x .well-known
 drwxrwsr-x acme-challenge
+ chown root:root /home/e-smith/files/ibays/Primary
+ chmod 0755 /home/e-smith/files/ibays/Primary
+ chown admin:shared /home/e-smith/files/ibays/Primary/html
+ chmod 2750 /home/e-smith/files/ibays/Primary/html
+ dehydrated -c
# INFO: Using main config file /etc/dehydrated/config
Processing gateway.<domain1> with alternative names: mail.<domain1>
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting new certificate order from CA...
 + Received 2 authorizations URLs from the CA
 + Handling authorization for gateway.<domain1>
 + Handling authorization for mail.<domain1>
 + 2 pending challenge(s)
 + Deploying challenge tokens...
 + Responding to challenge for gateway.<domain1> authorization...
 + Cleaning challenge tokens...
 + Challenge validation has failed :(
ERROR: Challenge is invalid! (returned: invalid) (result: ["type"]      "http-01"
["status"]      "invalid"
["error","type"]        "urn:ietf:params:acme:error:unauthorized"
["error","detail"]      "Invalid response from http://gateway.<domain1>/.well-known/acme-challenge/LuEbZqOy0ZIPQleeTYuwoMzCW0oHOPQDJNAB6il-m8c [xx.xx.xx.xx]: \"\u003c!DOCTYPE HTML PUBLIC \\\"-//IETF//DTD HTML 2.0//EN\\\"\u003e\\n\u003chtml\u003e\u003chead\u003e\\n\u003ctitle\u003e403 Forbidden\u003c/title\u003e\\n\u003c/head\u003e\u003cbody\u003e\\n\u003ch1\u003eForbidden\u003c/h1\u003e\\n\u003cp\""
["error","status"]      403
["error"]       {"type":"urn:ietf:params:acme:error:unauthorized","detail":"Invalid response from http://gateway.<domain1>/.well-known/acme-challenge/LuEbZqOy0ZIPQleeTYuwoMzCW0oHOPQDJNAB6il-m8c [xx.xx.xx.xx]: \"\u003c!DOCTYPE HTML PUBLIC \\\"-//IETF//DTD HTML 2.0//EN\\\"\u003e\\n\u003chtml\u003e\u003chead\u003e\\n\u003ctitle\u003e403 Forbidden\u003c/title\u003e\\n\u003c/head\u003e\u003cbody\u003e\\n\u003ch1\u003eForbidden\u003c/h1\u003e\\n\u003cp\"","status":403}
["url"] "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/1234416688/BLIc7w"
["token"]       "LuEbZqOy0ZIPQleeTYuwoMzCW0oHOPQDJNAB6il-m8c"
["validationRecord",0,"url"]    "http://gateway.<domain1>/.well-known/acme-challenge/LuEbZqOy0ZIPQleeTYuwoMzCW0oHOPQDJNAB6il-m8c"
["validationRecord",0,"hostname"]       "gateway.<domain1>"
["validationRecord",0,"port"]   "80"
["validationRecord",0,"addressesResolved",0]    "xx.xx.xx.xx"
["validationRecord",0,"addressesResolved"]      ["xx.xx.xx.xx"]
["validationRecord",0,"addressUsed"]    "xx.xx.xx.xx"
["validationRecord",0]  {"url":"http://gateway.<domain1>/.well-known/acme-challenge/LuEbZqOy0ZIPQleeTYuwoMzCW0oHOPQDJNAB6il-m8c","hostname":"gateway.<domain1>","port":"80","addressesResolved":["xx.xx.xx.xx"],"addressUsed":"xx.xx.xx.xx"}
["validationRecord"]    [{"url":"http://gateway.<domain1>/.well-known/acme-challenge/LuEbZqOy0ZIPQleeTYuwoMzCW0oHOPQDJNAB6il-m8c","hostname":"gateway.<domain1>","port":"80","addressesResolved":["xx.xx.xx.xx"],"addressUsed":"xx.xx.xx.xx"}]
["validated"]   "2021-12-22T12:49:32Z")

So is this issue to do with the www redirect as I suspect or something else I'm missing?
..................

Offline Jean-Philippe Pialasse

  • *
  • 2,166
  • +8/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Letsencrypt Dehydrated fails with "Challenge is invalid!"
« Reply #1 on: December 22, 2021, 02:56:37 PM »
you should use directly the contrib instead of a homemade script.

from what i understand you use a proxypass to display the site from another server when accessed by outside.


the order of few directives like redirect, proxypass , rewrite is quite complexe.

there are chances that your token exploration is redirected to the proxy BEFORE other rules are playing. 

also i would eliminate fail2ban or geoip filtering.  dehydrated use aleatory server on the globe.

Offline smeghead

  • *
  • 552
  • +0/-0
Re: Letsencrypt Dehydrated fails with "Challenge is invalid!"
« Reply #2 on: December 22, 2021, 06:04:46 PM »
you should use directly the contrib instead of a homemade script.
My script is based on the info in the Howto (https://wiki.koozali.org/Letsencrypt), the rush job section shows:
Rush jobs

for the test (adjust the domains and hosts):

config setprop letsencrypt ACCEPT_TERMS yes status test API 2
#foreach of your domains you want SSL do the following
db domains setprop domain1.com letsencryptSSLcert enabled
#foreach of your hosts (subdomains) you want SSL do the following
db hosts setprop www.domain1.com letsencryptSSLcert enabled
signal-event console-save
dehydrated -c

Hence operates as per these instructions as modified, I have NOT gone off on my own & dreamed up something completely left field.  I prefer doing it this way as I can see what happens as it happens, often what is displayed on the console is more than what's logged.  The script method often allows a finer level of control.

from what i understand you use a proxypass to display the site from another server when accessed by outside.
.. dunno where you got that idea, didn't mention anything like that in my post.  I did say that the clients website is external so the www hostname is redirected to that external location, which was done via the server manager panel.

the order of few directives like redirect, proxypass , rewrite is quite complexe.

there are chances that your token exploration is redirected to the proxy BEFORE other rules are playing. 
.. the httpd.conf file does not have any proxypass or rewrite directive, related to the external website, that I can see.  The TinyDNS data file (/service/tinydns/root/data) has the redirected hostname and it's external IP address.  If there is a proxypass or rewrite rule somewhere else that is created when a host is pointed to an external server then please enlighten me.

also i would eliminate fail2ban or geoip filtering.  dehydrated use aleatory server on the globe.
Good point, I'll try that.

One other thing I've noticed is that when I run this process I don't get any files added to the acme-challenge folder, which I thought was a necessary part of the process.
..................

Offline Jean-Philippe Pialasse

  • *
  • 2,166
  • +8/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Letsencrypt Dehydrated fails with "Challenge is invalid!"
« Reply #3 on: December 22, 2021, 07:43:34 PM »
My script is based on the info in the Howto (https://wiki.koozali.org/Letsencrypt), the rush job section shows:
Rush jobs

then you should not play with ibays ownership and access rights.


.. dunno where you got that idea, didn't mention anything like that in my post.  I did say that the clients website is external so the www hostname is redirected to that external location, which was done via the server manager panel.

I assumed that from your very vague "I have a server where the primary website is external to the SME & the host entry is configures as such in the server manager."

you will need explain more in details what you mean there, probably using a schema....



.. the httpd.conf file does not have any proxypass or rewrite directive, related to the external website, that I can see.  The TinyDNS data file (/service/tinydns/root/data) has the redirected hostname and it's external IP address.  If there is a proxypass or rewrite rule somewhere else that is created when a host is pointed to an external server then please enlighten me.
please read the wiki searching for proxypass there are actually 2 ways to do it...
but still you first need to explain exactly what is the architecture, because from my perspective it seems you are trying something that is not possible. but still because I have not enought information to do something else than assuming.

HELP US TO HELP YOU...
 

Good point, I'll try that.
from memory the token must be accessible by 3 or 5 different test servers to be validated.. a few of the tests initiated can fail, but still if too many fail the validation fails.

One other thing I've noticed is that when I run this process I don't get any files added to the acme-challenge folder, which I thought was a necessary part of the process.
how are you checking and when ?
at the end of the process all is removed...
do you check using cli or your browser ?
probably (As I said earlier), a directive make them to be different depending on the perspective, and also could be different from your lan and from the internet ...