Topic: Help please, certificate nightmare

[groutley]

Hi, appreciate some help from the wise here,
I run sme10 uptodate with latest updates.
On Friday 17th, my cacert.org certificates expired, and users can no longer access their emails.
I went to cacert.org to renew the certs, however it seems they have major problems and the functionality is not there to renew or create new certs.
So I started looking for alternatives, I decided to install phpki-ng latest .
Install no problem at all,
I created a new cert, hoping that would solve everything,
However I suspect there are more steps, than just generating a certificate.
My research found this.. https://forums.koozali.org/index.php/topic,51297.msg260373.html#msg260373
With it being a little dated, the concept I assumed to be correct,
So I downloaded the key.pem and crt.pem files using the phpki contib.
Then copied the text of these files into the certificate manager contrib panel and saved.
Noting this appeared to creat the appropriate crt and key files in /home/e-smith/ssl.crt and ssl.key directories.
Continued with the ‘ db configuration setprop modSSL’ command as detailed in the referenced forum post.
All ran no error, the httpd -t returned no error,
So proceeded with the service restarts,
2 of them did not work ‘not found’ error, assuming due to the date of the forum entry things have changed a lot, so ran ‘signal-event post-upgrade; signal-event reboot’
My server did not come back, on checking found it sitting there wanting a password entered,
Never had this happen before, I assumed dues to the password set on the phpki certificate I created, it wanted that password, but after entering that multiple times, tried the root password, and the boot continued to logon prompt, and sever became pingable.
But now I cannot access the server-manager page or the /phpki/ca pages
I can login to ssh fortunately!
I am assuming if I run ‘signal-event certificate-revert’ I will regain web access,
 However I simply am not moving forward here..
From previously looking, at /webmail it seems the server is no longer receiving emails either!

I am hoping for some guidance on how I cleanup my mess and get email functioning again.
This server is really only used for email, and some ibay file storage over samba, but the email is the important bit.

Looking forward to your advise
  Thank you
 Glen

[ReetP]

There is a method to reset your servers self-signed certificate as well but I can't renember how.

Have a search here or on the wiki. Someone else may post it. That will get you restarted.

Then why not use letsencrypt/dehydrated, at least in the short term?

smeserver-letsencrypt

https://wiki.koozali.org/Letsencrypt

[Jean-Philippe Pialasse]

as told by John go for lets encrypt, and i bet you will never go back !
Only exception would be if you have some insurances needs behind your cert.

phpki whil it could be used for that is not the best choice as it would not be better than simply use the self signed certificate of SME.  To use it you should just delete the modSSL property pointing to your old cert.

[groutley]

Thank you for your suggestions.

I had avoided letsencrypt, as I run it on my Home Assistant, and assumed I would end up with port forwarding issues pointing to the wrong system.
But, with your suggestions, I figured push forward with it and work out the issues as I go.

It is installed,
  but when I run the tests

Code: [Select]

dehydrated -cI get...

Code: [Select]

  "type": "urn:ietf:params:acme:error:malformed",
  "detail": "Error creating new order :: Order cannot contain more than 100 DNS names",
  "status": 400
and before that it lists all of my DNS entries for every device in my house.

I set

Code: [Select]

config setprop letsencrypt configure domainsbut it still 'Processes' every DNS entry.

Any suggestions on why / how I stop it doing that?
I used to use this SME server as my DNS for the house,
however since splitting the Network into different VLANs, I now have the UniFi Router do that.
Should I delete all the Entries in SME from the legacy DNS days?

[Stefano]

Hi Glen

first of all, another HA user here

IMVHO you don't need a letsencrypt cert for each device

[groutley]

Hi Stefano,
 Thank you for your comment, and great to see another SME and HA user

When you say I don’t need a cert for each device, are you suggesting I copy the one from HA to SME?
Only concern I have with that is I use different domains.. duckdns for HA and a dyndns $$ domain for SME / email.

[Stefano]

Hi Stefano,
 Thank you for your comment, and great to see another SME and HA user

When you say I don’t need a cert for apeach device, are you suggesting I copy the one from HA to SME?
Only concern I have with that is I use different domains.. duckdns for HA and a dyndns $$ domain for SME / email.

you'd tell us more about your setup; I mean, I guess you don't have all your devices exposed to wan

in any case, I'd use SME as DNS, both for local/internal access (something like *.home.lan) and for external.

Alternatively (but keep in mind I'm not so experienced with PKI) you's use PKI for "local" devices' certificate and letsencrypt for public.
Hope you get what I mean

[groutley]

No, I don’t have all devices exposed to WAN
I have a VLAN for IOT devices and another VLAN for general user / internet access
All running on a Unifi network,
Due to the separate VLANs I found I couldn’t use the SME for dns any more and turned off that functionality and rely on the Unifi network router to be dns for both VLANs.

Earlier I posted I was failing with ‘dehydrated -c’ due to too many dns entries,
I went ahead and deleted them all (other than ‘self’ entries) on SME..
 Now dehydrated -c gets further..
But I am now getting the dreaded ‘ Invalid response / 403’ issue
Yet ‘letsdebug’ shows all is OK..

I am not winning ;-/

[Stefano]

ok, let's start posting some info about your config and some logs

[ReetP]

You have probably configured Letsencrypt to use EVERY domain and EVERY host.

You need to specify JUST the hosts and domain/s that you require.

https://wiki.koozali.org/Letsencrypt#Step_by_step_configuration

You can obtain a certificate for either of the following: all domains, all hostnames, or all domains AND hostnames.

Only set one of the following.

config setprop letsencrypt configure domains
config setprop letsencrypt configure hosts
config setprop letsencrypt configure all

To use individually enabled hosts or domains leave the default none.

config setprop letsencrypt configure none

So set:

Code: [Select]

config setprop letsencrypt configure none
and then

Per host:

Code: [Select]

db hosts setprop $HOSTNAME letsencryptSSLcert enabled
Per domain

Code: [Select]

db domains setprop $DOMAIN letsencryptSSLcert enabled
Make sure you run test mode first!!

https://wiki.koozali.org/Letsencrypt#Enable_test_mode

When you are happy then:

https://wiki.koozali.org/Letsencrypt#Enable_Production_Mode

[groutley]

ok, let's start posting some info about your config and some logs

Thank you all for your patience and assistance, I just cannot get my head around how this works, (should work).
 I have been following the step by step process and definitely been in test mode.

Following ReetP instructions….

I previosly was attempting to create the cert with my $DOMAIN,
But after reading https://forums.koozali.org/index.php/topic,52028.msg266631.html#msg266631
I decided to just go with ‘www.xxxxx.homeip.net’
(I will have to reconfigure email clients pointing to mail.xxxxx.homeip.net, but if it is going to work….)
So I changed the db domains to be disabled… I hope that would be the correct thing todo?

Code: [Select]


************ Welcome to SME Server 10.1 *

[root@l1nuxsvr ~]# config setprop letsencrypt configure none
[root@l1nuxsvr ~]# db hosts setprop www.xxxxx.homeip.net letsencryptSSLcert enabled
[root@l1nuxsvr ~]# db domains setprop xxxxx.homeip.net letsencryptSSLcert disabled
[root@l1nuxsvr ~]# config setprop letsencrypt status enabled
[root@l1nuxsvr ~]# signal-event console-save
[root@l1nuxsvr ~]# dehydrated -c
# INFO: Using main config file /etc/dehydrated/config
+ Generating account key...
+ Registering account key with ACME server...
+ Fetching account URL...
Processing ftp.xxxxx.homeip.net with alternative names: mail.xxxxx.homeip.net smtp.xxxxx.homeip.net www.xxxxx.homeip.net
 + Creating new directory /etc/dehydrated/certs/ftp.xxxxx.homeip.net ...
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting new certificate order from CA...
 + Received 4 authorizations URLs from the CA
 + Handling authorization for ftp.xxxxx.homeip.net
 + Handling authorization for mail.xxxxx.homeip.net
 + Handling authorization for smtp.xxxxx.homeip.net
 + Handling authorization for www.xxxxx.homeip.net
 + 4 pending challenge(s)
 + Deploying challenge tokens...
 + Responding to challenge for ftp.xxxxx.homeip.net authorization...
 + Cleaning challenge tokens...
 + Challenge validation has failed :(
ERROR: Challenge is invalid! (returned: invalid) (result: ["type"]      "http-01"
["status"]      "invalid"
["error","type"]        "urn:ietf:params:acme:error:unauthorized"
["error","detail"]      "1.1.5.19: Invalid response from http://ftp.xxxxx.homeip.net/.well-known/acme-challenge/vDOl6pLFkd2P5Pw37_3FxxnkGY7M_HtCPRECaERXioo: 403"
["error","status"]      403
["error"]       {"type":"urn:ietf:params:acme:error:unauthorized","detail":"1.1.5.19: Invalid response from http://ftp.xxxxx.homeip.net/.well-known/acme-challenge/vDOl6pLFkd2P5Pw37_3FxxnkGY7M_HtCPRECaERXioo: 403","status":403}
["url"] "https://acme-v02.api.letsencrypt.org/acme/chall-v3/285819578896/h2X95g"
["token"]       "vDOl6pLFkd2P5Pw37_3FxxnkGY7M_HtCPRECaERXioo"
["validationRecord",0,"url"]    "http://ftp.xxxxx.homeip.net/.well-known/acme-challenge/vDOl6pLFkd2P5Pw37_3FxxnkGY7M_HtCPRECaERXioo"
["validationRecord",0,"hostname"]       "ftp.xxxxx.homeip.net"
["validationRecord",0,"port"]   "80"
["validationRecord",0,"addressesResolved",0]    "1.1.5.19"
["validationRecord",0,"addressesResolved"]      ["1.1.5.19"]
["validationRecord",0,"addressUsed"]    "1.1.5.19"
["validationRecord",0]  {"url":"http://ftp.xxxxx.homeip.net/.well-known/acme-challenge/vDOl6pLFkd2P5Pw37_3FxxnkGY7M_HtCPRECaERXioo","hostname":"ftp.xxxxx.homeip.net","port":"80","addressesResolved":["1.1.5.19"],"addressUsed":"1.1.5.19"}
["validationRecord"]    [{"url":"http://ftp.xxxxx.homeip.net/.well-known/acme-challenge/vDOl6pLFkd2P5Pw37_3FxxnkGY7M_HtCPRECaERXioo","hostname":"ftp.xxxxxx.homeip.net","port":"80","addressesResolved":["1.1.5.19"],"addressUsed":"1.1.5.19"}]
["validated"]   "2023-11-20T21:14:53Z")
[root@l1nuxsvr ~]#

Note I redacted the IP address and domain name, but the IP is correctly resoving to my public address.

So it is still picking up the ‘self’ entries in SME hostnames.. should I delete those entries also?

[ReetP]

Bit hard to tell when stuff is obfuscated but this gives a clue:

["error","detail"]      "1.1.5.19: Invalid response from http://ftp.xxxxx.homeip.net/.well-known/acme-challenge/vDOl6pLFkd2P5Pw37_3FxxnkGY7M_HtCPRECaERXioo: 403"

["error"]       {"type":"urn:ietf:params:acme:error:unauthorized","detail":"1.1.5.19: Invalid response from http://ftp.xxxxx.homeip.net/.well-known/acme-challenge/vDOl6pLFkd2P5Pw37_3FxxnkGY7M_HtCPRECaERXioo: 403","status":403}

Are you on a sub domain or something odd?

Creating new directory /etc/dehydrated/certs/ftp.xxxxx.homeip.net ...

I'd expect it to say this:

xxxxx.homeip.net

Not:

ftp.xxxxx.homeip.net

I'd start with:

configure none

Now individually configure JUST the following. Make sure all other domains & hosts are disabled.

Domain
xxxxx.homeip.netetsencrypt letsencryptSSLcert enabled

Host
www.xxxxx.homeip.net letsencryptSSLcert enabled

console-save then check

Code: [Select]

cat/etc/dehydrated/domains.txt
It should ONLY have the one domain and one host as above.

Make sure you can access the directory with a browser:

http://xxxxx.homeip.net/.well-known/acme-challenge

And

http://www.xxxxx.homeip.net/.well-known/acme-challenge

Now run test mode.

Beyond that we need to see some actual detail:

db domains show
db hosts show

(Sorry I've ommitted full commands but am on mobile. Check with wiki)

[Jean-Philippe Pialasse]

grep www /etc/group
probably hit by bug https://bugs.koozali.org/show_bug.cgi?id=12146

[groutley]

grep www /etc/group
probably hit by bug https://bugs.koozali.org/show_bug.cgi?id=12146

[root@l1nuxsvr ~]# grep www /etc/group
shared:x:500:admin,administrator,dani,groutley,jo,john,jowork,matt,mattorrents,music,public,sofia,torrents,www,zenphoto
www:x:102:admin,apache,www
thefam:x:5003:admin,dani,groutley,jo,matt,www
routley:x:5004:admin,dani,groutley,jo,matt,www
kids:x:5005:admin,dani,matt,www
parents:x:5014:admin,groutley,jo,www
mattonly:x:5021:admin,matt,mattorrents,www
danir:x:5024:admin,dani,groutley,jo,www
mattr:x:5025:admin,groutley,jo,matt,www

Not sure I follow the Bug to understand the concern.

[groutley]

Bit hard to tell when stuff is obfuscated

Sorry about that, I assumed best for privacy..  but I will paste complete outputs now.

Are you on a sub domain or something odd?

No, not that I am aware of,  my SME is directly cabled to the Router to the Internet, and is using dyndns plugin to refresh the DNS entry
for the domain 'routley.homeip.net'

cat /etc/dehydrated/domains.txt

ftp.routley.homeip.net l1nuxsvr.routley.homeip.net mail.routley.homeip.net proxy.routley.homeip.net wpad.routley.homeip.net www.routley.homeip.net
[root@l1nuxsvr ~]# config setprop letsencrypt configure none
[root@l1nuxsvr ~]# cat /etc/dehydrated/domains.txt
ftp.routley.homeip.net l1nuxsvr.routley.homeip.net mail.routley.homeip.net proxy.routley.homeip.net wpad.routley.homeip.net www.routley.homeip.net

db domains show

routley.homeip.net=domain
    Content=Primary
    Description=internet
    Nameservers=localhost
    Removable=no
    SystemPrimaryDomain=yes
    letsencryptSSLcert=disabled

db hosts show

ftp.routley.homeip.net=host
    ExternalIP=
    HostType=Self
    InternalIP=
    MACAddress=
l1nuxsvr.routley.homeip.net=host
    ExternalIP=
    HostType=Self
    InternalIP=
    MACAddress=
    ReverseDNS=yes
    static=yes
mail.routley.homeip.net=host
    ExternalIP=
    HostType=Self
    InternalIP=
    MACAddress=
proxy.routley.homeip.net=host
    ExternalIP=
    HostType=Self
    InternalIP=
    MACAddress=
wpad.routley.homeip.net=host
    ExternalIP=
    HostType=Self
    InternalIP=
    MACAddress=
www.routley.homeip.net=host
    ExternalIP=
    HostType=Self
    InternalIP=
    MACAddress=
    letsencryptSSLcert=enabled

config setprop letsencrypt configure none
signal-event console-save

cat /etc/dehydrated/domains.txt

www.routley.homeip.net

Looking a bit better? only the www host?

Proceed with the setup per your advice.....

[root@l1nuxsvr ~]# db domains setprop routley.homeip.net letsencryptSSLcert enabled
[root@l1nuxsvr ~]# db hosts setprop www.routley.homeip.net letsencryptSSLcert enabled
[root@l1nuxsvr ~]# signal-event console-save
[root@l1nuxsvr ~]# cat /etc/dehydrated/domains.txt
routley.homeip.net www.routley.homeip.net

Looks good, as you suggest, it only has one host and one domain

However:
http://routley.homeip.net/.well-known/acme-challenge

gives:
Forbidden

You don't have permission to access /.well-known/acme-challenge on this server.

http://www.routley.homeip.net/.well-known/acme-challenge
also gives the same 'Forbidden'

Also on the local network, http://192.168.37.251/.well-known/acme-challenge
Gives the same Forbidden, so it is the server, not the network access to it.

So no point in proceeding to Test…
 So why is it not serving this url?

Both Port 80 and 443 are port forwarded on my router to the respective ports on destination IP '192.168.37.1' which is the SME Server IP address. (these ports are normally portforwarded to my Home Assistant Server, but not for the time being while I try to get this working).


/var/log/httpd/access_log shows:

routley.homeip.net 192.168.38.49 - - [21/Nov/2023:18:44:39 +1100] "GET /.well-known/acme-challenge HTTP/1.1" 403 228 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15"
routley.homeip.net 192.168.38.49 - - [21/Nov/2023:18:44:40 +1100] "GET /favicon.ico HTTP/1.1" 403 213 "http://192.168.37.251/.well-known/acme-challenge" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15"


Permissions:
root@l1nuxsvr ~]# cd /home/e-smith/files/ibays/Primary/
[root@l1nuxsvr Primary]# ls -la
total 20
drwxr-xr-x 5 root  root   4096 Jan  3  2013 .
drwxr-xr-x 7 root  root   4096 Oct  1  2020 ..
drwxr-s--- 2 admin shared 4096 Jan  3  2013 cgi-bin
drwxr-s--- 5 admin shared 4096 May 26  2013 files
drwxr-s--- 3 admin shared 4096 Sep 21  2020 html
[root@l1nuxsvr Primary]# ls -la html/
total 16
drwxr-s--- 3 admin  shared 4096 Sep 21  2020 .
drwxr-xr-x 5 root   root   4096 Jan  3  2013 ..
-rw-r----- 1 admin  shared  202 Nov 21  2005 index.htm
drwxrwsr-x 3 apache shared 4096 Sep 21  2020 .well-known
[root@l1nuxsvr Primary]# cd html/.well-known/
[root@l1nuxsvr .well-known]# ls -la
total 12
drwxrwsr-x 3 apache shared 4096 Sep 21  2020 .
drwxr-s--- 3 admin  shared 4096 Sep 21  2020 ..
drwxrwsr-x 2 apache shared 4096 Nov 21 18:19 acme-challenge
[root@l1nuxsvr .well-known]# cd acme-challenge/
[root@l1nuxsvr acme-challenge]# ls -la
total 8
drwxrwsr-x 2 apache shared 4096 Nov 21 18:19 .
drwxrwsr-x 3 apache shared 4096 Sep 21  2020 ..
[root@l1nuxsvr acme-challenge]#