I've found out that the problem seems related to TLS.
If I setup NextCloud to connect via SMTP to the local IP-address of the SME-server, I get this in the SME log:
2025-04-19 10:10:01.249660500 16987 Accepted connection 0/40 from 192.168.1.179 / pc-00179.ibsgaarden.dk
2025-04-19 10:10:01.249784500 16987 Connection from pc-00179.ibsgaarden.dk [192.168.1.179]
2025-04-19 10:10:01.277469500 16987 (connect) relay: pass, octet match in relayclients (192.168.1.179)
2025-04-19 10:10:01.277637500 16987 220 sme10.ibsgaarden.dk ESMTP
2025-04-19 10:10:01.278468500 16987 dispatching EHLO myserver.ibsgaarden.dk
2025-04-19 10:10:01.279062500 16987 250-ibsgaarden.dk Hi pc-00179.ibsgaarden.dk [192.168.1.179]
2025-04-19 10:10:01.279086500 16987 250-PIPELINING
2025-04-19 10:10:01.279104500 16987 250-8BITMIME
2025-04-19 10:10:01.279122500 16987 250-SIZE 30000000
2025-04-19 10:10:01.279140500 16987 250 STARTTLS
2025-04-19 10:10:01.279803500 16987 dispatching STARTTLS
2025-04-19 10:10:01.279860500 16987 220 Go ahead with TLS
2025-04-19 10:10:01.304825500 16987 (unrecognized_command) tls: TLS setup returning
From the NextCloud server, if I try to connect to the mail-server with SSL/TLS I get this:
$ root@myserver:/home/holck/# openssl s_client -state -nbio -connect 192.168.1.11:25
SSL_connect:before SSL initialization
SSL_connect:SSLv3/TLS write client hello
SSL_connect:error in SSLv3/TLS write client hello
SSL_connect:error in error
4087FE8FA07F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:354:
CONNECTED(00000003)
Turned on non blocking io
write R BLOCK
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 5 bytes and written 297 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---