OpenBSD, relayd and acme-client: Unable to read TLS-certs for subdomains: "Empty reply from server"

Question

OpenBSD server with relayd/httpd/acme-client. Attempting to run https://mydomain.com (app server on port 32489), https://webmail.mydomain.com (48293) and https://forum.mydomain.com (28192). All three sites share the same TLS-cert. Regular HTTP is only for generating TLS-certs (acme-client).

So far https://mydomain.com works as expected, but the other two returns:

curl -vvv https://webmail.mydomain.com

* Rebuilt URL to: https://webmail.mydomain.com/
*   Trying XXX...
* TCP_NODELAY set
* Connected to webmail.mydomain.com (XXX) port 443 (#0)
* schannel: SSL/TLS connection with webmail.mydomain.com port 443 (step 1/3)
* schannel: checking server certificate revocation
* schannel: sending initial handshake data: sending 180 bytes...
* schannel: sent initial handshake data: sent 180 bytes
* schannel: SSL/TLS connection with webmail.mydomain.com port 443 (step 2/3)
* schannel: failed to receive handshake, need more data
* schannel: SSL/TLS connection with webmail.mydomain.com port 443 (step 2/3)
* schannel: encrypted data got 4096
* schannel: encrypted data buffer: offset 4096 length 4096
* schannel: encrypted data length: 4032
* schannel: encrypted data buffer: offset 4032 length 4096
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with webmail.mydomain.com port 443 (step 2/3)
* schannel: encrypted data got 907
* schannel: encrypted data buffer: offset 4939 length 5056
* schannel: sending next handshake data: sending 93 bytes...
* schannel: SSL/TLS connection with webmail.mydomain.com port 443 (step 2/3)
* schannel: encrypted data got 51
* schannel: encrypted data buffer: offset 51 length 5056
* schannel: SSL/TLS handshake complete
* schannel: SSL/TLS connection with webmail.mydomain.com port 443 (step 3/3)
* schannel: stored credential handle in session cache
> GET / HTTP/1.1
> Host: webmail.mydomain.com
> User-Agent: curl/7.55.1
> Accept: */*
>
* schannel: client wants to read 102400 bytes
* schannel: encdata_buffer resized 103424
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: encrypted data got 31
* schannel: encrypted data buffer: offset 31 length 103424
* schannel: server closed the connection
* schannel: schannel_recv cleanup
* Empty reply from server
* Connection #0 to host webmail.mydomain.com left intact
curl: (52) Empty reply from server

acme-client.conf

authority letsencrypt {
  api url "https://acme-v02.api.letsencrypt.org/directory"
  account key "/etc/ssl/private/letsencrypt.key"
}

domain mydomain.com {
  alternative names { www.mydomain.com webmail.mydomain.com forum.mydomain.com }
  domain key "/etc/ssl/private/mydomain.com.key"
  domain full chain certificate "/etc/ssl/mydomain.com.crt"
  sign with letsencrypt
}

httpd.conf

ext_if="vio0"

types {
  include "/usr/share/misc/mime.types"
}

server "mydomain.com" {
  alias "www.mydomain.com"
  listen on $ext_if port 80
  location "/.well-known/acme-challenge/*" {
    root "/acme"
    request strip 2
  }
  location "*" {
    block return 301 "https://mydomain.com$REQUEST_URI"
  }
}

server "webmail.mydomain.com" {
  listen on $ext_if port 80
  location "/.well-known/acme-challenge/*" {
    root "/acme"
    request strip 2
  }
  location "*" {
    block return 301 "https://webmail.mydomain.com$REQUEST_URI"
  }
}

server "forum.mydomain.com" {
  listen on $ext_if port 80
  location "/.well-known/acme-challenge/*" {
    root "/acme"
    request strip 2
  }
  location "*" {
    block return 301 "https://forum.mydomain.com$REQUEST_URI"
  }
}

relayd.conf

ip="XXX"

table <cms> { 127.0.0.1 }
cms_port="32489"

table <webmail> { 127.0.0.1 }
webmail_port="48293"

table <forum> { 127.0.0.1 }
forum_port="28192"

table <httpd> { 127.0.0.1 }
httpd_port="80"

log connection errors

http protocol "http" {
  match request header set "Connection" value "close"
  match response header remove "Server"
}

relay "http_relay" {
  listen on $ip port http
  protocol "http"
  forward to <httpd> port $httpd_port
}

http protocol "https" {
  match header log "Host"
  match header log "X-Forwarded-For"
  match header log "User-Agent"
  match header log "Referer"
  match url log

  match header set "X-Forwarded-For" value "$REMOTE_ADDR"
  match header set "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"
  match header set "Keep-Alive" value "$TIMEOUT"

  # Best practice security headers
  match response header remove "Server"
  match response header append "Strict-Transport-Security" value "max-age=31536000; includeSubDomains"
  match response header append "X-Frame-Options" value SAMEORIGIN
  match response header append "X-XSS-Protection" value "1; mode=block"
  match response header append "X-Content-Type-Options" value nosniff
  match response header append "Referrer-Policy" value strict-origin
  match response header append "Feature-Policy" value "accelerometer 'none'; camera 'none'; geolocation 'none'; gyroscope 'none'; magnetometer 'none'; microphone 'none'; payment 'none'; usb 'none'"

  pass request header "Host" value "mydomain.com" forward to <cms>
  pass request header "Host" value "www.mydomain.com" forward to <cms>
  pass request header "Host" value "webmail.mydomain.com" forward to <webmail>
  pass request header "Host" value "forum.mydomain.com" forward to <forum>
  tls keypair "mydomain.com"
}

relay "https_relay" {
  listen on $ip port https tls

  protocol "https"

  forward to <cms> port $cms_port
  forward to <webmail> port $webmail_port
  forward to <forum> port $forum_port
}

There is no problem with the TLS handshake and thus not with the certificates. It can be seen that the client is sending the HTTP request, which is only done after a successful TLS handshake. Thus there is some problem which can not seen from the information provided, like the backend server for webmail not running. You better check logs etc on the server side for this. — Steffen Ullrich, Nov 19 '22 at 06:45
Thanks a lot for helping me narrow it down @SteffenUllrich, I really appreciate it! — Mark Boulder, Nov 19 '22 at 06:58

Zé Loff · Answer 1 · 2022-11-20T11:30:13.637

relayd doesn't handle TLS SNI yet, so you need a tls keypair entry for each forwarded domain. This means you need individual .key and .crt files for each of the domains you serve. This can easily be handled with symbolic links, e.g.:

# cd /etc/ssl
# ln -s mydomain.com.crt www.mydomain.com.crt
# cd /etc/ssl/private
# ln -s mydomain.com.key www.mydomain.com.key

and then adding

tls keypair "www.mydomain.com"

to the http protocol "https" {...} section of relayd.conf. Running

# relayd -n

will tell you if everything is OK.

Note that you will need to reload relayd after the certificates are updated. This is usually handled for httpd by a cron job. The man page for acme-client states

A cron(8) job can renew the certificate as necessary.  On renewal,
httpd(8) is reloaded:

       ~ * * * * acme-client example.com && rcctl reload httpd

so in this case you'd need to add relayd to the end of the line, to reload both httpd and relayd. If you use some other method or script, you'll need to adapt it appropriately.

My setup is identical to the one you outlined. Also it seems relayd supports SNI: https://findelabs.com/post/relayd-using-sni-keypairs/ — Mark Boulder, Nov 24 '22 at 05:03
I've seen that post. My point is, you still need to specify a keypair for each domain. This means that either you (1) have separate entries on `acme-client.conf` and generate one certificate for each site, or (2) generate a single certificate and symlink it so that `relayd` can find them. But you MUST have multiple keypairs, which you don't have in your config. — Zé Loff, Nov 24 '22 at 13:49

OpenBSD, relayd and acme-client: Unable to read TLS-certs for subdomains: "Empty reply from server"

1 Answers1