Postfix sends no SNI, and shared hosting answers with the wrong certificate

Relaying through a cPanel mail server with strict TLS fails with "hostname mismatch" even though the certificate is valid. Postfix does not send SNI by default; smtp_tls_servername = nexthop fixes it.

I set up a Postfix relay in the cluster so that alerts could leave by mail. It authenticates to the mail server of my shared hosting account on port 587 with STARTTLS, and I set smtp_tls_security_level = secure, because a relay that sends a password should verify who it’s talking to.

The first test mail sat in the queue as deferred:

Server certificate not verified

The certificate on that server is valid, so this made no sense at first. Turning up the TLS logging with smtp_tls_loglevel = 1 showed the real reason:

hostname mismatch

Why the certificate doesn’t match

A shared hosting server carries the certificates of many customer domains. It chooses which one to present from the name the client sends in the TLS handshake, the SNI extension. If the client sends no name, the server falls back to its own default certificate, which covers the hosting node (something like cpNNN.<provider>.net) and none of the customer domains.

Postfix doesn’t send SNI unless you tell it to. So it got the node’s default certificate, compared it to mail.<your-domain>, and rightly refused.

The fix is one line:

smtp_tls_servername = nexthop

That makes Postfix send the name of the host it is connecting to. The server answers with the matching certificate, verification passes, and the mail went out.

Why openssl didn’t show the problem

I’d checked the certificate with openssl s_client beforehand and it looked fine. That’s because s_client does send SNI by default. To see what Postfix sees, turn it off:

openssl s_client -connect mail.<your-domain>:587 -starttls smtp -noservername </dev/null \
  | openssl x509 -noout -subject -ext subjectAltName

With -noservername you get the node’s default certificate, and the mismatch becomes obvious. It’s the same trap as an FTPS certificate mismatch, just reached a different way: there the client dials the wrong name, here it dials the right name but never says it out loud.

Two smaller things from the same relay

The container image I used (boky/postfix) enforces SMTP protocol synchronisation. A client that sends before the server’s greeting gets 554 5.5.0 SMTP protocol synchronization and is dropped. That bit my first scripted test with nc, which fired all the commands at once. Pause between the SMTP commands, or use a real client.

The pod network in my cluster has no IPv6 route, so every delivery first tried the AAAA record and waited for it to fail. inet_protocols = ipv4 stops that.

And check the hostname you’ve been given before debugging anything else. The one I started with didn’t exist at all. The mail server’s real name was a different subdomain of the same domain.