OpenBSD manual page server

Creating a private key and certificate for an MTA

This example assumes you are creating your own self-signed certificates for use with smtpd(8) and STARTTLS. If you have an existing private key and you simply wish to generate a new certificate (for example, if your old certificate has expired), see the section entitled Creating a certificate with an existing private key.

For the purposes of this example the certificates will be stored in /etc/ssl, though it is possible to use a different directory if needed.

Next, you must generate an RSA private key:

This would generate a 4096-bit RSA key stored in the file mail.example.com.key.

Once you have generated the RSA key, you can generate a certificate from it using the command:

# openssl req -x509 -new -key /etc/ssl/private/mail.example.com.key \
  -out /etc/ssl/mail.example.com.crt -days 365

You may adjust the lifetime of the certificate via the -days parameter (one year in this example).

You can verify that the newly-generated certificate has correct information with the following command:

If you don't intend to use TLS for authentication (and if you are using self-signed certificates you probably don't) you can simply link your new certificate to CAcert.pem.

If, on the other hand, you intend to use TLS for authentication you should install your certificate authority bundle as /etc/ssl/CAcert.pem.

Because the private key files are unencrypted, MTAs can be picky about using tight permissions on those files. The certificate directory and the files therein should be readable and writable only by the owner (root). A simple way to ensure this is to run the following:

Creating a certificate with an existing private key

This example assumes you already have an existing private key, /etc/ssl/private/mail.example.com.key. You can generate a new certificate based on this key using the command:

# openssl req -x509 -new -key /etc/ssl/private/mail.example.com.key \
  -out /etc/ssl/mail.example.com.crt -days 365
# chmod 600 /etc/ssl/mycert.pem

You may adjust the lifetime of the certificate via the -days parameter (one year in this example).

After having installed the certificates the mail server needs to be configured to accept TLS sessions and use the key and certificate. For smtpd(8), it's as simple as adding pki configuration to smtpd.conf(5):

pki mail.example.com certificate "/etc/ssl/mail.example.com.crt"
pki mail.example.com key "/etc/ssl/private/mail.example.com.key"

listen on [...] tls pki mail.example.com auth

After restarting the mail server, a new option should be presented for ESMTP transactions, STARTTLS. You can test this by connecting to the local host and issuing the “EHLO” command.

# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 localhost ESMTP OpenSMTPD
EHLO localhost

After typing EHLO localhost you should receive something like the following back.

250-localhost Hello localhost [127.0.0.1], pleased to meet you
250-8BITMIME
250-ENHANCEDSTATUSCODES
250-SIZE 36700160
250-DSN
250-STARTTLS
250 HELP

You should see “STARTTLS” listed along with the other options. If so, congratulations, the MTA will now use TLS to encrypt your mail traffic when the remote server supports it. If not, check /var/log/maillog to see whether the MTA has reported any security problems or other errors.

Uses for TLS equipped MTAs

The most obvious use of a cryptographically enabled MTA is for confidentiality of the electronic mail transaction and the integrity checking provided by the cipher suite. All traffic between the two mail servers is encrypted, including the sender and recipient addresses. TLS also allows for authentication of either or both systems in the transaction.

One use of public key cryptography is for strong authentication. We can use this authentication to selectively relay clients, including other mail servers and mobile clients like laptops. However, there have been some problems getting some mail clients to work using certificate-based authentication. Note that your clients will have to generate certificates and have them signed (for trust validation) by a CA (certificate authority) you also trust, if you configure your server to do client certificate checking. Two new entries are available for TLS options:

VERIFY can also accept the argument for {cipher_bits}. Here are a few example entries that illustrate these features, and the role based granularity as well:

Require strong (256-bit) encryption for communication with this server:

For a TLS client, require verification and a minimum of 128-bit encryption:

Much more complicated access maps are possible, and error conditions (such as permanent or temporary, PERM+ or TEMP+) can be set on the basis of various criteria. This allows you fine-grained control over the types of connections you can allow.

Note that it is unwise to force all SMTP clients to use TLS, as it is not yet widespread. The RFC document notes that publicly referenced SMTP servers, such as the MX servers for a domain, must not refuse non-TLS connections. However, restricted access SMTP servers, such as those for a corporate intranet, can use TLS as an access control mechanism.