One-Way TLS Authentication

In one-way TLS, only the client verifies the server's identity. The server presents its certificate, the client validates it, and a secure session is established without the server checking the client's certificate. This is typical for public websites.

The handshake proceeds as follows:

1. The client sends supported TLS versions, cipher suites, and a random number.
2. The server responds with its chosen TLS version, cipher suite, random number, and its certificate containing the public key.
3. The client validates the server certificate by checking:
   • Expiration date.
   • Trustworthiness of the issuing CA.
   • Whether the public key can verify the certificate's digital signature.
   • Whether the certificate's Common Name or Subject Alternative Name matches the server's hostname.
4. If validation succeeds, the client generates a pre-master secret, encrypts it with the server's public key, and sends it.
5. The server decrypts the pre-master secret with its private key. Both parties then derive the session keys used for symmetric encryption.

Generating the Certificate Authority

First, create a self-signed root CA that will issue all other certificates. Protect the private key with a passphrase.

# Create an AES-256 encrypted private key for the CA
openssl genrsa -aes256 -out secure-ca.key 4096

# Generate the root CA certificate valid for 10 years
openssl req -new -x509 -days 3650 -sha256 -extensions v3_ca -key secure-ca.key \
  -out secure-ca.crt \
  -subj "/C=CN/ST=shanghai/L=shanghai/O=example/OU=eng/CN=Root CA"

Issuing a Server Certificate

The server needs a private key and a certificate signed by our internal CA.

# 1. Generate private key for the server
openssl genrsa -out app-server.key 2048

# 2. Create a certificate signing request (CSR)
openssl req -new -key app-server.key -out app-server.csr \
  -subj "/C=CN/ST=shanghai/L=shanghai/O=example/OU=eng/CN=server.example.com"

# 3. Sign the CSR with the CA. A serial file tracks issued certificates.
echo "01" > secure-ca.srl
openssl x509 -req -days 3650 -sha256 \
  -CA secure-ca.crt -CAkey secure-ca.key \
  -in app-server.csr -out app-server.crt

# 4. (Optional) Inspect the issued certificate
openssl x509 -in app-server.crt -text -noout

Nginx Configuration

Instruct Nginx to use the server certificate and key for HTTPS connecsions.

server {
    listen 443 ssl;
    server_name server.example.com;

    ssl_certificate     /etc/ssl/certs/app-server.crt;
    ssl_certificate_key /etc/ssl/private/app-server.key;

    # ... other directives
}

Mutual TLS Authentication

Mutual TLS (mTLS) requires both the client and the server to present and validate certificates, establishing a high-trust identity model. The server must trust the client's certificate, which is issued by a CA known to the server.

Creating a Client Certificate

Generate a certificate for the client using the same internal CA.

# 1. Create a private key
openssl genrsa -out app-client.key 2048

# 2. Generate a CSR
openssl req -new -key app-client.key -out app-client.csr \
  -subj "/C=CN/ST=shanghai/L=shanghai/O=example/OU=eng/CN=client.device"

# 3. Issue the client certificate using your CA
echo "02" > secure-ca.srl
openssl x509 -req -days 3650 -sha256 \
  -CA secure-ca.crt -CAkey secure-ca.key \
  -in app-client.csr -out app-client.crt

Packaging for Browser Use (PKCS#12)

Most operating systems and browsers import client certificates in PKCS#12 format, which bundles the private key and certificate.

openssl pkcs12 -export -clcerts \
  -in app-client.crt -inkey app-client.key \
  -out app-client.p12

Load the resulting .p12 file into the client's certificate store under "Personal" certificates, supplying the export passphrase you set during creation. After installation, restart the browser.

Updating Nginx to Mutual TLS

The server must present its certificate and also request and validate the client's certificate against the trusted CA.

server {
    listen 443 ssl;
    server_name secure.example.com;

    # Server credentials
    ssl_certificate     /etc/ssl/certs/app-server.crt;
    ssl_certificate_key /etc/ssl/private/app-server.key;

    # Mutual TLS settings
    ssl_client_certificate /etc/ssl/certs/secure-ca.crt;
    ssl_verify_client on;
    ssl_verify_depth 2;

    # ... other directives
}