Environment Setup

A cluster with three CentOS 7 virtual machines is used: one master and two etcd nodes. The master also serves as a node for pod workloads. Certificate distribution and other operations are performed from a separate host configured with SSH key-based access to all cluster nodes.

IP Address	Node Roles
192.168.0.153	master, node, etcd
192.168.0.154	master, node, etcd
192.168.0.164	master, node, etcd

Since ETCD and Kubernetes rely on TLS for communication, TLS certificates must be generated first using cfssl.

Certificate Management

Certificate Overview

Certificate File	Configuration File	Purpose
etcd-root-ca.pem	etcd-root-ca-csr.json	Root CA certificate for ETCD
etcd.pem	etcd-gencert.json, etcd-csr.json	Cluster certificate for ETCD

Installing CFSSL

Download cfssl, grant execute permissions, and place it in the PATH directory.

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
chmod +x cfssl_linux-amd64 cfssljson_linux-amd64
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson

Configuration files required for ETCD certificate generation:

• etcd-root-ca-csr.json

{
   "key": {
     "algo": "rsa",
     "size": 4096
   },
   "names": [
     {
       "O": "etcd",
       "OU": "etcd Security",
       "L": "Beijing",
       "ST": "Beijing",
       "C": "CN"
     }
   ],
   "CN": "etcd-root-ca"
}

• etcd-gencert.json

{
  "signing": {
    "default": {
        "usages": [
          "signing",
          "key encipherment",
          "server auth",
          "client auth"
        ],
        "expiry": "87600h"
    }
  }
}

• etcd-csr.json

{
  "key": {
    "algo": "rsa",
    "size": 4096
  },
  "names": [
    {
      "O": "etcd",
      "OU": "etcd Security",
      "L": "Beijing",
      "ST": "Beijing",
      "C": "CN"
    }
  ],
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "localhost",
    "192.168.0.153",
    "192.168.0.154",
    "192.168.0.164",
    "master",
    "node1",
    "node2"
  ]
}

Generate ETCD certificates:

cfssl gencert --initca=true etcd-root-ca-csr.json | cfssljson --bare etcd-root-ca
cfssl gencert --ca etcd-root-ca.pem --ca-key etcd-root-ca-key.pem --config etcd-gencert.json etcd-csr.json | cfssljson --bare etcd

Deploying High-Availability ETCD

Pre-installation Steps

• Disable SELinux: setenforce 0
• Stop firewall: systemctl stop firewalld; iptables -F
• Synchronize time: ntpdate time1.aliyun.com

Installing ETCD

Perform these steps on the master node.

ETCD is installed via RPM. Obtain the spec file from the Fedora repository or search on rpmFind.

# Download RPM package
wget ftp://195.220.108.108/linux/fedora/linux/development/rawhide/Everything/x86_64/os/Packages/e/etcd-3.2.7-1.fc28.x86_64.rpm

# Distribute and install
NODE_IPS="192.168.0.153 192.168.0.154 192.168.0.164"
for NODE_IP in $NODE_IPS; do
    scp etcd-3.2.7-1.fc28.x86_64.rpm root@$NODE_IP:~
    ssh root@$NODE_IP rpm -ivh etcd-3.2.7-1.fc28.x86_64.rpm
done

Distributing Certificates

NODE_IPS="192.168.0.153 192.168.0.154 192.168.0.164"

for NODE_IP in $NODE_IPS; do
    ssh root@$NODE_IP mkdir /etc/etcd/ssl/
    scp *.pem root@$NODE_IP:/etc/etcd/ssl/
    ssh root@$NODE_IP chown -R etcd:etcd /etc/etcd/ssl/
    ssh root@$NODE_IP chmod -R 755 /etc/etcd/
done

Modifying Configuraton

After RPM installation, edit /etc/etcd/etcd.conf. Example configuration for one node (adjust names and IPs for others):

# [member]
ETCD_NAME=etcd0
ETCD_DATA_DIR="/var/lib/etcd/etcd0.etcd"
ETCD_WAL_DIR="/var/lib/etcd/wal"
ETCD_SNAPSHOT_COUNT="100"
ETCD_HEARTBEAT_INTERVAL="100"
ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="https://192.168.0.153:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.0.153:2379,http://127.0.0.1:2379"
ETCD_MAX_SNAPSHOTS="5"
ETCD_MAX_WALS="5"
#ETCD_CORS=""

# [cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.0.153:2380"
ETCD_INITIAL_CLUSTER="etcd0=https://192.168.0.153:2380,etcd1=https://192.168.0.154:2380,etcd2=https://192.168.0.164:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.0.153:2379"

# [security]
ETCD_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
ETCD_CLIENT_CERT_AUTH="true"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-root-ca.pem"
ETCD_AUTO_TLS="true"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
ETCD_PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-root-ca.pem"
ETCD_PEER_AUTO_TLS="true"

Adjust thece parameters for each node:

ETCD_NAME
ETCD_LISTEN_PEER_URLS
ETCD_LISTEN_CLIENT_URLS
ETCD_INITIAL_ADVERTISE_PEER_URLS
ETCD_ADVERTISE_CLIENT_URLS

Starting and Verifying

After configuration, start ETCD on each node. Ensure clock synchronization across nodes too prevent startup failures.

systemctl daemon-reload
systemctl start etcd
systemctl enable etcd

Verify cluster health:

export ETCDCTL_API=3
etcdctl --cacert=/etc/etcd/ssl/etcd-root-ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem --endpoints=https://192.168.0.153:2379,https://192.168.0.154:2379,https://192.168.0.164:2379 endpoint health