etcd Service Installation and Basic Operations

Installing etcd

wget https://github.com/coreos/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-amd64.tar.gz
tar -zxf etcd-v3.5.0-linux-amd64.tar.gz
cp etcd/etcd etcd/etcdctl /usr/local/bin/

Starting etcd Server

mkdir -p /var/lib/etcd
nohup etcd --name auto-scale --data-dir /var/lib/etcd/ \
     --listen-peer-urls 'http://10.0.1.100:2380,http://10.0.1.100:7001' \
     --listen-client-urls 'http://10.0.1.100:2379,http://10.0.1.100:4001' \
     --advertise-client-urls 'http://10.0.1.100:2379,http://10.0.1.100:4001' &

Storing Data in etcd

curl -s http://10.0.1.100:2379/v2/keys/app/config -XPUT -d value="production" | python -m json.tool

Retrieving Stored Data

curl -s http://10.0.1.100:2379/v2/keys/app/config | python -m json.tool

Deleting a Key

curl -s http://10.0.1.100:2379/v2/keys/app/config -XDELETE | python -m json.tool

Creating Keys with TTL

curl -s http://10.0.1.100:2379/v2/keys/session/active -XPUT -d value="user123" -d ttl=60 | python -m json.tool

Configuring SaltStack with etcd

Installing Python etcd Client

yum install -y python-pip
pip install python-etcd

Configuring Master for etcd Integration

Edit /etc/salt/master:

etcd_pillar:
  etcd.host: 10.0.1.100
  etcd.port: 2379

ext_pillar:
  - etcd: etcd_pillar root=/salt/services/

Restarting Salt Master

systemctl restart salt-master

Automating HAProxy Backend Pool Management

Creating Backend Configuration Template

Edit /srv/salt/prod/balancer/files/haproxy-front.cfg:

balance roundrobin
{% for node,node_addr in pillar.backend_api_gateway.iteritems() %}
server {{ node }} {{ node_addr }} check inter 2000 rise 30 fall 15
{% endfor %}

Creating Salt State File

Edit /srv/salt/prod/balancer/haproxy-front.sls:

include:
  - balancer.install

haproxy-config:
  file.managed:
    - name: /etc/haproxy/haproxy.cfg
    - source: salt://balancer/files/haproxy-front.cfg
    - user: root
    - group: root
    - mode: 644
    - template: jinja

haproxy-service:
  service.running:
    - name: haproxy
    - enable: true
    - reload: true
    - watch:
      - file: haproxy-config

Implementation Workflow

Step 1: Add Backend Node to etcd

curl -s http://10.0.1.100:2379/v2/keys/salt/services/backend_api_gateway/node-5 \
     -XPUT -d value="10.0.1.115:8080" | python -m json.tool

Step 2: Apply Configuration

salt '*' state.sls balancer.haproxy-front env=prod

Step 3: Verify Pillar Data

salt '*' pillar.items

Automation Script

#!/bin/bash

add_backend_node() {
    local node_name=$1
    local node_ip=$2
    local node_port=$3
    
    echo "Registering new backend node: ${node_name}"
    
    etcd_register "${node_name}" "${node_ip}:${node_port}"
    sync_load_balancer
    verify_registration "${node_name}"
}

etcd_register() {
    local key=$1
    local value=$2
    curl -s http://10.0.1.100:2379/v2/keys/salt/services/backend_api_gateway/${key} \
         -XPUT -d value="${value}" | python -m json.tool
}

sync_load_balancer() {
    salt '*' state.sls balancer.haproxy-front env=prod
}

verify_registration() {
    local node=$1
    echo "Verifying node ${node} is active..."
}

main() {
    backend_name=$1
    backend_address=$2
    backend_port=${3:-8080}
    
    add_backend_node "${backend_name}" "${backend_address}" "${backend_port}"
}

main "$@"

Verification Process

After executing the automation workflow, verify the configuration:

salt '*' pillar.item backend_api_gateway

Expected output shows all registered backend nodes:

node1:
    backend_api_gateway:
        ----------
        node-1:
            10.0.1.111:8080
        node-2:
            10.0.1.112:8080
        node-3:
            10.0.1.113:8080
        node-4:
            10.0.1.114:8080
        node-5:
            10.0.1.115:8080

Use Case: Auto-Scaling Triggered by Monitoring

When monitoring detects high load on nginx workers reaching threshold:

1. Monitoring system triggers alert
2. Automation platform provisions new instacne
3. Service deployment executes via Salt
4. Health check validates node status
5. New node registration in etcd
6. HAProxy configuration refresh
7. Load balancer reloads configuration
8. Monitoring resumes with updated pool