Implementing High Availability with Keepalived and Virtual IP Addresses
Keepalived is a software solution for achieving high availability in network services. It is primarily used to ensure service continuity and prevent single points of failure, commonly applied in load balancing and failover scenarios.
Core Concepts of Keeepalived
- VRRP-Based Implementation: Keepalived is built on the Virtual Router Redundancy Protocol (VRRP).
- Management Capabilities: It can manage LVS (Linux Virtual Server) clusters or operate independently to provide service high availability.
- Virtual IP Address (VIP): Failover is accomplished by configuring a shared virtual IP address.
- Health Monitoring: The software monitors the status of servers and services, triggering automatic switches upon detecting failures.
This makes it particularly suitable for network environments requiring rapid failover and load balancing.
Operational Mechanics
- VRRP Protocol: Keepalived utilizes VRRP to form a high-availability group. Multiple servers join a VRRP group, sharing one virtual IP address. One server acts as the master, while others are backups. The master sends periodic VRRP advertisement packets. If a backup server fails to receive these advertisements within a specified timeframe, it assumes control of the virtual IP.
- Health Checking: Keepalived performs regular health checks on local and remote services. It supports various check methods, including TCP, HTTP, and SSL. Detection of an unavailable service initiates a failover.
- Load Balancing (with LVS): When integrated with LVS, Keepalived can manage LVS configurations. It supports multiple load-balancing algorithms such as round-robin, weighted round-robin, and least connections.
- Configuration Management: Parameters for VRRP instances, virtual IPs, and health checks are defined in a configuration file. Dynamic configuration updates are supported without requiring a service restart.
- Failover Process: If the master server fails, a backup server automatically takes over the virtual IP. This transition is swift, typically completing within seconds, making it nearly imperceptible to end-users.
Through these mechanisms, Keepalived effectively delivers high availability and load balancing, ensuring system stability and reliability.
Lab Environment Setup
- Master Server: Physical IP
192.168.192.100 - Backup Server: Physical IP
192.168.192.111 - Shared Virtual IP:
192.168.192.200
Master Server Configuration
First, install the necessary packages.
# Install development tools and dependencies
yum -y install gcc openssl-devel libnfnetlink-devel libnl libnl3-devel
# Install Keepalived and IPVS administration tools
yum install -y keepalived ipvsadm
Next, edit the main Keepalived configuration file.
vim /etc/keepalived/keepalived.conf
Replace or modify the file with the following configuration. Key parameters for the master are state MASTER and a higher priority (e.g., 100).
! Configuration File for keepalived
global_defs {
notification_email {
admin@example.loc
ops@example.loc
}
notification_email_from keepalived@example.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id NodeA # Unique identifier for this node
}
vrrp_instance HA_Group {
state MASTER # This node is configured as the master
interface ens33 # Network interface for VRRP communication
virtual_router_id 51 # Must be identical across all nodes in the group
priority 100 # Higher priority wins the master election
advert_int 1 # Advertisement interval in seconds
authentication {
auth_type PASS # Simple password authentication
auth_pass secure123 # Authentication password (must match on all nodes)
}
virtual_ipaddress {
192.168.192.200/24 # The shared virtual IP address
}
}
# Optional: LVS configuration section for load balancing
virtual_server 192.168.192.200 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.255.0
protocol TCP
real_server 192.168.192.112 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.192.114 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
Keepalived performs regular health checks on the defined real servers. If a check fails, it attempts to reconnect. After exhausting the specified retry attempts, the server is marked unavailable, and traffic is redirected to other healthy servers.
Finally, start and enable the Keepalived service.
systemctl start keepalived # Start the service
systemctl enable keepalived # Enable auto-start on boot
systemctl status keepalived # Verify the service status
Backup Server Configuration
On the backup server, install the same packages.
yum -y install gcc openssl-devel libnfnetlink-devel libnl libnl3-devel
yum install -y keepalived ipvsadm
Edit the configuration file /etc/keepalived/keepalived.conf. The critical differences are the router_id, state BACKUP, and a lower priority (e.g., 95). The virtual_router_id and auth_pass must match the master's configuration.
! Configuration File for keepalived
global_defs {
router_id NodeB # Unique identifier for this backup node
}
vrrp_instance HA_Group {
state BACKUP # This node is configured as a backup
interface ens33
virtual_router_id 51 # Must match the master's ID
priority 95 # Lower priority than the master
advert_int 1
authentication {
auth_type PASS
auth_pass secure123 # Must match the master's password
}
virtual_ipaddress {
192.168.192.200/24
}
}
# The virtual_server block for LVS should be identical to the master if used.
virtual_server 192.168.192.200 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.255.0
protocol TCP
real_server 192.168.192.112 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.192.114 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
Start and enable the service on the backup server as well.
systemctl start keepalived
systemctl enable keepalived
systemctl status keepalived # Check for any errors
Validating Virtual IP Failover
With both services running, the virtual IP should be active on the master server. Verify this using the ip command.
On the Master Server:
ip address show dev ens33
The output should show both the physical IP (e.g., 192.168.192.100) and the virtual IP (192.168.192.200) assigned to the ens33 interface.
On the Backup Server:
ip address show dev ens33
The output should show only the physical IP (e.g., 192.168.192.111). The virtual IP is not present because this node is in backup state.
Testing Failover: Simulate a failure on the master server by stopping its Keepalived service.
# On the Master Server
systemctl stop keepalived
After a few seconds (governed by the advert_int and failure detection time), check the interface on the backup server again.
# On the Backup Server
ip address show dev ens33
The virtual IP (192.168.192.200) should now appear on the backup server's interface, confirming that the failover has successfully occurred. Restarting Keepalived on the original master should cause it to reclaim the virtual IP due to its higher priority.