Redis Deployment Options

When deploying Redis, several common approaches are available:

This article presents a mutual master-slave approach using only two servers, similar to MySQL's replication, without requiring keepalived by using crontab and iptables for automation.

System Requirements

Server Configuration

Software Requirements

Redis Installation

The following steps should be performed on both Server A and Server B:

Extract Package

# Upload the package to your server and navigate to the directory containing it

# Extract the archive
tar -zxvf redis-7.0.5.tar.gz

Install Compilation Environment

# For Ubuntu
apt-get install make gcc pkg-config libc6-dev

# For CentOS
# yum -y install gcc automake autoconf libtool make

Compile and Install

# Navigate to the source directory
cd redis-7.0.5

# Compile
make

# Install
make PREFIX=/usr/local/redis install

# Copy the default configuration file to the installation directory
cp redis.conf /usr/local/redis/

# If compilation fails, clean residual files before retrying
# For Ubuntu
# make distclean

# For CentOS
# make clean

Modify Configuration File

# Navigate to the installation directory
cd /usr/local/redis/

# Edit the configuration file
vi redis.conf

# Make the following changes:

# Allow remote connections to Redis
bind 0.0.0.0

# Enable daemon mode
daemonize yes

# Set log directory (disable logging by setting to "/dev/null")
logfile "/var/log/redis_6379.log"

# Data persistence directory
dir /usr/local/redis/

# Set password (example):
requirepass D83544E45CA39C7653BF21612FAD0FD1

Start and Test

# Start Redis
./bin/redis-server redis.conf

# Connect to Redis (replace password with your own)
./bin/redis-cli -a D83544E45CA39C7653BF21612FAD0FD1

# Set a variable 'a' with value 1
set a 1

# Get the value of variable 'a'
get a

# Exit
quit

Stop Redis

# D83544E45CA39C7653BF21612FAD0FD1 is the password
./bin/redis-cli -a D83544E45CA39C7653BF21612FAD0FD1 shutdown

Configure Firewall

Ubuntu 22

# If not enabled, start the firewall
ufw enable

# Allow TCP connections to port 6379 from all sources
ufw allow 6379/tcp

CentOS 7

# If not enabled, start the firewall
systemctl start firewalld

# Allow TCP connections to port 6379 from all sources
sudo firewall-cmd --zone=public --add-port=6379/tcp --permanent

# Reload firewall rules
firewall-cmd --reload

Master-Slave Configuration

Create two copies of the redis.conf file, named slave.conf and master.conf:

cp redis.conf slave.conf
cp redis.conf master.conf

Configure Slave Settings

Modify the slave.conf files on both servers as follows:

For Server A (slave.conf):

# Set Server B as the master
replicaof 192.168.88.103 6379

# Set the master's authentication password
masterauth D83544E45CA39C7653BF21612FAD0FD1

For Server B (slave.conf):

# Set Server A as the master
replicaof 192.168.88.102 6379

# Set the master's authentication password
masterauth D83544E45CA39C7653BF21612FAD0FD1

Start and Verify

Start Server A as master:

./bin/redis-server master.conf

Start Server B as slave:

./bin/redis-server slave.conf

Connect to Redis on both servers:

# Connect to Server A
./bin/redis-cli -h 192.168.88.102 -a D83544E45CA39C7653BF21612FAD0FD1

# Connect to Server B
./bin/redis-cli -h 192.168.88.103 -a D83544E45CA39C7653BF21612FAD0FD1

On Server A, set varible 'y' to 3:

set y 3

On Server B, query variable 'y' and verify it returns 3:

get y

This confirms the master-slave setup is working. Note that the master provides both read and write operations, while the slave only provides read operations.

Implementing Master-Slave Failover

The following steps should be performed on both Server A and Server B:

Create Redis Management Script

Create a redis.sh file in the Redis root directory:

vi /usr/local/redis/redis.sh

Add the following content:

#!/bin/bash

# Redis Management Script

# Get current directory
PRG="$0"
while [ -h "$PRG" ]; do
  ls=`ls -ld "$PRG"`
  link=`expr "$ls" : '.*-> \(.*\)$'`
  if expr "$link" : '/.*' > /dev/null; then
    PRG="$link"
  else
    PRG=`dirname "$PRG"`/"$link"
  fi
done
PRGDIR=`dirname "$PRG"`
SH_DIR=`cd "$PRGDIR" >/dev/null; pwd`

APP_NAME="redis-server"
APP_PATH=$SH_DIR
BIN_PATH=$APP_PATH/bin
CONF_PATH=$APP_PATH/
CONF_FILE_PATH=$CONF_PATH/redis.conf
MASTER_CONF_FILE_PATH=$CONF_PATH/master.conf
SLAVE_CONF_FILE_PATH=$CONF_PATH/slave.conf

APP_PORT=6379
APP_PASS="D83544E45CA39C7653BF21612FAD0FD1"

# Read port and password from config file
PORT_STR=$(cat $CONF_FILE_PATH | grep -E '^port'| sed 's/port \([0-9]*\).*/\1/g' )
PASS_STR=$(cat $CONF_FILE_PATH | grep -E '^requirepass' | sed 's/.*requirepass \(.*\)/\1/g' | sed 's/"//g')

if [ -n "$PORT_STR" ]; then
  APP_PORT=$PORT_STR;
fi
if [ -n "$PASS_STR" ]; then
  APP_PASS=$PASS_STR;
fi

# Handle password connection string
PASS_KEY=""
if [ -n "$APP_PASS" ]; then
  PASS_KEY="-a $APP_PASS"
fi

echo -e "\e[35m======================================================================\e[0m"

# Display usage information
usage(){
  echo -e "\e[33mAPP_PATH:\e[0m \e[36m $APP_PATH \e[0m"
  echo -e "\e[33mUsage:\e[0m    \e[36m $0 [start|stop|restart] \e[0m"
  echo -e "\e[33mUsage:\e[0m    \e[36m $0 [status|running|role|slaveup] \e[0m"
  echo -e "\e[33mUsage:\e[0m    \e[36m $0 [tomaster|toslave|startmaster|startslave|startauto] \e[0m"
  echo -e "\e[35m----------------------------------------------------------------------\e[0m"
  echo -e " -\e[32m start   \e[0m: Start Redis normally, $BIN_PATH/redis-server $CONF_FILE_PATH"
  echo -e " -\e[32m stop    \e[0m: Stop Redis, $BIN_PATH/redis-cli shutdown"
  echo -e " -\e[32m restart \e[0m: Restart Redis, stop && start"
  echo -e "\e[35m----------------------------------------------------------------------\e[0m"
  echo -e " -\e[32m status   \e[0m: Current running status and role type"
  echo -e " -\e[32m running  \e[0m: Current running status, returns 0 if running, 1 otherwise"
  echo -e " -\e[32m role     \e[0m: Current role type, returns 0 for master, 1 for slave, 2 for others, 9 if not running"
  echo -e " -\e[32m slaveup  \e[0m: When running as slave, check if master_link_status is up (returns 0 if yes, 1 if no)"
  echo -e " -\e[32m masterup \e[0m: Check the role of the other node (returns 0 for master, 1 for slave)"
  echo -e "\e[35m----------------------------------------------------------------------\e[0m"
  echo -e " -\e[32m tomaster   \e[0m: Convert current role to [master]"
  echo -e " -\e[32m toslave    \e[0m: Convert current role to [slave]"
  echo -e " -\e[32m startmaster\e[0m: Start as [master] if not running"
  echo -e " -\e[32m startslave \e[0m: Start as [slave] if not running"
  echo -e " -\e[32m startauto  \e[0m: Auto-determine role (master/slave) based on slave.conf master node status when starting"
  echo ""
  exit 1
}

# Check if Redis is running
is_running(){
  # Get process pid by application name
  pid=`ps -ef|grep $APP_NAME|grep -v grep|awk '{print $2}'
  
  # Return 1 if not running, 0 if running
  if [ -z "${pid}" ]; then
   return 1
  else
   return 0
  fi
}

# Start Redis
start(){
  is_running
  if [ $? -eq "0" ]; then
    echo "$APP_NAME - [PORT: $APP_PORT] is already running. pid=${pid}"
  else
    $BIN_PATH/redis-server $CONF_FILE_PATH
    sleep 1
    status
    if [ $? -eq "0" ]; then
      echo "$APP_NAME - [PORT: $APP_PORT] started successfully. pid=${pid}"
    else
      echo "$APP_NAME - [PORT: $APP_PORT] failed to start."
    fi
  fi
}

# Start with master configuration
startmaster(){
  echo "Preparing to start in [master] mode, replacing redis.conf..."
  cp -f $MASTER_CONF_FILE_PATH $CONF_FILE_PATH
  sleep 1
  start
}

# Start with slave configuration
startslave(){
  echo "Preparing to start in [slave] mode, replacing redis.conf..."
  cp -f $SLAVE_CONF_FILE_PATH $CONF_FILE_PATH
  sleep 1
  start
}

# Auto-determine master or slave mode
startauto(){
  is_running
  if [ $? -eq "0" ]; then
    echo "${APP_NAME} - [PORT: ${APP_PORT}] is already running. pid=${pid}"
    exit 1
  fi

  # Read master configuration from slave.conf
  arr=(`cat ${SLAVE_CONF_FILE_PATH} | grep ^replicaof`)
  master_host=${arr[1]}
  master_port=${arr[2]}
  authcmd=$(cat $SLAVE_CONF_FILE_PATH | grep ^masterauth | sed 's/masterauth //g' | sed 's/"//g')
  auth_key=""

  if [ -n "$authcmd" ]; then
    auth_key="-a $authcmd"
  fi
  
  echo "Checking master -h $master_host -p $master_port $auth_key"
  
  # Determine the role of the configured master
  roleName=$($BIN_PATH/redis-cli -h $master_host -p $master_port $auth_key info | grep role: | sed 's/.$//g')

  echo "Configured master's current role: $roleName"
  if [ "$roleName" = "role:master" ]; then
    startslave
  else
    startmaster
  fi
}

# Check if the other node is in master mode
masterup(){
  # Read master configuration from slave.conf
  arr=(`cat ${SLAVE_CONF_FILE_PATH} | grep ^replicaof`)
  master_host=${arr[1]}
  master_port=${arr[2]}
  authcmd=$(cat $SLAVE_CONF_FILE_PATH | grep ^masterauth | sed 's/masterauth //g' | sed 's/"//g')
  auth_key=""

  if [ -n "$authcmd" ]; then
    auth_key="-a $authcmd"
  fi
  
  echo "Checking master -h $master_host -p $master_port $auth_key"
  
  # Determine the role of the configured master
  roleName=$($BIN_PATH/redis-cli -h $master_host -p $master_port $auth_key info | grep role: | sed 's/.$//g')

  echo "Configured master's current role: $roleName"
  if [ "$roleName" = "role:master" ]; then
    return 0
  else
    return 1
  fi
}

# Stop Redis
stop(){
  is_running
  if [ $? -eq "0" ]; then
    echo "$APP_NAME - [PORT: $APP_PORT] is running. pid=${pid}. Stopping..."
    # Execute shutdown
    $BIN_PATH/redis-cli $PASS_KEY shutdown

    sleep 1
    is_running
    if [ $? -eq "0" ]; then
      echo "$APP_NAME - [PORT: $APP_PORT] failed to stop. pid=${pid}"
    else
      echo "$APP_NAME - [PORT: $APP_PORT] stopped successfully."
    fi
  else
    echo "$APP_NAME - [PORT: $APP_PORT] is not running."
  fi
}

# Restart Redis
restart(){
  stop
  start
}

# Determine the current role [master, slave]
# Returns 0 for master, 1 for slave, 2 for others, 9 if not running
role(){
  is_running
  if [ $? -eq "0" ]; then
    roleName=$($BIN_PATH/redis-cli $PASS_KEY info | grep role: | sed 's/.$//g')
    if [ "$roleName" = "role:master" ]; then
      return 0
    elif [ "$roleName" = "role:slave" ]; then
      return 1
    else
      echo "ERROR: Unknown role: [$roleName] !!!"
      return 2
    fi
  else
    echo "$APP_NAME - [PORT: $APP_PORT] is not running."
    return 9
  fi
}

# Check if current instance is slave and properly connected to master
# Simplified logic: checks for master_link_status:up in info (returns 0 if normal, 1 if not)
# Should be called only when the node is known to be running and a slave
slaveup(){
  linkStatus=$($BIN_PATH/redis-cli $PASS_KEY info | grep master_link_status:up)
  if [ -z "$linkStatus" ]; then
    return 1
  else
    return 0
  fi
}

# Change Redis to master role
tomaster(){
  role
  mode=$?
  if [ $mode -eq "0" ]; then
    echo "$APP_NAME - [PORT: $APP_PORT] is already [master]. No switch needed."
  elif [ $mode -eq "1" ]; then
    # Switch configuration file to avoid changes after restart
    cp -f $MASTER_CONF_FILE_PATH $CONF_FILE_PATH
    # Execute switch to master
    $BIN_PATH/redis-cli $PASS_KEY replicaof no one
    sleep 1
    status
  else
    echo "$APP_NAME - [PORT: $APP_PORT] cannot handle current mode [$mode]."
  fi
}

# Change Redis to slave role
toslave(){
  role
  mode=$?
  if [ $mode -eq "1" ]; then
    echo "$APP_NAME - [PORT: $APP_PORT] is already [slave]. No switch needed."
  elif [ $mode -eq "0" ]; then
    # Switch configuration file to avoid changes after restart
    cp -f $SLAVE_CONF_FILE_PATH $CONF_FILE_PATH

    authcmd=$(cat $CONF_FILE_PATH | grep ^masterauth | sed 's/"//g')
    echo "Setting master connection password. $authcmd"
    # Update password
    $BIN_PATH/redis-cli $PASS_KEY config set $authcmd

    repcmd=$(cat $CONF_FILE_PATH | grep ^replicaof)
    echo "Preparing to switch to [slave], $repcmd"
    # Execute switch to slave
    $BIN_PATH/redis-cli $PASS_KEY $repcmd

    sleep 1
    status
  else
    echo "$APP_NAME - [PORT: $APP_PORT] cannot handle current mode [$mode]."
  fi
}

# Display status
status(){
  is_running
  if [ $? -eq "0" ]; then
    echo "$APP_NAME - [PORT: $APP_PORT] is running. pid=${pid}"
    role
    if [ $? -eq "0" ]; then
      echo "Redis is master role"
    elif [ $? -eq "1" ]; then
      echo "Redis is slave role"
    else
      echo "Redis is unknown role"
    fi
  else
    echo "$APP_NAME - [PORT: $APP_PORT] is not running."
  fi
}

# Execute appropriate function based on input parameters
case "$1" in
  "start")
    start
    ;;
  "stop")
    stop
    ;;
  "running")
    is_running
    ;;
  "status")
    status
    ;;
  "role")
    role
    ;;
  "restart")
    restart
    ;;
  "tomaster")
    tomaster
    ;;
  "toslave")
    toslave
    ;;
  "startmaster")
    startmaster
    ;;
  "startslave")
    startslave
    ;;
  "startauto")
    startauto
    ;;
  "slaveup")
    slaveup
    ;;
  "masterup")
    masterup
    ;;
  *)
    usage
    ;;
esac

Make Script Executable

chmod +x redis.sh

Test Initial Setup

Restart Redis services on both servers:

./redis.sh restart

Switch Server B from slave to master:

./redis.sh tomaster

Switch Server A from master to slave:

./redis.sh toslave

Connect to both Redis instances:

On Server B, connect to local Redis and set variable 't' to 2:

set t 2

On Server A, connect to Server B's Redis and check variable 't', which should return 2:

get t

Automatic Master-Slave Failover

The following steps should be performed on both Server A and Server B:

Create Detection Script

Create a vip_keepalived.sh file:

vi /usr/local/redis/vip_keepalived.sh

Add the following content:

#!/bin/bash

BASE_PATH=/usr/local/redis

# Enable forwarding rules
iptables_up(){
  while ([ `iptables -t nat -nL PREROUTING --line | grep 6379$ | grep 6378 | wc -l` -gt 1 ])
  do
    iptables -t nat -D PREROUTING -p tcp --dport 6378 -j REDIRECT --to-port 6379
  done

  while ([ `iptables -t nat -nL OUTPUT --line | grep 6379$ | grep 6378 | wc -l` -gt 1 ])
  do
    iptables -t nat -D OUTPUT -p tcp --dport 6378 -j REDIRECT --to-port 6379
  done

  if [ `iptables -t nat -nL PREROUTING --line | grep 6379$ | grep 6378 | wc -l` -eq "0" ]; then
    iptables -t nat -A PREROUTING -p tcp --dport 6378 -j REDIRECT --to-port 6379
  fi

  if [ `iptables -t nat -nL OUTPUT --line | grep 6379$ | grep 6378 | wc -l` -eq "0" ]; then
    iptables -t nat -A OUTPUT -p tcp --dport 6378 -j REDIRECT --to-port 6379
  fi
}

# Remove forwarding rules
iptables_down(){
  while([ `iptables -t nat -nL PREROUTING --line | grep 6379$ | grep 6378 | wc -l` -gt 0 ])
  do
    iptables -t nat -D PREROUTING -p tcp --dport 6378 -j REDIRECT --to-port 6379
  done

  while([ `iptables -t nat -nL OUTPUT --line | grep 6379$ | grep 6378 | wc -l` -gt 0 ])
  do
    iptables -t nat -D OUTPUT -p tcp --dport 6378 -j REDIRECT --to-port 6379
  done
}

# Check if role needs to be switched
check(){
  # Determine current role [master, slave]
  # Returns 0 for master, 1 for slave, 2 for others, 9 if not running
  $BASE_PATH/redis.sh role
  redisRole=$?
  echo "Current Redis role value: [$redisRole]"
  
  if [ $redisRole -eq "0" ]; then
    iptables_up
    # Redis is running as master
    $BASE_PATH/redis.sh masterup
    if [ $? -eq "0" ]; then
      $BASE_PATH/redis.sh toslave
      iptables_down
    fi
    return 0
  elif [ $redisRole -eq "1" ]; then
    iptables_down
    # Redis is running as slave
    $BASE_PATH/redis.sh slaveup
    if [ $? -eq "0" ]; then
      return 1
    else
      # Check if the other node is not master before switching
      $BASE_PATH/redis.sh masterup
      if [ ! $? -eq "0" ]; then
        # Slave connection to master is down, switch this node to master
        $BASE_PATH/redis.sh tomaster
        iptables_up
        return 0
      else
        iptables_down
        return 1
      fi
    fi
  else
    iptables_down
    return $redisRole
  fi
}

check

Make Script Executable

chmod +x /usr/local/redis/vip_keepalived.sh

Create Cron Job

echo "* * * * * /usr/local/redis/vip_keepalived.sh > /tmp/check_redis.log" | crontab

Setting Up Nginx for High Availability

Install Nginx

Reference previous installation guides, ensuring to include the --with-stream option:

# Install Nginx core modules
tar -zxvf nginx-1.22.1.tar.gz
cd nginx-1.22.1
# Add TCP proxy module
./configure --with-stream
make
make install
chmod a+rwx -R /usr/local/nginx/logs/
chmod a+rwx -R /usr/local/nginx/
/usr/local/nginx/sbin/nginx -t
/usr/local/nginx/sbin/nginx

# Set to start on boot
cp $PRGDIR/nginx /etc/init.d/
chmod +x /etc/init.d/nginx
chkconfig --add nginx
chkconfig nginx on

Configure Nginx for Redis High Availability

Add the following to the end of your nginx.conf file:

stream {
  upstream redis_backend {
    # Server A Redis mapping
    server 192.168.88.102:6378;
    # Server B Redis mapping
    server 192.168.88.103:6378;
  }
  
  server {
    # Proxy port 6380 - applications should connect to Redis via 127.0.0.1:6380
    listen 6380;
    proxy_pass redis_backend;
  }
}

Restart Nginx to apply changes:

/usr/local/nginx/sbin/nginx -s quit
/usr/local/nginx/sbin/nginx

Summary of Redis High Availability Approaches

The best approach depends on your specific requirements and constraints.