System Environment and Prerequisites

This guide outlines the procedure for establishing a Redis Cluster with six nodes (three masters and three slaves) on a sinngle CentOS 7 server. In a production setting, these instances should be distributed across multiple physical servers to ensure true fault tolerance.

Environment Specifications:

• OS: CentOS Linux release 7.4.1708 (Core)
• Compiler: GCC 4.8.5
• Redis Version: 4.0.9 (Cluster support introduced in 3.0)
• Ruby Version: 2.5.1 (Required for the legacy redis-trib.rb utility)
• Target Ports: 7001 through 7006
• Deployment Path: /opt/redis-cluster/

All operations are performed using the root user to avoid permission issues during compilation and directory creation. The installation method described below uses offline source packages.

Step 1: Installing Development Tools

First, verify if the GNU Compiler Collection (GCC) is installed. If not, install it along with C++ support using the system package manager.

gcc -v
yum -y install gcc gcc-c++

Step 2: Compiling and Installing Redis

Extract the Redis source code archive and compile the binaries. We will use a specific working directory for the source code.

mkdir -p /opt/source
cd /opt/source
tar -xvf redis-4.0.9.tar.gz
cd redis-4.0.9/
make && make install

Upon completion, the executable files (such as redis-server and redis-cli) will be available in /usr/local/bin.

Step 3: Setting Up the Ruby Environment

Since we are using Redis 4.x, the cluster creation script redis-trib.rb reelies on Ruby. It is crucial to use a version newer than 2.2.2, as the default CentOS yum repository provides an outdated version (2.0.0).

Extract and compile Ruby:

cd /opt/source
tar -xvf ruby-2.5.1.tar.gz
cd ruby-2.5.1/
./configure --prefix=/opt/ruby
make && make install

Configure the system environment variables to include the Ruby binary path:

echo 'export PATH=$PATH:/opt/ruby/bin' >> /etc/profile
source /etc/profile
ruby -v

Step 4: Handling Ruby Dependencies (zlib and OpenSSL)

The Redis gem often requires the zlib and openssl extensions. You may encounter errors during gem installation if these are not correctly linked.

Configuring zlib:

cd /opt/source/ruby-2.5.1/ext/zlib
ruby extconf.rb
# Update the Makefile to fix the include path
# Replace $(top_srcdir) with ../..
sed -i 's/top_srcdir = ..\/../top_srcdir = ../' Makefile
make && make install

Configuring OpenSSL:

cd /opt/source/ruby-2.5.1/ext/openssl
ruby extconf.rb
# Similar to zlib, fix the Makefile paths
sed -i 's/top_srcdir = ..\/../top_srcdir = ../' Makefile
make && make install

Step 5: Preparing Cluster Utilities

Create the directory structure for the cluster and copy the necesary management tools and gems.

mkdir -p /opt/redis-cluster
cd /opt/source/redis-4.0.9/src
cp redis-trib.rb /opt/redis-cluster/
cd /opt/redis-cluster
# Install the specific Redis gem version required
gem install redis-4.0.0.gem

Step 6: Configuring Node Directories

Generate six directories to represent the six nodes (ports 7001-7006). We will automate the creation and configuration process.

cd /opt/redis-cluster
mkdir 700{1,2,3,4,5,6}

# Loop to copy configuration and binaries
for dir in 700{1,2,3,4,5,6}; do
    cp /opt/source/redis-4.0.9/redis.conf /opt/redis-cluster/$dir/
    cp /usr/local/bin/redis-server /opt/redis-cluster/$dir/
    cp /usr/local/bin/redis-cli /opt/redis-cluster/$dir/
done

Modify the configuration files in each directory. Key parameters must be adjusted for cluster operation. The following script demonstrates how to update these values programmatically (replace 192.168.1.100 with your actual server IP):

SERVER_IP="192.168.1.100"
BASE_DIR="/opt/redis-cluster"

for port in 7001 7002 7003 7004 7005 7006; do
    CONF_FILE="$BASE_DIR/$port/redis.conf"
    
    sed -i "s/^bind .*/bind $SERVER_IP/" $CONF_FILE
    sed -i "s/^port .*/port $port/" $CONF_FILE
    sed -i "s/^daemonize .*/daemonize yes/" $CONF_FILE
    sed -i "s/^pidfile .*/pidfile $BASE_DIR\/$port\/redis_$port.pid/" $CONF_FILE
    sed -i "s/^cluster-enabled .*/cluster-enabled yes/" $CONF_FILE
    sed -i "s/^cluster-config-file .*/cluster-config-file nodes-$port.conf/" $CONF_FILE
    sed -i "s/^cluster-node-timeout .*/cluster-node-timeout 5000/" $CONF_FILE
    sed -i "s/^appendonly .*/appendonly yes/" $CONF_FILE
done

Step 7: Starting the Instances

Create a unified startup script to launch all Redis nodes simultaneously.

cat >> /opt/redis-cluster/start-all.sh <<EOF
#!/bin/bash
BASE_DIR="/opt/redis-cluster"
for port in 7001 7002 7003 7004 7005 7006; do
    cd $BASE_DIR/$port
    ./redis-server redis.conf
done
EOF
chmod +x /opt/redis-cluster/start-all.sh

Execute the script and verify that all processes are running:

cd /opt/redis-cluster
./start-all.sh
ps -ef | grep redis

Step 8: Initializing the Cluster

Use the redis-trib.rb utility to create the cluster. We will assign one replica to each master node using the --replicas 1 flag.

cd /opt/redis-cluster
./redis-trib.rb create --replicas 1 192.168.1.100:7001 192.168.1.100:7002 192.168.1.100:7003 192.168.1.100:7004 192.168.1.100:7005 192.168.1.100:7006

When prompted, type yes to accept the configuration. The script will assign hash slots to the master nodes and pair replicas accordingly.

Step 9: Verifying Cluster Functionality

Connect to the cluster using the -c (cluster) flag. This flag is essential; without it, the client will not handle redirections when keys reside on different nodes.

cd /opt/redis-cluster/7001
./redis-cli -c -h 192.168.1.100 -p 7001

Test data insertion and retrieval. Notice how the client automatically redirects to the node holding the specific hash slot.

192.168.1.100:7001> set user:1001 "Alice"
-> Redirected to slot [124] located at 192.168.1.100:7002
OK
192.168.1.100:7002> get user:1001
"Alice"
192.168.1.100:7002> exit

To check the cluster status and node topology, you can use the following commands within the CLI:

cluster info
cluster nodes