Fading Coder

One Final Commit for the Last Sprint

Home > Tech > Content

Redis Data Durability: RDB Snapshots and AOF Logs

Tech May 11 3

Redis achieves exceptional speed by keeping the entire dataset in memory. To prevent data loss during restarts, crashes, or power failures, it provides two persistence strategies: RDB (Redis Database) snapshots and AOF (Append Only File) logs.

RDB Persistence

RDB acts as the default persistence mechanism unless AOF is explicitly enabled. It periodically writes a point-in-time snapshot of the in-memory dataset to a binary dump file named dump.rdb. Upon restart, Redis loads this file to reconstruct the data.

Triggering an RDB Snapshot

Snapshots can be initiated automatically or manually.

Automatic Triggers

Configuration Rules
The redis.conf file contains save directives that define frequency thresholds. Commenting out or emptying these directives disables RDB.

save 900 1   # At least 1 write in 900 seconds
save 300 10  # At least 10 writes in 300 seconds
save 60 10000 # At least 10,000 writes in 60 seconds

Multiple rules can coexist; satisfying any one triggers a snapshot. The lastsave command reports the timestamp of the most recent successful snapshot.

Directory and Compression Settings

dir ./
dbfilename dump.rdb
rdbcompression yes
rdbchecksum yes
  • dir specifies the working directory for RDB files (relative path by default).
  • rdbcompression applies LZF compression to reduce file size at a minor CPU cost.
  • rdbchecksum uses CRC64 verification, adding roughly 10% performance overhead for data integrity.

Other Automatic Events

  • A graceful shutdown via SHUTDOWN generates a snapshot before the server exits.
  • A FLUSHALL command also produces an RDB file, but it will be empty.

Manual Triggers

Two commands let administrators generate snapshots on demand.

  • SAVE synchronously freezes the server during snapshot creation. It blocks all other requests and is discouraged in production.
  • BGSAVE forks a child process using copy-on-write semantics to handle the disk write asynchronously. The main process remains responsive, blocking only briefly during the initial fork operation. Writes occurring after the fork are not captured in the snapshot.

Recovery Using RDB Files

Standard Recovery Procedure

  1. Insert test data:
    redis> set k1 1
    redis> set k2 2
    redis> set k3 3
    redis> set k4 4
    redis> set k5 5
    
  2. Shut down the server to trigger a final snapshot:
    redis> shutdown
    
  3. Back up the generated file:
    cp dump.rdb dump.rdb.bak
    
  4. Restart the server and verify that all keys persist:
    redis> keys *
    

Simulating and Recovering from Data Loss

First, flush all data and restart:

redis> flushall
# restart server
redis> keys *  # returns empty

Stop the server, restore the backup, and start again:

redis> shutdown
mv dump.rdb.bak dump.rdb
# restart server
redis> keys *  # data restored

Advantages and Disadvantages

Strengths

  • Produces a single compact file ideal for backups and disaster recovery.
  • Offloads disk I/O to a child process, keeping the main process free.
  • Restores large datasets faster than AOF.

Weaknesses

  • Cannot guarantee real-time or second-level durability. Frequent BGSAVE invocations become expensive due to repeated forking.
  • Data written between snapshots is lost if the server crashes unexpectedly.

AOF Persistence

AOF logs every write operation and replays them at startup to rebuild the dataset. It is not enabled by default.

Configuration

Modify redis.conf:

appendonly yes
appendfilename "appendonly.aof"

The file path follows the dir setting.

Internally, AOF records commands in Redis protocol format. For example:

*2
$6
SELECT
$1
0
*3
$3
set
$5
mykey
$5
hello

Synchronization Strategies

Data is initially buffered by the operating system. The appendfsync parameter controls how often the buffer is physically written to disk:

  • no: Delegates synchronization to the OS; fastest but potentially unsafe.
  • always: Calls fsync after every write; safe but drastically reduces performance.
  • everysec (default): Synchronizes once per second, balancing safety and speed.

AOF Rewriting

As operations accumulate, the AOF file expands indefinitely, consuming storage and slowing restarts. Redis mitigates this through rewriting: it reads the current dataset and generates a compact set of commands that produces the same final state.

Triggering rewrite:

bgrewriteaof

Automatic triggers are based on growth relative to the last rewritten file:

auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
  • rewrite-percentage: When the file size is double the previous rewrite size (100% growth), a new rewrite starts.
  • rewrite-min-size: Prevents rewriting files that are still small despite the percentage threshold being met.

Concurrent Writes During Rewrite

While a child process performs the rewrite, the parent server continues handling commands and appends them both to the existing AOF file and an in-memory rewrite buffer. Once the child finishes, the buffered commands are flushed into the new file, which then replaces the old one atomically.

Additional Parameters

  • no-appendfsync-on-rewrite: During heavy I/O operations, fsync calls can cause latency spikes. Setting this to yes pauses fsync during rewrites, temporarily buffering writes in memory until the rewrite completes. The default is no.
  • aof-load-truncated: If the AOF file gets truncated due to a system crash, setting this to yes (default) loads the partial data and emits a log warning. Setting it to no aborts startup and requires manual repair via redis-check-aof.

Recovery and Trade-offs

Redis reads and replays the AOF log automatically on startup.

Advantages

  • Granular durability; with everysec, at most one second of data is lost.

Disadvantages

  • AOF files usually grow larger than RDB files containing the same dataset.
  • Under extreme throughput, RDB tends to provide better overall performance.

Choosing a Persistence Strategy

For workloads tolerant of losing a few minutes of data, RDB is usually the superior choice—it simplifies backups and accelerates restores. If minimizing data loss is critical, adopt AOF. For comprehensive protection, enable both mechanisms. On restart, Redis preferentially loads the AOF file because it is typically more complete than the latest RDB snapshot.

Tags: Redis

Related Articles

Understanding Strong and Weak References in Java

Strong References Strong reference are the most prevalent type of object referencing in Java. When an object has a strong reference pointing to it, the garbage collector will not reclaim its memory. F...

Comprehensive Guide to SSTI Explained with Payload Bypass Techniques

Introduction Server-Side Template Injection (SSTI) is a vulnerability in web applications where user input is improper handled within the template engine and executed on the server. This exploit can r...

Implement Image Upload Functionality for Django Integrated TinyMCE Editor

Django’s Admin panel is highly user-friendly, and pairing it with TinyMCE, an effective rich text editor, simplifies content management significantly. Combining the two is particular useful for bloggi...

Leave a Comment

Anonymous

◎Feel free to join the discussion and share your thoughts.