Environment and dependencies

A local Redis instance is requried. Benchmarks are executed with JMH. Maven dependencies:

<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-core</artifactId>
    <version>1.21</version>
</dependency>
<dependency>
    <groupId>org.openjdk.jmh</groupId>
    <artifactId>jmh-generator-annprocess</artifactId>
    <version>1.21</version>
    <scope>provided</scope>
</dependency>

<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
    <version>2.9.0</version>
</dependency>
<dependency>
    <groupId>io.lettuce</groupId>
    <artifactId>lettuce-core</artifactId>
    <version>5.0.3.RELEASE</version>
</dependency>

JMH annotation choices

• @BenchmarkMode(Mode.Throughput): measure operations per time unit.
• @OutputTimeUnit(TimeUnit.MILLISECONDS): emit results in milliseconds.
• @Warmup(iterations = 1): warmup runs to stabilize the JVM.
• @Measurement(iterations = 2, time = 600, timeUnit = MILLISECONDS): measure runs and duration.
• @Threads(100): run with 100 concurrent worker threads.
• @State(Scope.Thread) vs @State(Scope.Benchmark): per-thread state for thread-unsafe clients (Jedis), shared state to thread-safe cleints (Lettuce).

Jedis benchmark (per-thread connection)

Jedis instances are not thread-safe; allocate one per benchmarking thread.

@BenchmarkMode(Mode.Throughput)
@Warmup(iterations = 1)
@Measurement(iterations = 2, time = 600, timeUnit = TimeUnit.MILLISECONDS)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Threads(100)
@State(Scope.Thread)
public class JedisGetBenchmark {

    private static final String KEY = "a";

    @Param({"1"})
    private int repeat; // number of GETs per benchmark invocation

    private redis.clients.jedis.Jedis client;

    @Setup
    public void open() {
        client = new redis.clients.jedis.Jedis("127.0.0.1", 6379);
    }

    @TearDown
    public void close() {
        if (client != null) {
            client.close();
        }
    }

    @Benchmark
    public void getLoop() {
        for (int i = 0; i < repeat; i++) {
            client.get(KEY);
        }
    }
}

Runner:

public class JedisRunner {
    public static void main(String[] args) throws Exception {
        Options opt = new OptionsBuilder()
                .include(JedisGetBenchmark.class.getSimpleName())
                .output("benchmark/jedis-throughput.log")
                .forks(1)
                .build();
        new Runner(opt).run();
    }
}

Lettuce benchmark (shared, async connection)

Lettuce connections are thread-safe; a single connection can be shared across threads. Asynchronous commands are issued and then awaited.

@BenchmarkMode(Mode.Throughput)
@Warmup(iterations = 1)
@Measurement(iterations = 2, time = 600, timeUnit = TimeUnit.MILLISECONDS)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@Threads(100)
@State(Scope.Benchmark)
public class LettuceAsyncGetBenchmark {

    private static final String KEY = "a";

    @Param({"1"})
    private int repeat; // number of GETs per benchmark invocation

    private io.lettuce.core.RedisClient redisClient;
    private io.lettuce.core.api.StatefulRedisConnection<String, String> conn;

    @Setup
    public void init() {
        redisClient = io.lettuce.core.RedisClient.create("redis://127.0.0.1:6379");
        conn = redisClient.connect();
    }

    @TearDown
    public void shutdown() {
        if (conn != null) conn.close();
        if (redisClient != null) redisClient.shutdown();
    }

    @Benchmark
    public void asyncGetBatch() throws Exception {
        io.lettuce.core.api.async.RedisAsyncCommands<String, String> async = conn.async();
        java.util.List<io.lettuce.core.RedisFuture<String>> batch = new java.util.ArrayList<>(repeat);

        for (int i = 0; i < repeat; i++) {
            batch.add(async.get(KEY));
        }

        // Wait for all issued GETs to complete
        for (io.lettuce.core.RedisFuture<String> f : batch) {
            f.get();
        }
    }
}

Runner:

public class LettuceRunner {
    public static void main(String[] args) throws Exception {
        Options opt = new OptionsBuilder()
                .include(LettuceAsyncGetBenchmark.class.getSimpleName())
                .output("benchmark/lettuce-async-throughput.log")
                .forks(1)
                .build();
        new Runner(opt).run();
    }
}

Notes on state scoping

• Scope.Thread: instantiates state per worker thread. Appropriate for clients that are not thread-safe (Jedis). Each thread holds its own client instance created in @Setup.
• Scope.Benchmark: a single shared state for all thread. Safe for thread-safe resources (Lettuce connection) and reduces connection overhead during measurement.

Sample results (ops/ms)

• Threads: 100, GETs per invocation: 1
  • Jedis: 46.628
  • Lettuce: 106.589
• Threads: 100, GETs per invocation: 10
  • Jedis: 5.307
  • Lettuce: 14.802
• Threads: 100, GETs per invocation: 100
  • Jedis: 0.483
  • Lettuce: 1.599