Implementing Distributed Tracing with Spring Cloud Sleuth and Zipkin

In microservices architecture, applications are typically composed of numerous services that interact with each other. When issues arise, identifying the root cause can be challenging due to the complex web of service interactions. Distributed tracing systems address this challenge by providing visibility into request flows across services.

Understanding Distributed Tracing

Distributed tracing is a method used to monitor and debug requests as they travel through distributed systems. Each request is assigned a unique identifier, allowing systems to track the path of the request as it moves between services.

The theoretical foundation for modern distributed tracing systems originates from Google's 2010 paper "Dapper, a Large-Scale Distributed Systems Tracing Infrastructure." Twitter's Zipkin emerged as the most widely adopted open-source implementation based on this research. To ensure platform- and vendor-agnostic tracing, the CNCF (Cloud Native Computing Foundation) introduced the OpenTracing standard.

Spring Cloud Sleuth Overview

Spring Cloud Sleuth is a component of the Spring Cloud ecosystem that provides distributed tracing capabilities for Spring-based applications. It integrates seamlessly with Spring Boot and Spring Cloud, enabling developers to monitor request flows across microservices.

Core Concepts

In distributed tracing terminology, a complete request flow is referred to as a trace. Each trace consists of multiple spans, which represent individual operations or service calls within the trace. Spans contain metadata such as timestamps, duration, and key-value annotations that provide context about the operation.

Benefits of Using Sleuth

Implementing Spring Cloud Sleuth in your microservices architecture provides several advantages:

• Request Flow Visualization: Clear understanding of how requests propagate through your system
• Performance Analysis: Identify which service calls consume the most time
• Error Tracking: Locate unhandled exceptions across service boundaries
• Dependency Mapping: Visualize relationships between services

Zipkin: The Tracing Backend

Zipkin is an open-source distributed tracing system originally developed by Twitter. It collects timing data to help identify performance issues in microservice architectures. Zipkin consists of four main components:

• Collector: Receives tracing data from instrumentation libraries and converts it to Zipkin's internal format
• Storage: Persists trace data, with options including in-memory, MySQL, Cassandra, and Elasticsearch
• REST API: Provides interfaces for querying and managing trace data
• Web UI: Offers a user-friendly interface for searching and analyzing traces

Setting Up Distributed Tracing

Prerequisites

To implement distributed tracing with Sleuth and Zipkin, you'll need:

• A service registry (Eureka in this example)
• A Zipkin server for collecting and displaying trace data
• Client applications instrumented with Sleuth

Method 1: HTTP-based Data Collection

Deploying Zipkin Server

The recommended approach for deploying Zipkin with Spring Boot 2.x is to use the pre-built jar package rather than custom compilation. The @EnableZipkinServer annotation has been deprecated in favor of using the official distribution.

To start Zipkin using the official jar:

curl -sSL https://zipkin.io/quickstart.sh | bash -s
java -jar zipkin.jar

Alternatively, you can run Zipkin using Docker:

docker run -d -p 9411:9411 openzipkin/zipkin

Once started, access the Zipkin UI at http://localhost:9411/

Configuring Client Applications

Let's configure two client applications: a service provider (product-service) and an API gateway (api-gateway).

Service Provider Configuration

Add the following dependencies to your service provider's pom.xml:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-zipkin</artifactId>
</dependency>

Configure your application.yml:

server:
  port: 8081
spring:
  application:
    name: product-service
  sleuth:
    sampler:
      probability: 1.0
  zipkin:
    base-url: http://localhost:9411
eureka:
  client:
    service-url:
      defaultZone: http://localhost:8761/eureka/

Create a simple controlller to expose an endpoint:

@RestController
@RequestMapping("/products")
public class ProductController {

    @GetMapping("/{id}")
    public Product getProduct(@PathVariable String id) {
        return new Product(id, "Sample Product", 19.99);
    }
}

API Gateway Configuration

Add the following dependencies to your API gateway's pom.xml:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-zuul</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-zipkin</artifactId>
</dependency>

Configure your application.yml:

server:
  port: 8080
spring:
  application:
    name: api-gateway
  sleuth:
    sampler:
      probability: 1.0
  zipkin:
    base-url: http://localhost:9411
eureka:
  client:
    service-url:
      defaultZone: http://localhost:8761/eureka/
zuul:
  routes:
    product-service:
      path: /products/**
      serviceId: product-service

Enable Zuul proxy in your main application class:

@SpringBootApplication
@EnableZuulProxy
public class ApiGatewayApplication {
    public static void main(String[] args) {
        SpringApplication.run(ApiGatewayApplication.class, args);
    }
}

Testing the Setup

Start the Zipkin server, followed by the Eureka server, product-service, and api-gateway. Make a request to the API gateway:

curl http://localhost:8080/products/123

Check the Zipkin UI at http://localhost:9411/ to view the trace data. You should see the request flow from the API gateway to the product service.

Method 2: RabbitMQ-based Data Collection

For high-volume systems, you can use RabbitMQ as a message broker to transmit trace data to Zipkin. This approach decouples the client applications from the Zipkin server, improving scalability.

Configuring Zipkin with RabbitMQ

Start Zipkin with RabbitMQ support:

java -jar zipkin.jar --zipkin.collector.rabbitmq.addressed=localhost

Ensure your RabbitMQ instance is running with default credentials (guest/guest) or modify the command accordingly if you're using different credentials.

Configuring Client Applications

Add the Spring Cloud Stream RabbitMQ binder to both client applications:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream-binder-rabbit</artifactId>
</dependency>

Update the application.yml files to remove the direct Zipkin URL (since data will be sent via RabbitMQ):

spring:
  sleuth:
    sampler:
      probability: 1.0
  zipkin:
    rabbitmq:
      addresses: localhost
      username: guest
      password: guest

Restart the client applications and make test requests. The trace data will be sent to Zipkin via RabbitMQ instead of direct HTTP calls.