Designing Scalable Notification Read/Unread States

Broadcasting system alerts to large user bases creates a classic storage bottleneck. A naive approach relies on two tables: a central message registry and a per-user notification log. When broadcasting to millions, this triggers N*M database inserts, inflating storage and degrading query latency.

Two optimized patterns resolve this:

1. Cursor-Based Offset Tracking: Eliminate the heavy notification log. Introduce a lightweight progress table storing each user's last_consumed_msg_id. Retrieving unread items becomes a single indexed scan: SELECT * FROM messages WHERE id > cursor LIMIT N. This reduces broadcast inserts from millions to exactly one.
2. Redis Set Diffusion: Maintain a Set under the key user:read:{userId}. Every acknowledged message ID is added to this set. Unread items are derived by calculating the difference between the global message stream and the consumed set. This approach leverages memory speed and avoids relational table bloat.

Strategies for Distributed ID Generation

Generating globally unique keys across distributed nodes requires balancing throughput, ordering, and storage efficiency. Understanding the underlying storage engine is critical. InnoDB organizes data via B+ trees where leaf nodes contain the actual rows. Sequential primary keys append data linearly to the rightmost leaf. Random inserts trigger page splits: the engine divides an overflowed page, allocates a new block, and updates bidirectional pointers. This rebalancing causes severe I/O penalties.

Algorithm Comparison

• UUID: Zero coordination, high generation speed, but random ordering destroys insert locality and index efficiency.
• Database Auto-Increment: Simple and strictly sequential, but creates a single write bottleneck and exposes business volume.
• Segment-Based Fetching: Clients acquire ID ranges from a central DB, caching them locally until exhaustion. High fault tolerance, but depends heavily on DB I/O during range refreshes.
• Redis Atomic Increment: Leverages single-threaded INCR commands for speed and ordering. Monotonicity remains a security and analytical concern.
• Snowflake Pattern: Combines timestamp, node identifier, and sequence counter in a 64-bit integer. Delivers high throughput, trend-incremental ordering, and zero central coordination.

Refactored Snowflake Implementation

import java.util.concurrent.atomic.AtomicLong;

public class TimestampUniqueIdFactory {
    private static final int NODE_BITS = 10;
    private static final int SEQ_BITS = 12;
    private static final long TIMESTAMP_SHIFT = NODE_BITS + SEQ_BITS;
    private static final long EPOCH = 1700000000000L;
    private static final long MAX_SEQUENCE = ~(-1L << SEQ_BITS);

    private final long nodeId;
    private volatile long lastGeneratedTimestamp;
    private final AtomicLong currentSequence = new AtomicLong(0L);

    public TimestampUniqueIdFactory(long workerId) {
        this.nodeId = workerId & ((1L << NODE_BITS) - 1);
        this.lastGeneratedTimestamp = -1L;
    }

    public synchronized long acquireId() {
        long now = System.currentTimeMillis();
        
        if (now < lastGeneratedTimestamp) {
            throw new IllegalStateException("Clock moved backwards. Refusing to generate ID for " 
                + (lastGeneratedTimestamp - now) + " milliseconds");
        }

        if (now == lastGeneratedTimestamp) {
            long seq = currentSequence.incrementAndGet() & MAX_SEQUENCE;
            if (seq == 0) {
                now = spinUntilNextMillis();
            }
        } else {
            currentSequence.set(0);
        }

        lastGeneratedTimestamp = now;
        return ((now - EPOCH) << TIMESTAMP_SHIFT) 
               | (nodeId << SEQ_BITS) 
               | currentSequence.get();
    }

    private long spinUntilNextMillis() {
        long ts = System.currentTimeMillis();
        while (ts <= lastGeneratedTimestamp) {
            ts = System.currentTimeMillis();
        }
        return ts;
    }
}

Traffic Shaping and Rate Limiting Mechanisms

Effective throttling requires mapping traffic flow across DNS, load balancers, gateways, and microservices. Throttling can be enforced at the infrastructure layer (Nginx/Envoy) or application layer. Core algorithms dictate how traffic is metered:

• Fixed Counter: Simple window counting, but vulnerable to boundary spikes where two consecutive windows except double the allowed load.
• Sliding Window: Divides the timeframe into smaller buckets. Summing active buckets provides a moving average, smoothing boundary anomalies at the cost of higher memory overhead.
• Leaky Bucket: Enforces a strict, constant outflow rate. Excess requests queue or drop. Ideal for smoothing bursty traffic but inflexible.
• Token Bucket: Tokens regenerate at a fixed pace. Requests consume tokens; absence of tokens delays or rejects the call. Permits controlled bursts up to bucket capacity, making it the default for API rate limiting.

Refactored Throttling Implementations

import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.TimeUnit;

public class AdaptiveTokenBucket {
    private final double tokensPerSecond;
    private final double maxCapacity;
    private final AtomicLong storedTokensMicro;
    private volatile long lastRefillMicros;

    public AdaptiveTokenBucket(double rate, double capacity) {
        this.tokensPerSecond = rate;
        this.maxCapacity = capacity;
        this.storedTokensMicro = new AtomicLong(0);
        this.lastRefillMicros = System.nanoTime();
    }

    public boolean tryConsume() {
        return tryConsume(1.0);
    }

    public boolean tryConsume(double weight) {
        refill();
        long current = storedTokensMicro.updateAndGet(val -> 
            Math.max(val - (long)(weight * 1_000_000), 0L));
        return current == 0; // simplified consumption logic
    }

    private void refill() {
        long now = System.nanoTime();
        long deltaNanos = now - lastRefillMicros;
        lastRefillMicros = now;
        
        long generated = (long)((deltaNanos * tokensPerSecond) / 1_000_000_000L * 1_000_000);
        if (generated > 0) {
            storedTokensMicro.updateAndGet(val -> 
                Math.min(val + generated, (long)(maxCapacity * 1_000_000)));
        }
    }
}

Role-Based Access Control Data Modeling

Standardized authorization decouples user identities from operational capabilities. The schema revolves around four core entities:

1. Subject: The user or service account.
2. Role: A named collection of permissions (e.g., ADMIN, AUDITOR).
3. Permission: Granular action-resource pairs (e.g., ORDER:READ, REPORT:EXPORT).
4. Mapping Tables: subject_role and role_permission many-to-many relationships. This structure allows policy updates without schema migrations and supports hierarchical role inheritance.

Architecture for High-Performance URL Shortening

Converting lengthy URIs into compact aliases demands efficient encoding and sub-millisecond lookups. A 64-bit distributed ID, when base-encoded using alphanumeric characters (62 symbols), requires approximately 11 digits too prevent collisions. Truncating to 7 characters still covers trillions of combinations, safely exceeding global URL volumes.

The read path prioritizes cache hits: Redis checks for the alias, returns an HTTP 302 redirect, or falls back to a database query. The write path checks existing mappings, generates a new code via the ID factory, persists it, and warms the cache. Abuse mitigation relies on IP-based request caps and Bloom filters to reject invalid lookups before hitting the DB.

Core Implementation Components

import java.util.HashMap;
import java.util.Map;

public class CompactUrlEncoder {
    private static final char[] ALPHABET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ".toCharArray();
    private static final Map<Character, Integer> REVERSE_INDEX = new HashMap<>(62);

    static {
        for (int i = 0; i < ALPHABET.length; i++) REVERSE_INDEX.put(ALPHABET[i], i);
    }

    public static String encode(long numericId) {
        StringBuilder buffer = new StringBuilder(7);
        if (numericId == 0) return String.valueOf(ALPHABET[0]);
        
        while (numericId > 0) {
            buffer.append(ALPHABET[(int)(numericId % 62)]);
            numericId /= 62;
        }
        return buffer.reverse().toString();
    }

    public static long decode(String code) {
        long result = 0;
        for (char c : code.toCharArray()) {
            result = result * 62 + REVERSE_INDEX.get(c);
        }
        return result;
    }
}

@RestController
@RequestMapping("/links")
@RequiredArgsConstructor
public class LinkResolutionController {
    private final StringRedisTemplate cache;
    private final JdbcTemplate db;

    @GetMapping("/{code}")
    public ResponseEntity<Void> resolve(@PathVariable String code) {
        String target = cache.opsForValue().get("link:" + code);
        if (target == null) {
            target = db.queryForObject("SELECT target_url FROM mappings WHERE code = ?", 
                String.class, code);
            if (target != null) cache.opsForValue().set("link:" + code, target, 12, TimeUnit.HOURS);
        }
        if (target == null) return ResponseEntity.status(404).build();
        HttpHeaders headers = new HttpHeaders();
        headers.setLocation(URI.create(target));
        return new ResponseEntity<>(headers, HttpStatus.FOUND);
    }
}