Design and implement a thread-safe in-memory cache in Java with the following features

by

Summary

The thread-safe in-memory cache implementation in Java exhibited race conditions and inefficient cleanup, leading to inconsistent data retrieval and performance degradation. The root cause was lack of proper synchronization and inefficient expiration handling.

Root Cause

  • Race conditions in get() and cleanUp() methods due to unsynchronized access to the cache map.
  • Inefficient cleanup caused by iterating over cache.keySet() in cleanUp(), leading to potential blocking and unnecessary iterations.
  • No atomicity in checking and removing expired entries, resulting in stale data being returned.

Why This Happens in Real Systems

  • Concurrency challenges: Multiple threads accessing shared resources without proper synchronization.
  • Time-based operations: TTL expiration requires precise timing and efficient cleanup mechanisms.
  • Performance trade-offs: Balancing between read/write speed and memory management.

Real-World Impact

  • Data inconsistency: Users retrieve expired or incorrect data.
  • Performance degradation: High contention and inefficient cleanup slow down operations.
  • Resource wastage: Stale entries consume memory unnecessarily.

Example or Code (if necessary and relevant)

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicReference;

public class FixedTTLCache {
    private final ConcurrentHashMap<K, AtomicReference> cache = new ConcurrentHashMap();
    private final ScheduledExecutorService cleaner = Executors.newSingleThreadScheduledExecutor();

    private static class CacheItem {
        V value;
        long expiryTime;

        CacheItem(V value, long expiryTime) {
            this.value = value;
            this.expiryTime = expiryTime;
        }
    }

    public FixedTTLCache() {
        cleaner.scheduleAtFixedRate(this::cleanUp, 0, 1, TimeUnit.SECONDS);
    }

    public void put(K key, V value, long ttlMillis) {
        long expiryTime = System.currentTimeMillis() + ttlMillis;
        cache.put(key, new AtomicReference(new CacheItem(value, expiryTime)));
    }

    public V get(K key) {
        AtomicReference ref = cache.get(key);
        if (ref == null) return null;
        CacheItem item = ref.get();
        if (item == null || System.currentTimeMillis() > item.expiryTime) {
            cache.remove(key);
            return null;
        }
        return item.value;
    }

    private void cleanUp() {
        long now = System.currentTimeMillis();
        cache.entrySet().removeIf(entry -> {
            CacheItem item = entry.getValue().get();
            return item != null && now > item.expiryTime;
        });
    }
}

How Senior Engineers Fix It

  • Use ConcurrentHashMap: Ensures thread-safe operations without explicit locks.
  • Atomic references: Use AtomicReference for safe updates to CacheItem.
  • Efficient cleanup: Leverage removeIf() for non-blocking expiration checks.
  • Scheduled executor: Regularly clean up expired entries without blocking reads/writes.

Why Juniors Miss It

  • Underestimating concurrency: Failing to recognize race conditions in multi-threaded environments.
  • Inefficient data structures: Using HashMap instead of ConcurrentHashMap for thread safety.
  • Lack of atomicity: Not ensuring atomic checks and updates for expiration.
  • Overlooking cleanup impact: Ignoring how cleanup logic affects performance and consistency.

Leave a Comment