std::pmr monotonic buffer and pool combo leads to bad_alloc

by

Summary

The system experienced a std::bad_alloc crash during a high-frequency loop utilizing Polymorphic Memory Resources (PMR). The engineer attempted to combine a std::pmr::unsynchronized_pool_resource with a std::pmr::monotonic_buffer_resource to create a high-performance, recyclable memory arena. The expectation was that the pool would recycle memory blocks, preventing the monotonic buffer from exhausting its preallocated space. However, the system crashed once the monotonic buffer’s capacity was reached, proving that memory was not being recycled as expected.

Root Cause

The root cause is a fundamental misunderstanding of how std::pmr::monotonic_buffer_resource interacts with std::pmr::unsynchronized_pool_resource.

  • Monotonic Nature: A monotonic_buffer_resource never releases memory until the resource itself is destroyed. It only increments a pointer.
  • Pool Behavior: While a pool_resource manages blocks of memory and “returns” them to its own internal free list, it only returns memory to the upstream resource (the monotonic buffer) when the pool itself is destroyed.
  • The Leak Path: When the std::pmr::vector is destroyed at the end of the loop, it returns memory to the pool_resource. The pool marks that block as “available,” but it does not call do_deallocate on the monotonic buffer.
  • Capacity Exhaustion: When the pool needs a new chunk of memory (either because the requested size exceeds the current pool blocks or the current pool blocks are exhausted), it requests a large slab from the monotonic buffer. Since the monotonic buffer never reclaims space, it eventually hits the end of the provided char buffer[8094].
  • Null Resource Crash: Because the monotonic buffer was configured with std::pmr::null_memory_resource() as its upstream, it cannot request more memory from the heap once the initial buffer is full, resulting in an immediate std::bad_alloc.

Why This Happens in Real Systems

This pattern typically emerges when developers attempt to optimize latency-critical paths (like game loops or HFT engines) by avoiding the global heap.

  • Abstraction Leak: Developers assume that because a pool “recycles,” the underlying memory source also “recycles.”
  • Incorrect Layering: Mixing a “grow-only” allocator (monotonic) with a “recycling” allocator (pool) creates a system that is only as sustainable as the largest possible peak memory usage multiplied by the number of times the pool needs to expand its internal slabs.
  • Hidden Slab Allocation: Pool resources do not allocate exactly the bytes requested; they allocate slabs (larger chunks) to reduce fragmentation. This accelerates the exhaustion of the monotonic buffer.

Real-World Impact

  • Deterministic Crashes: The application works during short smoke tests but crashes in production after a specific number of iterations (a “time bomb” bug).
  • Memory Fragmentation: In systems without a null_memory_resource, this leads to massive memory bloat as the monotonic buffer continues to grow despite the pool thinking it is recycling memory.
  • Performance Degradation: As the pool requests larger and larger slabs to satisfy alignment or size requirements, the overhead of managing these slabs increases until the system fails.

Example or Code

alignas(int) char buffer[8094];
// This resource NEVER releases memory to its upstream until destruction
std::pmr::monotonic_buffer_resource monotonic_mem(buffer, sizeof(buffer), std::pmr::null_memory_resource());

// The pool manages blocks, but only requests NEW slabs from monotonic_mem
std::pmr::unsynchronized_pool_resource pool({32, 64}, &monotonic_mem);

// Vector returns memory to 'pool', NOT to 'monotonic_mem'
std::pmr::vector vec(64, &pool);

How Senior Engineers Fix It

To fix this, the memory lifecycle must be aligned with the resource lifecycle. Senior engineers use one of two patterns:

  1. Scope-Based Reset: If the memory is only needed for one iteration, the monotonic_buffer_resource should be declared inside the loop. This ensures that all memory is reclaimed at the end of every iteration.
  2. Heap-Backed Pool: If the pool must persist across iterations, use std::pmr::new_delete_resource() as the upstream. This allows the pool to eventually return memory to the OS if the pool is destroyed or redesigned.
  3. Custom Reset Logic: For extreme performance, use a custom arena allocator that implements a reset() method to move the pointer back to the start of the buffer manually, though this is not provided by the standard monotonic_buffer_resource.

Why Juniors Miss It

  • Assuming “Delete” means “Free”: Juniors often assume that calling a destructor or deallocate always leads back to the original memory source. They overlook that monotonic_buffer_resource explicitly ignores deallocation requests.
  • Ignoring Upstream Chains: They treat the memory resource as a single entity rather than a chain of responsibility. They fail to trace the path: Vector $\rightarrow$ Pool $\rightarrow$ Monotonic $\rightarrow$ Null.
  • Confusion over Pool Slabs: They assume the pool asks for exactly 64 bytes, whereas the pool actually asks for a large “slab” of memory (e.g., 4KB or 8KB) to slice into 64-byte chunks. One single pool expansion can wipe out the entire preallocated buffer.

Leave a Comment