How to read array size from over-allocated memory block?

by

Summary

The question revolves around understanding how delete[] arr knows the size of arr when it’s allocated using new[], and how to read the size of an array or memory block from a pointer when over-allocation is used. Key takeaway: The size of the array is stored in the memory block before the actual array data.

Root Cause

The root cause of the confusion lies in the implementation details of new[] and delete[] in C++. When using new[], the compiler allocates extra memory to store the size of the array. The reasons for this include:

  • Storing the size of the array for later deallocation
  • Alignment requirements for the memory block
  • Over-allocation to reduce the number of allocations and improve performance

Why This Happens in Real Systems

In real systems, this happens due to the way memory allocation works:

  • Memory blocks have a minimum size, which can lead to over-allocation
  • Alignment requirements can cause the actual allocated memory to be larger than the requested size
  • Implementation details of the memory allocator can affect the layout of the memory block

Real-World Impact

The real-world impact of this includes:

  • Memory waste due to over-allocation
  • Performance improvements due to reduced number of allocations
  • Debugging challenges when trying to understand memory allocation and deallocation

Example or Code

#include 

int main() {
    int* arr = new int[10];
    std::cout << "Array address: " << arr << std::endl;
    std::cout << "Size stored before array: " << *(size_t*)(arr - sizeof(size_t)) << std::endl;
    delete[] arr;
    return 0;
}

How Senior Engineers Fix It

Senior engineers fix this by:

  • Understanding the implementation details of the memory allocator
  • Using debugging tools to inspect the memory layout
  • Writing custom memory allocators to optimize memory usage and performance
  • Using smart pointers to manage memory and avoid manual deallocation

Why Juniors Miss It

Juniors miss this because:

  • Lack of understanding of the implementation details of new[] and delete[]
  • Insufficient experience with debugging and memory allocation
  • Overreliance on high-level abstractions without understanding the underlying details
  • Not considering the over-allocation and alignment requirements** when working with memory allocation