Fix Pillow Import Errors in AWS Lambda Layers with Docker

by

Summary

Import errors in Lambda layers are usually caused by binary incompatibility between the compiled Pillow wheels and the execution environment. When you build the layer on CloudShell (an Amazon Linux 2 container) but target a Lambda runtime that uses a different glibc version or architecture, the native _imaging extension cannot be loaded, resulting in the “cannot import _imaging from PIL” error.

Root Cause

  • Pillow ships a compiled C extension (_imaging.cpython-*.so).
  • The compiled extension must match exactly the OS libraries and architecture of the Lambda runtime.
  • Building the layer in CloudShell on x86 64 does not guarantee compatibility with the Lambda runtime (Python 3.14, Amazon Linux 2023) because:
    • Different glibc version.
    • Different OpenSSL / libjpeg / libz versions.
    • CloudShell uses a different base image than the Lambda execution environment.

Why This Happens in Real Systems

  • Lambda layers are just zip files; they don’t re‑compile on invocation.
  • Developers often reuse a local or CloudShell build environment assuming it matches Lambda, which is not true for compiled dependencies.
  • The “works locally, fails in Lambda” pattern is common whenever native extensions are involved (Pillow, numpy, pandas, etc.).

Real-World Impact

  • Cold‑start failures: The function never becomes healthy; every invocation crashes with an ImportError.
  • Increased operational cost: Teams spend time debugging environment mismatches rather than fixing business logic.
  • Reduced reliability: End‑users experience intermittent failures if the layer is occasionally rebuilt in a mismatched environment.

Example or Code (if necessary and relevant)

# Build the layer in a Docker container that matches the Lambda runtime
docker run --rm -v "$PWD":/var/task -w /var/task public.ecr.aws/lambda/python:3.14 \
    bash -c "pip install --target python/lib/python3.14/site-packages pillow==10.2.0 && zip -r9 layer.zip python"

How Senior Engineers Fix It

  • Build the layer inside a Docker image that mirrors the Lambda runtime (e.g., public.ecr.aws/lambda/python:3.14).
  • Pin Pillow to a version known to have compatible binary wheels for the target runtime.
  • Strip unnecessary binaries to keep the layer under the 250 MiB limit.
  • Test the layer locally by invoking the Lambda runtime container with the same zip.
  • Automate the build with CI/CD pipelines so every release is reproducible.

Why Juniors Miss It

  • They assume “install‑once‑works‑everywhere” and ignore the need for binary compatibility.
  • Lack of experience with Lambda’s stripped‑down Amazon Linux environment versus a full‑featured development VM.
  • Tend to focus on Python code and overlook the underlying C libraries that Pillow depends on.

Leave a Comment