Docker volume masking wipes custom WordPress theme during core updates in produc

by

Summary

A production WordPress instance experienced data loss of custom application code during an automated core update. The issue stems from a conflict between Docker Named Volumes and Bind Mounts. While the engineer intended to persist plugins and uploads via a named volume, the custom theme—which was also part of the container image—was effectively “shadowed” and then overwritten or wiped due to how Docker manages volume initialization and mounting precedence.

Root Cause

The failure is caused by Volume Masking and the lifecycle of Docker Named Volumes:

  • Named Volume Initialization: When a named volume (like wp_content_seaf_data) is first created, Docker populates it with the contents of the image directory (/var/www/html/wp-content).
  • The Masking Effect: By mounting wp_content_seaf_data to /var/www/html/wp-content, the engineer effectively hid everything inside that directory in the original image (including the custom-theme folder) and replaced it with the volume’s contents.
  • Update Collision: When WordPress performs an auto-update, it modifies files within wp-content. Because the named volume is mounted at the parent level of the theme, the update process or the container restart sequence causes a mismatch between the Image Layer (which contains the code) and the Volume Layer (which contains the data).
  • Bind Mount Shadowing: The attempt to fix this using a bind mount (./custom-theme:/var/www/html/wp-content/themes/custom-theme:ro) failed because the parent directory (wp-content) was already being controlled by a named volume, leading to inconsistent file system states during container lifecycle events.

Why This Happens in Real Systems

In complex containerized environments, this happens due to a misunderstanding of the Docker Mount Hierarchy:

  • Layered Filesystems: Developers often forget that a mount at a high level (e.g., /var/www/html/wp-content) completely obscures the contents of the underlying image at that path.
  • State vs. Code Confusion: In a mature CI/CD pipeline, Code (themes/plugins) should be immutable and part of the image, while State (uploads/database) should be in volumes. This configuration attempted to treat a directory containing both as a single unit of state.
  • Automated Lifecycle Events: Orchestrators like Kubernetes or Docker Compose frequently restart containers. If a volume is re-initialized or if the container engine handles the mount order strictly, the “shadowed” files appear to vanish.

Real-World Impact

  • Service Downtime: The website fails to render because the theme files are missing, leading to 500 Internal Server Errors.
  • Deployment Fragility: Updates that should be seamless become destructive events, requiring manual intervention to restore code from Git.
  • Configuration Drift: The running container no longer matches the Dockerfile definition, making the environment non-reproducible.

Example or Code

To fix this, we must separate Immutable Code from Mutable Data. The theme should remain in the image, and only the specific sub-directories requiring persistence should be mounted.

services:
  wordpress:
    build:
      context: .
      dockerfile: Dockerfile
    volumes:
      # Mount ONLY the specific directories that need persistence
      - wp_uploads:/var/www/html/wp-content/uploads
      - wp_plugins:/var/www/html/wp-content/plugins
      # DO NOT mount wp-content as a whole if you want the image's themes to work
    networks:
      - seaf-backend-wrwmqu

volumes:
  wp_uploads:
  wp_plugins:
FROM wordpress:latest

# Install necessary modules
RUN a2enmod rewrite headers

# Copy the custom theme into the image
# This makes the theme part of the immutable application layer
COPY ./custom-theme /var/www/html/wp-content/themes/custom-theme

# Copy config
COPY uploads.ini /usr/local/etc/php/conf.d/uploads.ini

How Senior Engineers Fix It

Senior engineers follow the principle of Separation of Concerns:

  • Immutable Infrastructure: They treat the container image as the “Source of Truth” for application logic (Themes, Plugins, Core Code).
  • Granular Volume Mapping: Instead of mounting /var/www/html/wp-content, they mount specific sub-paths (e.g., /wp-content/uploads) to ensure the container’s internal directory structure remains intact.
  • Environment Parity: They ensure that the Dockerfile contains everything needed to run the app from scratch, so that a docker compose up without any volumes would still result in a functional (though non-persistent) website.
  • Layer Optimization: They structure Dockerfiles to leverage layer caching, placing frequent changes (like code) above infrequent changes (like OS dependencies).

Why Juniors Miss It

  • The “Everything is State” Fallacy: Juniors often view the entire application folder as “data” that needs to be saved, not realizing that code is part of the “environment.”
  • Surface-Level Learning: They learn how to use volumes but not the underlying mechanics of how Docker overlays a mount onto a filesystem layer.
  • Debugging the Symptom, Not the Cause: When files disappear, a junior might try to add more bind mounts to “force” the files back, which actually complicates the mount hierarchy and creates more unpredictable behavior.

Leave a Comment