How to exclude Pydantic fields from deserialisation?

by

Summary

This postmortem examines a subtle Pydantic behavior: fields cannot be excluded from deserialization while still having a public name, and common mechanisms like PrivateAttr or Field(exclude=True) do not solve the problem. The issue arises when engineers expect Pydantic to treat certain fields as write‑protected during input parsing.

Root Cause

The root cause is that Pydantic does not support “public but non-deserializable” fields.
Pydantic’s model parsing pipeline always attempts to populate declared fields unless:

  • The field is marked as a private attribute, which forces a leading underscore
  • The field is declared with Field(exclude=True), which only affects serialization, not deserialization
  • The field is removed from the model schema entirely

Because none of these options allow a public, non-deserializable field, developers hit a design limitation.

Why This Happens in Real Systems

Real systems often need fields that:

  • Have defaults but must not be overridden by user input
  • Act as internal state, but still need a public name for readability or API exposure
  • Should raise errors if clients attempt to set them
  • Must remain visible in serialization, even if not writable

Pydantic’s design optimizes for predictable data validation, not for enforcing write-protection semantics.

Real-World Impact

This limitation can cause:

  • Silent overwrites of internal fields when user input unexpectedly includes them
  • Security issues, if clients can set fields meant to be system-controlled
  • Confusing behavior, where developers assume exclude=True blocks input but it does not
  • Inconsistent state, when defaults are bypassed by external data

Example or Code (if necessary and relevant)

A common workaround is to override model_validate (Pydantic v2) or __init__ (v1) to manually strip or reject fields.

from pydantic import BaseModel, ValidationError

class MyModel(BaseModel):
    internal_flag: bool = False

    @classmethod
    def model_validate(cls, data):
        if "internal_flag" in data:
            raise ValidationError("internal_flag cannot be provided by clients")
        return super().model_validate(data)

How Senior Engineers Fix It

Experienced engineers typically solve this by:

  • Overriding validation to enforce write-protection
  • Using computed fields when the value is derived, not stored
  • Using private attributes and exposing public read-only properties
  • Implementing custom root validators to strip or reject forbidden fields
  • Documenting API contracts so clients know which fields are system-managed

They recognize that Pydantic’s defaults are not designed for this pattern and compensate accordingly.

Why Juniors Miss It

Junior engineers often miss this issue because:

  • They assume exclude=True applies to both serialization and deserialization
  • They expect Pydantic to enforce immutability or write-protection automatically
  • They do not yet understand the distinction between model schema, serialization, and input parsing
  • They rely on field definitions alone without inspecting the validation pipeline

The result is a subtle but impactful misunderstanding of how Pydantic treats incoming data.

Leave a Comment