UK Met Office migrating from DataPoint to DataHub

by

Summary

This postmortem examines a common migration pitfall: moving from the Met Office’s legacy DataPoint API to the newer DataHub API and expecting identical data structures. The issue arises when teams assume that symbolic weather codes, descriptive strings, or visibility categories will be preserved across API generations. They are not — and this mismatch leads to confusion, broken integrations, and missing features.

Root Cause

The failure stems from incorrect assumptions about backward compatibility between DataPoint and DataHub.

Key factors:

  • DataHub exposes a different schema focused on numerical, machine‑readable fields.
  • Legacy weather-condition text codes (e.g., “Clear night”, “Sunny day”) are not included in DataHub responses.
  • Visibility categories (e.g., VP, PO, MO, GO) are also not part of the new API’s default dataset.
  • DataHub requires developers to map weather codes themselves using the documentation’s lookup tables.

Why This Happens in Real Systems

Real-world API migrations often break expectations because:

  • New platforms prioritize standardization over legacy convenience fields.
  • Human-readable strings are brittle, language-specific, and hard to maintain.
  • Modern APIs favor numeric codes that downstream systems can interpret consistently.
  • Backward compatibility is expensive, and legacy fields are often removed to simplify the data model.

Real-World Impact

Teams experience:

  • Missing weather descriptions that previously powered dashboards or UI labels.
  • Broken integrations where code expects fields that no longer exist.
  • Incorrect assumptions that DataHub is “missing data” when it is simply structured differently.
  • Increased engineering effort to rebuild mapping layers that DataPoint once provided.

Example or Code (if necessary and relevant)

Below is an example of how a team might rebuild the missing mapping layer using their own lookup table:

WEATHER_CODES = {
    0: "Clear night",
    1: "Sunny day",
    2: "Partly cloudy (night)",
    3: "Partly cloudy (day)",
    5: "Mist",
    6: "Fog",
    7: "Cloudy",
    8: "Overcast",
    9: "Light rain shower (night)",
    10: "Light rain shower (day)",
    11: "Drizzle",
    12: "Light rain",
    13: "Heavy rain shower (night)",
    14: "Heavy rain shower (day)",
    15: "Heavy rain",
    16: "Sleet shower (night)",
    17: "Sleet shower (day)",
    18: "Sleet",
    19: "Hail shower (night)",
    20: "Hail shower (day)",
    21: "Hail",
    22: "Light snow shower (night)",
    23: "Light snow shower (day)",
    24: "Light snow",
    25: "Heavy snow shower (night)",
    26: "Heavy snow shower (day)",
    27: "Heavy snow",
    28: "Thunder shower (night)",
    29: "Thunder shower (day)",
    30: "Thunder"
}

How Senior Engineers Fix It

Experienced engineers approach this by:

  • Reading the new API schema carefully rather than assuming parity with the old one.
  • Building a translation layer that maps DataHub’s numeric codes to human-readable strings.
  • Versioning their internal data models to avoid breaking downstream consumers.
  • Documenting the mapping logic so future migrations are painless.
  • Automating schema validation to detect missing or changed fields early.

Why Juniors Miss It

Less experienced engineers often:

  • Assume APIs evolve additively, not subtractively.
  • Rely on legacy examples without checking updated documentation.
  • Expect human-readable fields to be included by default.
  • Skip schema diffing, leading to silent failures.
  • Underestimate the importance of data contracts, especially during migrations.

This incident is a classic example of why API migrations require careful schema analysis, not blind substitution.

Leave a Comment