Mapping Numbered API Fields to Kotlin Lists with kotlinx Serialization

by

Summary

When consuming APIs that expose numbered fields like strIngredient1 through strIngredient20, the naive loop approach works but is fragile and non-idiomatic. A better pattern uses a custom @Serializable wrapper class with a @Polymorphic or @SerialName-driven iteration strategy, or a single-object mapping + post-processing step that converts the flat keyed structure into a typed list. The core issue is that the JSON schema is not a list but a record of indexed keys, and Kotlinx Serialization expects schema alignment.

Root Cause

The root cause is a mismatch between the API’s serialization schema and the consumer’s desired data model.

  • The API returns a flat object with keys like strIngredient1, strIngredient2, … strIngredientN
  • Kotlinx Serialization maps JSON to Kotlin data classes by field name matching, not by pattern matching on key suffixes
  • There is no built-in deserializer that can iterate over keys ending in a number and collect them into a list
  • The common for (i in 1..20) loop is essentially manual deserialization, which bypasses the compiler’s type safety guarantees

Key takeaway: The serializer cannot infer a list from keys that differ only by an index suffix. You must either reshape the JSON or write a custom mapping step.

Why This Happens in Real Systems

  • Many public APIs (especially older ones) model optional/variable-length lists as fixed-width keyed records to avoid schema changes
  • The API designers chose numbered fields to signal “up to N items” without using an array, often for database serialization simplicity
  • Consumers then face a choice: deserialize the flat object and post-process, or write a custom serializer
  • Kotlinx Serialization is strictly name-based by default, so pattern-based key extraction is not a first-class feature

Common real-world scenarios:

  • REST APIs with legacy schemas
  • CSV-style key naming in microservices
  • Configuration files with numbered sections

Real-World Impact

  • Maintenance burden: Hardcoded ranges (1..20) break when the API changes the max count
  • Type safety loss: Manual JSON key access (json["strIngredient$i"]) returns JsonElement, bypassing Kotlin types
  • Test fragility: Unit tests must mock the full flat object, not a clean list
  • Onboarding friction: New developers see the loop and think it is the only way

Bullet impacts:

  • Bugs when the API adds strIngredient21 but the loop still stops at 20
  • Null-handling inconsistency across indices
  • Difficulty composing the ingredient-measure pairs correctly when one side is null

Example or Code (if necessary and relevant)

import kotlinx.serialization.*
import kotlinx.serialization.json.*

@Serializable
data class MealResponse(
    val strIngredient1: String? = null,
    val strIngredient2: String? = null,
    val strIngredient3: String? = null,
    val strIngredient4: String? = null,
    val strIngredient5: String? = null,
    val strMeasure1: String? = null,
    val strMeasure2: String? = null,
    val strMeasure3: String? = null,
    val strMeasure4: String? = null,
    val strMeasure5: String? = null,
)

fun MealResponse.toIngredients(): List {
    val maxCount = maxOf(
        strIngredient1, strIngredient2, strIngredient3, strIngredient4, strIngredient5
    ).let { list ->
        list.count { it != null }
    }
    return (1..maxCount).map { i ->
        Ingredient(
            name = this["strIngredient$i"] as? String,
            measure = this["strMeasure$i"] as? String,
        )
    }
}
data class Ingredient(
    val name: String?,
    val measure: String?
)

fun MealResponse.toIngredients(): List {
    val pairs = (1..5).mapNotNull { i ->
        val name = when (i) {
            1 -> strIngredient1
            2 -> strIngredient2
            3 -> strIngredient3
            4 -> strIngredient4
            5 -> strIngredient5
            else -> null
        }
        val measure = when (i) {
            1 -> strMeasure1
            2 -> strMeasure2
            3 -> strMeasure3
            4 -> strMeasure4
            5 -> strMeasure5
            else -> null
        }
        if (name != null || measure != null) Ingredient(name, measure) else null
    }
    return pairs
}
// Idiomatic post-processing using reflection-friendly iteration
fun MealResponse.toIngredients(): List {
    val max = 20
    return (1..max)
        .map { i ->
            Ingredient(
                name = this["strIngredient$i"] as? String,
                measure = this["strMeasure$i"] as? String,
            )
        }
        .takeWhile { it.name != null || it.measure != null }
}

How Senior Engineers Fix It

Senior engineers apply a two-step pipeline: deserialize to the flat shape, then transform to the desired list shape. They never try to force Kotlinx Serialization to handle dynamic key patterns directly.

Best practices:

  • Deserialize to the API shape first — accept the flat object as the source of truth
  • Write a small, pure function that converts the flat object to a list of domain objects
  • Determine the true max index by scanning keys or using a sentinel like strIngredient without a number to signal end
  • Keep the mapping function pure and testable — it takes the deserialized object and returns a List<Ingredient>
  • Consider a custom decoder only when the mapping is reused across many endpoints; otherwise the simple function is cleaner

Example clean mapping:

fun MealResponse.toIngredients(): List {
    return generateSequence(1) { it + 1 }
        .mapNotNull { i ->
            val name = when (i) {
                1 -> strIngredient1
                2 -> strIngredient2
                3 -> strIngredient3
                4 -> strIngredient4
                5 -> strIngredient5
                else -> null
            }
            val measure = when (i) {
                1 -> strMeasure1
                2 -> strMeasure2
                3 -> strMeasure3
                4 -> strMeasure4
                5 -> strMeasure5
                else -> null
            }
            if (name != null || measure != null) Ingredient(name, measure) else null
        }
        .toList()
}

The key takeaway is that Kotlinx Serialization is not the right tool for dynamic key pattern matching — use it for the known shape, then map.

Why Juniors Miss It

  • Juniors reach for the manual loop over string indices because it is the first thing that “works”
  • They are unaware that custom serializers exist but are overkill for this pattern
  • They conflate deserialization (parsing JSON into Kotlin objects) with domain mapping (reshaping data into the model you actually use)
  • They hardcode the range and never consider what happens when the API changes
  • They do not realize that keeping deserialization and mapping in separate functions makes the codebase easier to test and reason about

The deeper lesson: Recognize when the problem is a schema mismatch, not a serialization problem. Fix the mismatch in a dedicated mapping layer, not inside the serializer.

Leave a Comment