Avoid Firebase N+1 URL Calls by Denormalizing at Write Time

by

Summary

The system architecture relied on storing Firebase Storage paths within Firestore documents rather than signed Download URLs. While this approach maintained a “source of truth” for file locations, it introduced a critical latency bottleneck during the initial application boot and chat list rendering. Because getDownloadURL() is an asynchronous network request, the UI was forced to await multiple Future resolutions before displaying media, leading to a poor user experience characterized by flickering, loading spinners, and high API overhead.

Root Cause

The performance degradation stemmed from a mismatch between data retrieval patterns and asynchronous execution models:

  • Asynchronous Waterfall: To render a single chat message, the app first fetched the Firestore document, then initiated a secondary network request to Firebase Storage to resolve the path into a URL.
  • Sequential Dependencies: The UI could not determine the ImageProvider source until the Future<String> from the Storage SDK resolved, creating a “waterfall” effect where the UI was blocked by the speed of the Storage API.
  • Redundant Network Roundtrips: Every time a user scrolled or refreshed, the client performed N additional calls to the Firebase Storage API just to retrieve metadata that had not changed.
  • Lack of Client-Side Memoization: Storing the path required the client to re-calculate the URL constantly rather than fetching a static, cacheable asset.

Why This Happens in Real Systems

In large-scale distributed systems, this is known as the N+1 Query Problem, applied to cloud storage rather than databases.

  • Normalized vs. Denormalized Data: Developers often favor normalized data (storing a path/ID) to ensure data integrity and minimize storage costs. However, in high-read environments, denormalization (storing the ready-to-use URL) is required for performance.
  • Abstraction Leaks: Firebase SDKs provide high-level abstractions like getDownloadURL() that hide the cost of the underlying HTTP request. Developers often treat these as “cheap” local functions when they are actually remote procedure calls (RPCs).
  • The “Single Source of Truth” Fallacy: There is a misconception that storing the path is “safer.” In reality, if the file is public or has a predictable token, the URL is simply a different representation of the same data.

Real-World Impact

  • Increased Time to Interactive (TTI): Users experience a significant delay between the app loading and the content becoming usable.
  • Increased Operational Costs: Every call to getDownloadURL() counts as a Firebase Storage API request, which can lead to higher costs at scale compared to simple GET requests for static assets.
  • Poor Perceived Performance: Even if the actual data transfer is fast, the “pop-in” effect of images loading one-by-one destroys the user’s sense of a “premium” interface.
  • Browser Cache Inefficiency: When using ephemeral URLs or performing repeated resolution calls, the browser’s ability to leverage HTTP cache headers is often bypassed or neutralized.

Example or Code (if necessary and relevant)

// BAD: The N+1 Approach (Causes latency and flickering)
FutureBuilder(
  future: FirebaseStorage.instance.ref(pathFromFirestore).getDownloadURL(),
  builder: (context, snapshot) {
    if (snapshot.hasData) {
      return Image.network(snapshot.data!);
    }
    return CircularProgressIndicator();
  },
)

// GOOD: The Denormalized Approach (Instant rendering)
// Firestore Document Structure:
// {
//   "message": "Hello!",
//   "mediaUrl": "https://firebasestorage.googleapis.com/.../image.png?token=xyz"
// }
Image.network(
  messageDoc['mediaUrl'],
  cacheWidth: 300, // Optimization for Web/Mobile
)

How Senior Engineers Fix It

Senior engineers move the computational and network cost from the Client-side (Runtime) to the Server-side (Write-time).

  • Denormalization at Write-time: Modify the Cloud Function or backend logic that uploads the file. Once the upload is successful, fetch the DownloadURL immediately and write that URL directly into the Firestore document.
  • Edge Caching: Use the direct URL to allow the browser’s native cache and CDNs to handle the heavy lifting.
  • Pre-computation: Instead of resolving paths on the fly, treat the mediaUrl as a permanent piece of metadata for that specific message.
  • Optimized Image Loading: Use CachedNetworkImage (for mobile) or optimized Image.network parameters (for web) to manage memory footprint and prevent re-downloading assets.

Why Juniors Miss It

  • Focus on Correctness over Performance: Juniors tend to write code that is “correct” (it retrieves the right file) without considering the complexity of the execution path.
  • Misunderstanding Asynchronicity: A junior may see getDownloadURL() as a simple getter rather than a high-latency network operation.
  • Over-Normalization: There is a tendency to follow strict database normalization rules learned in academic settings, failing to realize that Read-Heavy applications require Denormalization.
  • Neglecting the “Perceived” UX: Juniors often focus on whether the image eventually appears, whereas seniors focus on how the image appears to the user.

Leave a Comment