Fixing Stack Overflow Flaky Karma Tests on Jenkins via Cleanup

by

Summary

We encountered an intermittent, non-deterministic failure in our CI/CD pipeline (Jenkins) where Karma tests would fail with a RangeError: Maximum call stack size exceeded at Array.sort. This error was highly elusive, appearing only once in hundreds of test runs. While the application logic appeared sound during manual QA and local development, the flaky test behavior in the headless Chrome environment suggested a race condition or leaked state between test suites rather than a logic error in the production code.

Root Cause

The investigation revealed that the error was not caused by actual deep recursion in the application logic, but by asynchronous state leakage and uncontrolled event listener accumulation within the testing environment.

  • Global Event Pollution: The $(window).on("resize") listener was being attached to the global window object. While afterEach attempted to clean up, the asynchronous nature of setTimeout and jasmine.Clock.tick meant that some timers were still scheduled to fire after the test context had already been torn down.
  • Mock Clock Desynchronization: Using jasmine.Clock.useMock() manipulates the passage of time. If a test triggers a setTimeout and the clock is advanced, but the cleanup doesn’t account for pending tasks, the next test suite inherits a “dirty” environment.
  • The Array.sort Red Herring: The error occurring at Array.sort is a classic symptom of a corrupted execution context. When a callback from a previous, “dead” test fires during the execution of a new test, it can interfere with the internal state of the JavaScript engine or the DOM, leading to stack exhaustion when the engine attempts to process interrupted or circular references in the event loop.

Why This Happens in Real Systems

In modern web development, non-deterministic failures (Flakiness) usually stem from the gap between a clean execution environment and the reality of shared global resources.

  • Shared Singletons: In testing, the window, document, and process objects are singletons. If one test modifies them and fails to revert them perfectly, it poisons all subsequent tests.
  • Event Loop Latency: In CI environments (like Jenkins), CPU contention is higher than on a developer’s MacBook. This changes the timing of microtasks and macrotasks, making race conditions much more likely to manifest.
  • Memory Leaks: Small leaks in event listeners or timers don’t crash a browser during a single session, but in a test runner that executes 1,000 tests in one process, they accumulate until the stack or heap is exhausted.

Real-World Impact

  • Developer Friction: Engineers lose trust in the CI pipeline. When tests fail “randomly,” developers start ignoring failures, which is the precursor to missing real bugs.
  • Deployment Delays: Flaky tests block the merge queue, requiring manual restarts of Jenkins jobs, wasting expensive compute time and human engineering hours.
  • False Negatives: A “Maximum call stack size exceeded” error might be ignored as a “glitch,” masking a genuine recursion bug that could crash the production application under heavy load.

Example or Code

The following snippet demonstrates the dangerous pattern found in the controller, specifically how the setTimeout creates a closure that survives the test lifecycle.

_resizeFix: function() {
    var self = this;
    if (this._fixApplied) return;

    // DANGER: This attaches to a global singleton (window)
    $(window).off("resize.myElement").on("resize.myElement", function() {
        clearTimeout(waitTimer);

        // This timer lives in the macrotask queue
        waitTimer = setTimeout(function() {
            self.renderFunction();

            // This second timer creates a massive window for leakage
            setTimeout(function() {
                isInProgress = false;
            }, 1000);
        }, 200);
    });
    this._fixApplied = true;
}

How Senior Engineers Fix It

A senior engineer doesn’t just “reduce the clock tick”; they address the architectural weakness in the test isolation.

  1. Strict Cleanup (The “Janitor” Pattern): Ensure that every single on() has a corresponding off() in afterEach, and specifically use jasmine.clock().uninstall() to prevent clock pollution.
  2. Dependency Injection: Instead of relying on the global window, pass an eventTarget to the controller. In tests, you can pass a mock event target that is destroyed after each test, ensuring no listeners can ever leak to the real window.
  3. Idempotent Tests: Ensure that the beforeEach setup explicitly resets the state of all singletons used by the component, rather than assuming a clean slate.
  4. Deterministic Timers: Instead of jasmine.Clock.tick(1010), calculate the exact time required based on the business logic to avoid “overshooting” into the next test’s execution window.

Why Juniors Miss It

  • Focus on the Symptom: Juniors often focus on the error message (RangeError) and try to optimize the code causing it (the recursion), rather than looking at the test environment.
  • The “Works on My Machine” Fallacy: Because the error is rare locally, a junior might conclude it is a “flaky runner issue” or a “browser bug” rather than a logic flaw in their cleanup code.
  • Lack of Understanding of the Event Loop: Juniors often treat setTimeout as a synchronous delay. They don’t realize that a setTimeout callback is a scheduled task that can execute at any point in the future, even after the function that created it has finished.

Leave a Comment