How to emit Socket.IO events from a BullMQ worker running in a separate process?

by

Summary

A Node.js backend using BullMQ for background jobs and Socket.IO for real-time updates faced a common architectural challenge: emitting Socket.IO events from a separate BullMQ worker process after job completion. The root issue was the forced separation of concerns between the Socket.IO server (bound to HTTP connections) and the BullMQ worker (isolated process lacking client connections).

Root Cause

  • Process isolation: BullMQ workers run in standalone processes without access to the Socket.IO io server instance bound to the API server’s HTTP life cycle.
  • Socket.IO server/client asymmetry: Workers cannot re-use the main Socket.IO server’s resources because they:
    • Have no HTTP server binding
    • Lack connection state/context for clients
  • Misconfigured inter-process communication (IPC) between the API server and workers.

Why This Happens in Real Systems

  • Background job processing demands isolation for resilience and scalability (e.g., workers crashing shouldn’t crash API servers).
  • Real-time notifications often tie into long-running background tasks (order processing, report generation) but emit from a central web server.
  • Engineers decouple systems prematurely without defining IPC protocols