Summary
This incident analyzes a multi‑application user‑management architecture where authentication is centralized, but authorization (roles & permissions) is unclear and inconsistently implemented. The lack of a clear ownership model for authorization caused confusion, duplicated logic, and operational delays when onboarding users.
Root Cause
The core issue was mixing authentication concerns with authorization concerns across multiple independently deployed applications.
Key contributing factors included:
- No single source of truth for authorization
- Kafka‑based propagation of user identity, but not of roles/permissions
- Each app having its own domain‑specific permission model
- User‑management UI attempting to manage data it does not own
- Tight coupling between user‑management and app‑specific authorization schemas
Why This Happens in Real Systems
Distributed systems frequently fall into this trap because:
- Authentication is easy to centralize, but authorization is domain‑specific
- Teams want a “single admin UI,” but each app has different permission semantics
- Microservices encourage data ownership boundaries, but UI teams want centralized control
- Event‑driven propagation works for identity, but not for complex authorization rules
- Organizations underestimate the complexity of RBAC/ABAC models across domains
Real-World Impact
Systems with this architecture typically experience:
- Onboarding delays because roles cannot be assigned until user records propagate
- Inconsistent permissions when apps interpret roles differently
- Operational overhead from admins switching between multiple UIs
- Security gaps when stale permissions are not synchronized correctly
- Tight coupling that makes schema changes risky and slow
Example or Code (if necessary and relevant)
Below is a minimal example of how apps can subscribe to identity events while keeping authorization local:
// Identity event handler inside App-1
func HandleUserCreatedEvent(evt UserCreated) {
db.InsertUser(evt.UserID, evt.Email, evt.FullName)
}
// Local RBAC assignment inside App-1
func AssignRole(userID string, role string) {
db.InsertUserRole(userID, role)
}
This illustrates the separation:
identity comes from the central system,
authorization stays inside the app.
How Senior Engineers Fix It
Experienced engineers solve this by enforcing clear ownership boundaries:
1. Centralize Authentication & Identity Only
- User‑management owns:
- user creation
- identity lifecycle
- login/authentication
- app membership (which apps a user can access)
2. Decentralize Authorization
Each app owns:
- its own roles
- its own permissions
- its own authorization logic
- its own admin UI for role assignment
3. Provide a Unified Admin Experience Without Breaking Boundaries
Senior engineers implement a federated admin UI, not a centralized authorization engine:
- The user‑management UI calls each app’s admin API to:
- fetch available roles
- assign roles
- display user permissions
- Each app remains the source of truth for its own authorization data
- No cross‑app schema knowledge is stored in user‑management
4. Use an Authorization Gateway (Optional)
Some teams introduce:
- OPA (Open Policy Agent)
- AuthZ microservice per domain
- Fine‑grained permission APIs
But the key rule remains:
authorization logic stays with the domain that understands it.
Why Juniors Miss It
Junior engineers often struggle with this because:
- They assume “centralizing everything” is simpler
- They underestimate the complexity of domain‑specific permissions
- They don’t yet recognize data ownership boundaries
- They try to make the user‑management app “smart,” which creates coupling
- They overlook the operational cost of schema drift across apps
The senior mindset is different:
centralize identity, decentralize authorization, federate the admin experience.