Fixing TLS Unknown Certificate Handshake Errors

by

Summary

During a security testing engagement using EvilGinx 3.3.0, a critical failure occurred during the TLS handshake process. The user reported the error: Cannot handshake client www.mywebsite.com remote error: tls: unknown certificate. This occurred while attempting to proxy traffic through a custom phishlet on a local environment. The core of the issue is a TLS/SSL handshake mismatch where the client (the victim’s browser) or the proxy server rejects the certificate chain presented during the connection attempt.

Root Cause

The failure stems from a broken Chain of Trust. Specifically:

  • Self-Signed Certificates: The proxy is presenting a certificate that is not signed by a Trusted Root Certificate Authority (CA).
  • Hostname Mismatch: The certificate presented by the EvilGinx proxy does not match the Subject Alternative Name (SAN) or the common name of the requested domain (www.mywebsite.com).
  • Improper Phishlet Configuration: The phishlet is configured to proxy traffic to a legitimate site, but the local EvilGinx instance is failing to impersonate the target’s certificate requirements correctly.
  • Missing Intermediate Certificates: The proxy is serving only the leaf certificate without the necessary intermediate CA certificates, causing modern browsers to terminate the handshake.

Why This Happens in Real Systems

In production-grade distributed systems, this error is rarely about “phishing” and more about infrastructure misconfiguration:

  • Load Balancer Misconfiguration: A Load Balancer (ALB/NLB) is configured with a certificate that doesn’t cover all subdomains used by the application.
  • Expired Certificate Chains: While the leaf certificate might be valid, the Intermediate Certificate in the bundle has expired, breaking the validation path.
  • SNI (Server Name Indication) Mismatches: In multi-tenant environments, the server receives an SNI header but does not have a certificate mapping for that specific hostname, leading to a default (and invalid) certificate being served.

Real-World Impact

  • Service Downtime: Automated API calls between microservices will fail immediately due to strict TLS verification.
  • User Trust Erosion: End-users see “Your connection is not private” warnings, leading to high bounce rates.
  • Security Tooling Failures: Automated scanners and WAFs may fail to inspect encrypted traffic if the handshake cannot be established.
  • Operational Overhead: On-call engineers are alerted to “connection reset” or “unknown certificate” errors, requiring deep packet inspection to resolve.

Example or Code

// Example of a failed TLS handshake check in Go
package main

import (
    "crypto/tls"
    "fmt"
    "net/http"
)

func main() {
    // This will fail if the server uses a self-signed cert 
    // or an untrusted chain (simulating the EvilGinx error)
    client := &http.Client{
        Transport: &http.Transport{
            TLSClientConfig: &tls.Config{InsecureSkipVerify: false},
        },
    }

    resp, err := client.Get("https://www.mywebsite.com")
    if err != nil {
        fmt.Printf("Handshake Failed: %v\n", err)
        return
    }
    defer resp.Body.Close()
    fmt.Println("Status:", resp.Status)
}

How Senior Engineers Fix It

Senior engineers approach this via observability and chain validation:

  • Certificate Chain Auditing: Use tools like openssl s_client -connect host:port -showcerts to inspect the entire certificate chain and identify missing intermediates.
  • Automated Certificate Management: Implement ACME protocols (Let’s Encrypt) to ensure certificates are always valid, auto-renewed, and correctly scoped.
  • Infrastructure as Code (IaC) Validation: Integrate TLS configuration checks into the CI/CD pipeline using tools like Terratest to ensure Load Balancer certificates match the intended hostnames.
  • Strict SNI Enforcement: Ensure that the proxy/web server is configured to strictly match the server_name to the provided certificate.

Why Juniors Miss It

  • Surface-Level Troubleshooting: Juniors often try to fix the error by simply setting InsecureSkipVerify: true (in code) or bypassing browser warnings, which masks the symptom instead of fixing the cause.
  • Ignoring the Intermediate: They focus only on the leaf certificate validity and forget that a certificate is only as good as the chain that connects it to a Root CA.
  • Misunderstanding SNI: They may assume that if a certificate is “valid,” it will work for any domain, failing to realize that the Hostname-to-Certificate mapping is a fundamental requirement of the TLS handshake.

Leave a Comment