Write passphrase to a waiting/sleeping cryptsetup process

by

Summary

A senior production engineer analyzes an issue where a cryptsetup process in interruptible sleep mode fails to accept a passphrase sent via /proc/<pid>/fd/0, despite correctly writing to it. The root cause stems from incorrect assumptions about terminal handling and process input paths in an embedded Linux environment. The solution involves understanding how cryptsetup interacts with TTY devices and ensuring input is directed to the correct terminal rather than generic console endpoints.

Root Cause

The cryptsetup process reads input from /dev/tty0 or a dynamically allocated TTY (e.g., via /dev/pts/*) instead of /dev/console. Key factors include:

  • TTY Mismatch: Writing to /proc/200/fd/0 (pointing to /dev/console) does not trigger decryption because the process’s interactive_pass function likely opens /dev/tty or its own TTY.
  • Process Sleep State: The S mode indicates the process is waiting for blocking I/O (e.g., a TTY read), which is not satisfied by generic console writes.
  • Missing Terminal Context: Initramfs environments often lack full TTY initialization, leaving the process without a valid terminal for passphrase input.

Why This Happens in Real Systems

  • Interactive TTY Assumptions: Applications like cryptsetup expect input from a controlling terminal (TTY), not arbitrary file descriptors. The interactive_pass function explicitly opens /dev/tty to ensure secure, unbuffered input.
  • Initramfs Limitations: Minimal environments often skip full terminal setup, leading to mismatches between the process’s expected TTY and actual system state.
  • Console vs. TTY: /dev/console may not correspond to the active terminal used by the process, especially in headless or network-booted systems.

Real-World Impact

  • Boot Failures: Systems with remote unlocking requirements (e.g., cloud VMs) fail to decrypt root volumes, causing unrecoverable boot loops.
  • Security Risks: Incorrect TTY handling might expose passphrases to unintended devices or logs.
  • Operational Overhead: Manual intervention is required to resolve boot hangs, increasing downtime and administrative burden.

Example or Code (if necessary and relevant)

# Identify the correct TTY for the cryptsetup process
cat /proc/200/stat | awk '{print $4}'  # Check the tty number (e.g., "4" = /dev/tty4)
# Write passphrase to the correct TTY
echo -e "password" > /dev/tty$(cat /proc/200/stat | awk '{print $4}')

How Senior Engineers Fix It

  • TTY Resolution: Use /proc/<pid>/stat to determine the controlling terminal of the cryptsetup process (field 4: tty_nr).
  • Direct TTY Write: Send the passphrase to the resolved TTY (e.g., /dev/tty4) instead of /dev/console or /proc/<pid>/fd/0.
  • PTY Allocation: If possible, allocate a pseudo-terminal (PTY) for Dropbear sessions and configure the cryptsetup process to read from it.
  • Signal Handling: Trigger the process to re-read input by sending signals (e.g., SIGUSR1) if it supports dynamic TTY switching.

Why Juniors Miss It

  • Misunderstanding TTY Layers: Confusing /dev/console, /dev/tty0, and /dev/pts/* leads to targeting the wrong input device.
  • Assuming /proc/<pid>/fd/0 is Authoritative: File descriptors can dynamically change; the process may re-open TTYs or switch contexts after initialization.
  • Overlooking Process Internals: Without tools like strace, junior engineers guess at interactive_pass behavior instead of researching cryptsetup source code or documentation.

Leave a Comment