Can Python install packages on a shorter path on Windows?

by

Summary

A developer on a locked-down Windows machine encountered pip install grpcio failing with WinError 206 (The filename or extension is too long). This happened because the grpcio source distribution requires deep directory nesting and long filenames during the build process, exceeding the default Windows MAX_PATH limit of 260 characters, even after redirecting TEMP environment variables. The root cause is the legacy Windows path limitation clashing with complex source structures.

Root Cause

The failure stems from a collision between Windows’ default path constraints and Python’s build isolation behavior.

  • MAX_PATH Limitation: By default, Windows restricts file paths to 260 characters. The grpcio build artifacts involve deeply nested directories (e.g., third_party/boringssl-with-bazel/src/pki/testdata/...) that easily exceed this.
  • Build Isolation: pip downloads and extracts source distributions to a temporary directory (%TEMP%), which defaults to AppData\Local\Temp. Even if TEMP is changed, pip constructs complex relative paths for the build environment.
  • Environment Variable Scope: Changing TEMP, TMP, or TMPDIR is often insufficient because Python’s tempfile module or the underlying Windows API calls may still reference system defaults or append long suffixes, and the limitation applies to the total path from the root drive, not just the temp folder name.

Why This Happens in Real Systems

Windows imposes the 260-character limit to maintain compatibility with legacy filesystem APIs, though modern Windows (Windows 10+ via registry or Group Policy) supports longer paths. However, in corporate environments:

  • Administrative Lockdowns: IT policies often disable the “Enable Win32 long paths” registry setting or Group Policy, leaving the system vulnerable to path overflows.
  • Complex Dependencies: Packages like grpcio (which wraps C++ libraries such as BoringSSL) include vast trees of source files and test data. PIP’s build process appends unique identifiers and nested structures, rapidly hitting the limit.
  • Virtual Environments: Even inside a venv, pip operations rely on global temporary paths, and relative path resolution doesn’t mitigate absolute path length.

Real-World Impact

  • Build Failures: Installation halts, preventing development or deployment workflows.
  • Productivity Loss: Developers must manually clone repositories and build locally or switch to wheel-only installations, delaying feature work.
  • Incompatibility Risks: If wheels aren’t available for the specific Python/Windows version, the project becomes blocked, potentially forcing environment switches (e.g., WSL) or admin intervention.

Example or Code

To reproduce or verify the issue, check the current environment variables and attempt to calculate the path length during installation:

import os
import tempfile

# Check current temp settings
print("TEMP:", os.environ.get('TEMP'))
print("TMP:", os.environ.get('TMP'))

# Simulate a long pip install path
base_temp = os.environ.get('TEMP', tempfile.gettempdir())
long_suffix = 'grpcio_7653fcf0041e457db8a6b1d95d413a7b/third_party/boringssl-with-bazel/src/pki/testdata/verify_certificate_chain_intermediate-wrong-signature-no-authority-key-identifier/keys'

full_path = os.path.join(base_temp, 'pip-install', '6hu4fcd9', long_suffix)
print(f"Simulated Path Length: {len(full_path)}")

Output will show the length exceeding 260, confirming the issue.

How Senior Engineers Fix It

  • Enable Long Paths via Registry (If Possible): Even without admin, if the machine allows user-level registry edits, add Computer\HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Group Policy\Objects\{GUID}\Software\Microsoft\Windows\CurrentVersion\Policies\System and set LongPathsEnabled to 1. (Note: Group Policy may override this.)
  • Use a Shallow Temporary Directory: Explicitly set TEMP and TMP to a root-level, short path like C:\t or D:\temp via command prompt before running pip: set TEMP=C:\t && set TMP=C:\t && pip install grpcio.
  • Install Pre-built Wheels: Force pip to use wheels without building from source: pip install --only-binary=:all: grpcio==1.74.0. If no wheel exists, download the wheel manually from PyPI or a trusted mirror.
  • Use Windows Subsystem for Linux (WSL): Run the installation in a Linux environment, which ignores Windows path limits.
  • Git Clone and Editable Install: Clone the repo shallowly (git clone --depth 1), then pip install -e . from a short path like C:\repo.

Why Juniors Miss It

  • Surface-Level Fixes: Juniors often assume setting TEMP environment variables alone resolves temp file issues, not realizing path length accumulates from the drive root.
  • Lack of Windows API Awareness: Many are unaware of the historical MAX_PATH constraint or that it’s enforced by the filesystem API, not Python itself.
  • Ignorance of Binary Options: They default to pip install without checking for --only-binary or verifying wheel availability on PyPI.
  • Overlooking Admin Constraints: They may not consider that corporate policies prevent registry changes, leading to frustration without exploring alternatives like WSL.