Python type hints with optional dependencies

by

Summary

The issue at hand is how to include np.ndarray in the type hint for a function parameter without strictly requiring numpy to be present when type checking is enabled. This is a problem because the function can take either a list or a np.ndarray if numpy is available.

Root Cause

The root cause of this problem is that mypy cannot find the numpy library when it is not installed, resulting in a Cannot find implementation or library stub for module named “numpy” error. This error occurs because mypy requires the numpy library to be installed in order to check the types of np.ndarray.

Why This Happens in Real Systems

This happens in real systems because mypy is designed to check the types of all imports, including optional dependencies like numpy. When numpy is not installed, mypy cannot check the types of np.ndarray, resulting in an error. This can be a problem when there are many optional dependencies, as it can be inconvenient to install all of them just for type checking purposes.

Real-World Impact

The real-world impact of this issue is:

  • Inconvenience of installing optional dependencies just for type checking purposes
  • Potential for legitimate type checking errors to be hidden among extraneous mypy import errors
  • Difficulty in maintaining a clean and efficient development environment

Example or Code

import typing as ty
if ty.TYPE_CHECKING:
    import numpy as np

def foo(bar: list | np.ndarray[tuple[int]]) -> list:
    try:
        import numpy as np
    except ImportError:
        np = None
    if (np is not None) and isinstance(bar, np.ndarray):
        bar = bar.tolist()
    bar.append(0)
    return bar

How Senior Engineers Fix It

Senior engineers can fix this issue by:

  • Using a try-except block to catch the ImportError exception when numpy is not installed
  • Using the typing.TYPE_CHECKING constant to conditionally import numpy only when type checking is enabled
  • Using a type hint that includes np.ndarray only when numpy is installed

Why Juniors Miss It

Junior engineers may miss this issue because:

  • They may not be familiar with the typing.TYPE_CHECKING constant and its use in conditional imports
  • They may not understand the implications of using optional dependencies in type hints
  • They may not know how to use try-except blocks to handle ImportError exceptions in a way that works with mypy

Leave a Comment