feat: make numpy an optional dependency

[isaacbmiller]

Summary

Move numpy from a required runtime dependency to an optional extra.

numpy-dependent features (KNN, Embeddings, SIMBA, MIPROv2, DummyVectorizer, Embedder post-processing, etc.) raise a clear ImportError pointing at pip install dspy[numpy] only when actually invoked.

Changes

New: dspy/utils/lazy_import.py

Centralized lazy-import infrastructure (ported from dspy-core e8e3e196):

• require(module, *, extra, feature) — import or raise a friendly ImportError with install hint
• optional(module, attr, default) — import-or-default for module-level sentinels
• is_available(module) — cheap find_spec-based check, lru_cached
• _INSTALL_HINTS registry mapping import names to dspy extras (numpy, optuna, mcp, langchain, weaviate, anthropic)

Replaced numpy with stdlib where possible

• math.log2 in mipro_optimizer_v2, gepa
• math.inf in infer_rules
• statistics.pstdev in copro_optimizer
• sum()/len() in teleprompt/utils
• Removed dead np.random.seed() in mipro (optuna uses its own seed param)

Lazy numpy via require("numpy")

Files that genuinely need numpy arrays (knn.py, embeddings.py, embedding.py, dummies.py, simba.py) now call require("numpy") inside method bodies instead of top-level import numpy.

pyproject.toml

• Removed numpy from hard dependencies
• Added numpy = ["numpy>=1.26.0"] optional extra
• Added numpy to dev and test_extras so CI still gets it

Tests

• Added tests/utils/test_lazy_import.py — 10 tests covering require, optional, is_available, registry/fallback behavior, and pyproject extras consistency check (Python 3.10 compatible)

Notes

• import dspy works without numpy installed
• Users who want numpy-backed features install via pip install dspy[numpy]
• Upstream PR: feat: make numpy an optional dependency stanfordnlp/dspy#9659

[greptile-apps]

Greptile Summary

This PR makes numpy an optional dependency by introducing a lazy_import.py helper and replacing direct numpy usage with either stdlib equivalents (math.log2, math.inf, statistics.pstdev, sum/len) or a require("numpy") stub that defers the ImportError with a pip install dspy[numpy] hint until the feature is actually used.

• dspy/utils/lazy_import.py (new): require() returns a LazyLoader-wrapped module when the package is installed, or a _MissingModule stub that raises ImportError with an install hint on first attribute access; is_available() provides a cached find_spec check without triggering an import.
• Stdlib substitutions: np.log2 → math.log2, -np.inf → -math.inf, np.std → statistics.pstdev, np.average → sum/len across mipro_optimizer_v2, infer_rules, copro_optimizer, gepa, and utils.
• from __future__ import annotations correctly added to embedding.py, dummies.py, embeddings.py, and simba.py — preventing evaluation of -> np.ndarray return annotations at class-definition time when numpy is absent.

Confidence Score: 5/5

Safe to merge. All numpy usages are either replaced with stdlib equivalents or guarded by the new lazy-import infrastructure, and CI still receives numpy via the updated dev/test_extras groups.

The stdlib substitutions are semantically equivalent, the lazy-import machinery follows a well-established pattern (LazyLoader + stub), and from __future__ import annotations is applied exactly where needed to prevent numpy annotation evaluation at class-definition time. All previously raised issues have been addressed in this diff.

No files require special attention.

Important Files Changed

Filename	Overview
dspy/utils/lazy_import.py	New centralized lazy-import infrastructure: require() returns a LazyLoader-wrapped module or a _MissingModule stub that defers ImportError with install hints; is_available() uses a cached find_spec check.
dspy/clients/embedding.py	Replaced hard import numpy as np with module-level np = require("numpy"); added from __future__ import annotations to prevent evaluation of -> np.ndarray return annotation at class-definition time when numpy is absent.
dspy/utils/dummies.py	Same lazy-require pattern; from __future__ import annotations is necessary because DummyVectorizer.__call__ carries a -> np.ndarray return annotation.
dspy/retrievers/embeddings.py	Lazy-require pattern applied; from __future__ import annotations defers evaluation of np.ndarray parameter annotations in _faiss_search, _rerank_and_predict, and _normalize.
dspy/predict/knn.py	Replaced hard numpy import with module-level require("numpy"); no numpy type annotations so from __future__ import annotations is correctly omitted.
dspy/teleprompt/copro_optimizer.py	All three np.std calls replaced with statistics.pstdev; semantically equivalent (both compute population std dev). Guarded numpy import removed cleanly.
dspy/teleprompt/mipro_optimizer_v2.py	np.log2 → math.log2 and np.random.seed removed; numpy dependency fully eliminated from this file.
dspy/teleprompt/utils.py	np.array/np.average replaced with list + sum/len; np.sum replaced with sum(); int() cast removed (no longer needed since builtin sum returns a Python int).
dspy/teleprompt/infer_rules.py	-np.inf replaced with -math.inf; top-level numpy import removed entirely.
dspy/teleprompt/gepa/gepa.py	Local import numpy as np inside auto_budget replaced with math.log2; numpy dependency removed from this method.
dspy/teleprompt/simba.py	Module-level require("numpy") pattern applied; from __future__ import annotations added for consistency with other files using union-type hints.
tests/utils/test_lazy_import.py	10 focused unit tests for require, is_available, and _MissingModule; pyproject extras consistency check implicitly relies on tomli being installed as a pytest transitive dependency on Python 3.10.
pyproject.toml	numpy removed from hard dependencies, added as numpy optional extra; numpy added to both dev and test_extras so CI retains it.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["import dspy"] --> B["module-level: np = require('numpy')"]
    B --> C{numpy installed?}
    C -- Yes --> D["LazyLoader wraps real numpy\nadded to sys.modules"]
    C -- No --> E["_MissingModule stub\n(stores call-site frame)"]
    D --> F["First np.attr access\ntriggers real numpy import"]
    E --> G["First np.attr access\nraises ImportError with\npip install dspy[numpy] hint"]
    F --> H["Normal numpy operation"]

    subgraph "stdlib replacements (no lazy import)"
        I["math.log2 (was np.log2)"]
        J["math.inf (was np.inf)"]
        K["statistics.pstdev (was np.std)"]
        L["sum/len (was np.average)"]
    end

Loading

_{Reviews (14): Last reviewed commit: "docs(lazy_import): switch RST double bac..." | Re-trigger Greptile}

[MaximeRivest]

In embedding.py and knn.py do you consider putting that at the bottom of the file instead:

# mymodule.py
def __getattr__(name):
    if name == 'np':
        globals()['np'] = require("numpy")
        return np
    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

or that in the init of the class:

# knn.py
import ...

np = None

class MyClass:
    def __init__(self):
          global np
          if np is None:
              np = require('numpy')

[MaximeRivest]

single back-tick is preferred in our settings of mkdocs.

[MaximeRivest]

optional is not used, can we wait that it's needed before making this util?