Make LiteLLM imports lazy - #49
Conversation
Co-authored-by: Copilot <copilot@github.com>
Greptile SummaryThis PR makes LiteLLM an import-time-lazy dependency:
Confidence Score: 4/5Safe to merge with one caveat: The deferred configuration in dspy/clients/_litellm.py — the unconditional Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant DSPy
participant _litellm as dspy/clients/_litellm.py
participant require as lazy_import.require()
participant LiteLLM
User->>DSPy: import dspy
Note over DSPy,LiteLLM: litellm NOT imported yet
User->>DSPy: dspy.LM("model")("Hello")
DSPy->>_litellm: "get_litellm(feature="dspy.LM")"
_litellm->>require: require("litellm")
alt litellm installed
require-->>_litellm: LazyLoader-backed module
_litellm->>LiteLLM: _materialize_litellm (access .completion)
LiteLLM-->>_litellm: fully loaded module
_litellm->>LiteLLM: _configure_litellm_defaults (once, cached)
_litellm-->>DSPy: litellm module
DSPy->>LiteLLM: litellm.completion(...)
LiteLLM-->>DSPy: response
DSPy-->>User: result
else litellm not installed
require-->>_litellm: _MissingModule stub
_litellm->>_litellm: _materialize_litellm stub.__getattr__
_litellm-->>User: ImportError("pip install dspy[litellm]")
end
Reviews (3): Last reviewed commit: "Refine lazy LiteLLM loading" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| def _set_random_seeds(self, seed): | ||
| self.rng = random.Random(seed) |
There was a problem hiding this comment.
The removal of np.random.seed(seed) from _set_random_seeds silently breaks reproducibility for any code in the optimization loop that relies on numpy's global RNG (e.g., components using np.random.choice, np.random.shuffle, or any other np.random.* function). Since simba.py still requires numpy through require("numpy"), and optimizers often compose, runs with the same seed may now return different results each time numpy happens to be available. A conditional seed based on is_available("numpy") would restore the old behaviour without making numpy required.
| def _is_litellm_model_response_stream(value: Any) -> bool: | ||
| cls = type(value) | ||
| return cls.__name__ == "ModelResponseStream" and cls.__module__.startswith("litellm") |
There was a problem hiding this comment.
ModelResponseStream
type(value).__name__ == "ModelResponseStream" only matches the concrete class, not subclasses. If litellm ever introduces a more-specific subclass (e.g., per-provider variants), those objects would fall through and be yielded as raw values rather than being dispatched to listeners. The previous isinstance(value, ModelResponseStream) correctly walked the MRO. Using issubclass on the type preserves the lazy-import approach while restoring inheritance-aware matching.
| def _is_litellm_model_response_stream(value: Any) -> bool: | |
| cls = type(value) | |
| return cls.__name__ == "ModelResponseStream" and cls.__module__.startswith("litellm") | |
| def _is_litellm_model_response_stream(value: Any) -> bool: | |
| cls = type(value) | |
| return cls.__module__.startswith("litellm") and any( | |
| c.__name__ == "ModelResponseStream" for c in cls.__mro__ | |
| ) |
| @functools.cache | ||
| def _configure_litellm_defaults(litellm: types.ModuleType) -> None: | ||
| """Apply DSPy's global LiteLLM defaults once when LiteLLM is first imported.""" | ||
| litellm.telemetry = False | ||
| litellm.cache = None # By default we disable LiteLLM cache and use DSPy on-disk cache. | ||
| if not getattr(litellm, "_dspy_logging_configured", False): | ||
| litellm.suppress_debug_info = True | ||
| litellm._dspy_logging_configured = True |
There was a problem hiding this comment.
litellm.cache and litellm.telemetry unconditionally overridden at first use
Previously these were set at import dspy time, meaning any user who configured litellm.cache after import dspy would see their value respected on all subsequent calls. With the deferred approach, _configure_litellm_defaults now runs on the first LM call, which is after user code that configures litellm has already run. A user who sets litellm.cache = RedisCache(...) between import dspy and their first dspy.LM(...) invocation will have their cache silently reset to None. The _dspy_logging_configured guard was added for the logging path but there is no equivalent guard for cache or telemetry.
| @functools.cache | |
| def _configure_litellm_defaults(litellm: types.ModuleType) -> None: | |
| """Apply DSPy's global LiteLLM defaults once when LiteLLM is first imported.""" | |
| litellm.telemetry = False | |
| litellm.cache = None # By default we disable LiteLLM cache and use DSPy on-disk cache. | |
| if not getattr(litellm, "_dspy_logging_configured", False): | |
| litellm.suppress_debug_info = True | |
| litellm._dspy_logging_configured = True | |
| @functools.cache | |
| def _configure_litellm_defaults(litellm: types.ModuleType) -> None: | |
| """Apply DSPy's global LiteLLM defaults once when LiteLLM is first imported.""" | |
| if not getattr(litellm, "_dspy_telemetry_configured", False): | |
| litellm.telemetry = False | |
| litellm._dspy_telemetry_configured = True | |
| if not getattr(litellm, "_dspy_cache_configured", False): | |
| litellm.cache = None # By default we disable LiteLLM cache and use DSPy on-disk cache. | |
| litellm._dspy_cache_configured = True | |
| if not getattr(litellm, "_dspy_logging_configured", False): | |
| litellm.suppress_debug_info = True | |
| litellm._dspy_logging_configured = True |
1 participant
Summary
Tests