Skip to content

experiment: run DSPy without Deno - #77

Draft
isaacbmiller wants to merge 3 commits into
monty-ci-basefrom
experiment/monty-only-ci
Draft

experiment: run DSPy without Deno#77
isaacbmiller wants to merge 3 commits into
monty-ci-basefrom
experiment/monty-only-ci

Conversation

@isaacbmiller

@isaacbmiller isaacbmiller commented Aug 3, 2026

Copy link
Copy Markdown

Purpose

Measure the CI and compatibility impact of removing Deno/Pyodide entirely and using Monty as DSPy's only first-party code interpreter.

This is an experiment, not a merge proposal. The base is pinned to Stanford DSPy main at c69136b2 for comparison with both upstream CI and the Monty-default/Deno-compatible experiment in #76.

Changes

  • use MontyInterpreter by default in RLM, ProgramOfThought, and CodeAct
  • remove PythonInterpreter, its Deno runner, public export, backend-specific tests, and docs
  • port module-level interpreter contract tests to Monty
  • remove Deno setup and pytest flags from test, dependency-range, and release workflows
  • continue running optional-dependency tests separately

Net diff: 111 insertions, 2,516 deletions.

Validation

  • focused Monty/RLM/ProgramOfThought/CodeAct suite: 171 passed, 2 skipped
  • local full normal suite: 1164 passed, 247 skipped, 2 xfailed in 40.23s
  • local extra suite: 35 passed in 7.52s
  • CI test/build matrix passed on Python 3.10–3.14 twice
  • docs build passed
  • dependency-range minimum-direct passed; latest has the same current LiteLLM/FastAPI incompatibility as upstream main
  • Ruff on touched Python files and git diff --check: pass

CI measurement

Comparisons:

Matrix test job Deno baseline Monty + retained Deno Monty-only run 1 Monty-only run 2
Python 3.10 198s 200s 123s 121s
Python 3.11 205s 185s 248s* 106s
Python 3.12 201s 194s 116s 116s
Python 3.13 163s 195s 107s 125s
Python 3.14 208s 210s 135s 242s*
Median 201s 195s 123s 121s

Across both Monty-only runs, the median test job is 122s: 79s (39%) below the Deno baseline and 73s (37%) below the Monty-default/retained-Deno experiment.

Step timing explains the gain. The normal suite remains approximately flat (median 92s baseline vs 96s Monty-only on the second run), while the optional/Deno phase falls from a baseline median of 91s to 13s after the 105 Deno tests are removed. The speedup is therefore specifically the eliminated Deno compatibility suite, not faster unrelated DSPy tests.

* Each run had one unrelated ~120s normal-suite outlier on a different Python version. Those outliers make total workflow wall time misleading (258s and 246s respectively), but do not affect the repeatable per-job median or the 78s reduction in the former extra+Deno phase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant