Runner Component: Interactive Execution Workbench

[AnselmAdrian]

Description

This enables the users to edit parameters, preview and launch runs, monitor jobs with live logs and history, and provide stakeholders with visibility into experiments—all without dropping to the CLI.

With the Runner, users gain:

• Parameter editing & validation: Modify parameters directly in the UI with YAML editing, safe reset options, and clear diffs before execution.
• Command preview & control: See the exact kedro run command that will execute, copy it for CLI use, or reset back to defaults.
• Job monitoring: Track multiple concurrent or historical runs with status (running, finished, error), start times, and durations.
• Live logs: Stream and expand logs per job in real time, or open them in modal views for deep inspection.
• Run history persistence: Jobs, parameter edits, watch lists, and custom command overrides survive reloads, making iteration faster.
• Safe termination: Cancel jobs mid-run without leaving the UI.
• Watch list tracking: Keep a curated list of parameters or datasets, quickly access their metadata, and re-edit across runs.
• Stakeholder visibility: Non-technical stakeholders can open the Runner to observe progress, parameter choices, and experiment reproducibility without CLI access.

See issue 2483: Runner Component: Interactive Execution Workbench

Development notes

Demo of Current Draft

Runner.Page.Demo.Fast.mp4

API changes

• Updates to python API with the following changes:
  • (POST) /api/run-kedro-command: Run a Kedro command provided as a string in a subprocess and return the output
  • (POST) /api/kedro-command-cancel/{job_id}: Attempt to terminate a running Kedro command.
  • (GET) /api/kedro-command-status/{job_id}: Get the status of a previously run Kedro command.
• Kedro runs use subprocess

UI changes

• Modular runner architecture: runner-manager orchestrates command panel, watch list panel, jobs panel.
• Command panel orchestrates executing the kedro command by handling the following:
  • command assembly: base command + args + validated params
  • Command validation: Displayed commands and parameter edits for the user to validate
• Watch list panel helps the user to view key parameters and datasets while allowing quick parameter edits.
  • Quick access to the metadata panel for selected items
  • Parameter workflows: editable YAML/JSON access via the metadata panel allow for changes that is stored only in the UI and not saved to the kedro project.
• Jobs model: local list of {id, command, status, timing, exitCode} with clear status transition hooks.
• Summary of key components:
  • KedroRunManager (runner-manager.js): Top-level coordinator; wires jobs, watch list, command builder, metadata, dialogs, toasts.
  • ControlPanel: Builds & edits final run command, shows argument summary, opens parameter diff dialog, copy/reset actions.
  • JobListPanel: Displays active/previous jobs with log preview, expansion, termination, removal, and per/all clear confirmations.
  • WatchPanel: Parameter/dataset watch list UI with drag reorder, tab switch (parameters/datasets), changed-value highlighting.
  • WatchListDialog: Modal for selecting parameters/datasets from the graph (includes embedded FlowChart) to stage additions to watch list.
  • ParamMetadataEditor: Inline YAML editor for a selected parameter with save/reset + diff tracking status.
  • ParameterDialog: Modal diff viewer listing changed parameters, showing original vs current YAML and per-param/combined --params args.
  • useCommandBuilder: Assembles full kedro run command string (env/pipeline/tags/params), builds param diff model, tracks changes.
  • useJobs: Manages job list lifecycle (add/update/persist), polling job status, termination, log accumulation.
  • useWatchList: Orchestrates watch list state, hydration from storage, sync with parameter editor, metadata fetch triggers.
  • useParameterEditor: Tracks original vs edited parameter values, persistence, dirty detection, per-key reset/remove ops.

QA notes

No changes are expected in the flowchart and workflow pages
A new runner page will be available for testing with the expected functionality described in below:

• Command lifecycle:
  • edit/reset/copy command work
  • command arguments and parameter edits are visible
  • able to override commands if needed using text input
• Watch list:
  • add via graph/search
  • reorder, tab switch
  • clear/remove, persists
  • highlights changed values.
  • Params editing allows YAML load/save/reset
• Jobs:
  • poll to terminal
  • terminate running job
  • expand logs/log modal
  • clear single/all with confirm

Checklist

• Update Runner API to trigger and manage kedro run executions
• Build interface to construct kedro run commands from pipeline, tag, and parameter filters
• Add parameter editing with YAML validation and safe reset
• Implement parameter diff model for --params
• Integrate job lifecycle management (start, monitor, terminate, remove)
• Add log streaming with expandable/collapsible views and modal support
• Persist jobs, watch list, and parameter edits across reloads via localStorage
• Enforce parameter change validation (list arguments cannot be modified due to --params limitation)

[ravi-kumar-pilla]

Hi @AnselmAdrian ,

Appreciate your efforts in putting up this PR and the time you invested into this. We will be looking into this and prioritize the next steps.

Thank you

[rashidakanchwala]

haven't tried this yet. but the gif looks amazing. I love the UI!<3

[ravi-kumar-pilla]

Hi @AnselmAdrian ,

Thank you for the contribution. The UI looks impressive and we would like you to participate in a Tech Design session to go over this with the TSC. As first steps, could you please join our Kedro slack org .

We have our next Tech design session on Sep 17, 2025 (2pm-3pm BST). Please let us know if you will be able to present this to the group. We can provide more details of the event once you join the org. Looking forward to hearing from you.

Thank you

[AnselmAdrian]

I'll be glad to join the slack. As for the discussion, I will be free then, and I can join. Just let me know if there's anything I need to prepare?

[ravi-kumar-pilla]

I'll be glad to join the slack. As for the discussion, I will be free then, and I can join.

Hi @AnselmAdrian , Please ping me (Ravi Kumar Pilla) on slack once you join the Kedro slack Org and I can share further information there regarding the meeting.

Just let me know if there's anything I need to prepare?

You will be given a 15min slot at the Tech Design and we would love to see you cover:

1. High level architecture of how it works / Demo
2. Why is it important
3. Follow-up and future improvements if any/planned
4. Q&A from the TSC

Thank you

[rashidakanchwala]

Hi @AnselmAdrian , we wanted to share the GIF of your Workbench solution in the #user-research slack channel to get some high-level feedback from users on whether this is something that would be helpful for them. Are you happy for us to do that?

[AnselmAdrian]

Sure, that sounds great