Skip to content

Let users edit run.sbatch and copy Slurm .out/.err back - #96

Open
kirillmitrofanov wants to merge 2 commits into
mainfrom
feat/cluster-sbatch-logs
Open

Let users edit run.sbatch and copy Slurm .out/.err back#96
kirillmitrofanov wants to merge 2 commits into
mainfrom
feat/cluster-sbatch-logs

Conversation

@kirillmitrofanov

Copy link
Copy Markdown
Collaborator

Summary

  • Cluster Run now prepares a draft run.sbatch in codes/projects/<id>/batch/<run_id>/ so it can be edited in the modal or in Jupyter before sbatch.
  • After COMPLETED or FAILED, slurm-*.out / .err plus Python stdout.log / stderr.log are copied into batch/<run_id>/logs/ and listed in the Runs panel (failed jobs are why .err exists).
  • #SBATCH --chdir / --output / --error are pinned to the remote run dir after any edit so copy-back can find the files. Independent of Add internal vs hackathon Jupyter spaces #88Feat/schema improvements and optimization #95.

Test plan

  • Open Cluster Run, confirm a draft appears and the script is visible; edit it; Reload from project / Edit in Jupyter still round-trip.
  • Submit; on success, Runs panel shows Logs + Results; Jupyter can open codes/projects/<id>/batch/<run_id>/.
  • Failed/OOM job still copies .err (not only results/ on COMPLETED).
  • Do not merge until an ops deploy is planned; live dbrain.jp is still the old bundle (no editor in /assets/index-CbQzE8nY.js).
  • ssh-agent still required for a real sbatch.

nw-kirill added 2 commits September 10, 2026 00:04
Cluster jobs were submitting an unseen generated script and leaving
slurm-*.out/.err on the compute server (including failed jobs). Prepare a
draft in the project tree, pin log paths after edits, and rsync logs on
COMPLETED and FAILED so the Runs panel and Jupyter can open them.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant