Add publication-pipeline guide: prompt -> code -> paper -> journal - #7
Open
mdenolle wants to merge 1 commit into
Open
Add publication-pipeline guide: prompt -> code -> paper -> journal#7mdenolle wants to merge 1 commit into
mdenolle wants to merge 1 commit into
Conversation
A validated, end-to-end workflow for version-controlling a paper the same way code is version-controlled: a Quarto Markdown source in the same repo as the analysis code, one build script that renders it to a journal's LaTeX class, and a GitHub Action that syncs the built manuscript to a paper-only repo Overleaf's GitHub Sync reads from -- so co-authors keep working in Overleaf with no new tooling, and their feedback comes back to the code repo as GitHub issues instead of evaporating in email/Slack. Extracted from a real submission draft (Denolle-Lab/codameter / Denolle-Lab/codameter-paper), including a gotchas appendix of the specific LaTeX/pandoc/class conflicts that cost real debugging time (a natbib length-register collision, a hardcoded bibliography-heading case Quarto's own template does not expose a metadata knob for, a class that silently drops the LaTeX float-placement argument, and a Pandoc raw-table parsing edge case), plus a copyable template (build.py + sync workflow) genericized from the same two files. Linked from research/README.md alongside the existing Gaia entry.
There was a problem hiding this comment.
Pull request overview
Adds a new research guide describing an end-to-end “prompt → code → paper → journal” workflow, including a copyable template (build.py + GitHub Actions sync) for maintaining a Quarto/LaTeX manuscript in the same repo as analysis code while syncing compiled artifacts to an Overleaf-linked paper-only repo.
Changes:
- Add
research/publication-pipeline/guide documenting the workflow, rationale, and common LaTeX/Pandoc gotchas. - Add
template/with a Python build script and a GitHub Actions workflow for syncing built manuscript artifacts to an Overleaf-linked repo. - Link the new guide from
research/README.md.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| research/README.md | Links the new publication-pipeline guide from the research index. |
| research/publication-pipeline/README.md | New end-to-end workflow write-up, worked example, and gotchas appendix. |
| research/publication-pipeline/template/README.md | Quick-start instructions for copying/adapting the template files. |
| research/publication-pipeline/template/build.py | Template build script to render Quarto → TeX/PDF with optional patch+recompile. |
| research/publication-pipeline/template/.github/workflows/sync-to-overleaf.yml | Template GitHub Actions workflow to sync built paper artifacts to a paper-only repo for Overleaf GitHub Sync. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| This guide describes a pipeline where **the code, the figures, and the paper text are versioned together in one git history**, the LaTeX a journal wants is *generated*, not hand-typeset, and the one genuinely manual step — turning a colleague's Overleaf comment into a code change — is captured as a GitHub issue instead of a Slack message that evaporates. | ||
|
|
||
| It is not a framework to install. It is five small, standard tools wired together in a specific order. Every piece is something you already have a account/license for as an academic: Python, git, GitHub, Quarto, Overleaf. |
| This pipeline is not hypothetical — it is running today on: | ||
| - **Code repo**: [`Denolle-Lab/codameter`](https://github.com/Denolle-Lab/codameter) — `paper/manuscript_marine.qmd` is the source, `paper/build.py` is the generator, `src/codameter/` is the analysis code the figures come from. | ||
| - **Paper repo**: [`Denolle-Lab/codameter-paper`](https://github.com/Denolle-Lab/codameter-paper) — synced automatically from the code repo, linked to an Overleaf project via Overleaf's GitHub Sync. | ||
| - **Sync automation**: `.github/workflows/sync-paper-to-overleaf.yml` in the code repo. |
Comment on lines
+94
to
+98
| # .github/workflows/sync-paper-to-overleaf.yml — the shape of it (full version in template/) | ||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: ["paper/manuscript.tex", "paper/*.bib", "paper/*.cls", "literature/figs/**"] |
Comment on lines
+113
to
+114
| if not changed: | ||
| return |
| ## Files | ||
|
|
||
| - **`build.py`** → copy to `paper/build.py`. Edit the three `CONFIG` blocks at the top: your manuscript's filename(s), any figure-regeneration commands, and (only if your journal's class needs it) the literal `.tex` string patches described in the parent README's gotchas appendix. | ||
| - **`.github/workflows/sync-to-overleaf.yml`** → copy to `.github/workflows/sync-to-overleaf.yml` in your code repo. Edit the four `CONFIG` blocks: branch name, trigger paths, paper repo name, and the exact filenames staged in the commit step. |
Comment on lines
+55
to
+57
| - "paper/*.cls" # your journal's class file, if any | ||
| - "paper/*.bst" # your journal's bibliography style, if any | ||
| - "figures/**" # wherever your figures live |
Comment on lines
+92
to
+94
| cp paper/*.cls paper-repo/ 2>/dev/null || true | ||
| cp paper/*.bst paper-repo/ 2>/dev/null || true | ||
| rsync -a --delete figures/ paper-repo/figures/ |
| # co-authors' Overleaf project reads, so only stage what this | ||
| # workflow is actually meant to manage. | ||
| git add manuscript.tex references.bib figures/ | ||
| git add *.cls *.bst 2>/dev/null || true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
research/publication-pipeline/guide: a validated, end-to-end workflow for version-controlling a paper the same way code is version-controlled (Quarto Markdown source in the same repo as the analysis code, an automated Markdown-to-journal-LaTeX build, and a GitHub Action that syncs the built manuscript to a paper-only repo Overleaf's GitHub Sync reads from).template/(genericizedbuild.py+ sync workflow YAML) so a new project can adapt rather than write from scratch.research/README.mdalongside the existing Gaia entry.Worked example this is extracted from
Denolle-Lab/codameter(code + paper source) synced toDenolle-Lab/codameter-paper(Overleaf-linked), both currently running this exact pipeline.Test plan
build.pytemplate: syntax-checked,--helpruns cleanlycodametertemplate/files verified to resolve to real paths🤖 Generated with Claude Code