OMR Instrumentation Pipeline

Fully automated pipeline that takes a single orchestral score page image and produces a structured CSV of every staff line annotated with its instrument, part, and transposition.

input image
    │
    ▼
[Step 1]  DocTR OCR          → raw text tokens + bounding boxes
[Step 2]  Staff detection    → YOLO boxes + OMR staff segmentation
[Step 3]  JSON conversion    → staff-count JSON (YOLO & OMR paths)
[Step 4]  OCR filter         → tokens near staff regions only
[Step 5]  OCR filter plot    → visual QC image
[Step 6]  GPT classify       → normalised instrument labels (JSON)
[Step 8]  GPT Vision         → staff-group / ensemble structure
[Step b]  JSON → CSV         → raw GPT CSV
[Step 7]  Matching & reformat→ staff-to-instrument mapping
[Step f]  Final post-process → cleaned CSV

---

Prerequisites

Requirement	Notes
Miniconda / Anaconda	Environment management
CUDA-capable GPU	Optional but recommended for steps 2c and 6/8
OpenAI API key	Required for steps 6 and 8

---

1. Conda Environment Setup

Option A — Single merged environment (recommended)

conda create -n omr_all python=3.11 -y
conda activate omr_all
pip install -r requirements_merged.txt

Note: omr10 was developed against numpy 1.26.4; the merged environment upgrades to numpy 2.x. Step 2c (OMR staff segmentation) works in testing but verify it on your machine. If it breaks, fall back to Option B.

Option B — Four separate environments

# Step 1 — DocTR OCR
conda create -n ocrt python=3.11 -y
conda activate ocrt
pip install -r requirements_ocrt.txt

# Steps 2a/2b/3a/3b/4/5/7/b/f — YOLO + post-processing
conda create -n stave python=3.11 -y
conda activate stave
pip install -r requirements_stave.txt

# Steps 6, 8 — GPT classification + Vision
conda create -n intern python=3.11 -y
conda activate intern
pip install -r requirements_intern.txt

# Step 2c — OMR staff segmentation
conda create -n omr10 python=3.11 -y
conda activate omr10
pip install -r requirements_omr10.txt

If using Option B, edit run_pipeline.sh and set the four PYTHON_* variables to point to each separate environment instead of omr_all.

Environment–to–script mapping (Option B reference)

Conda env	Scripts	Key packages
ocrt	1_doctr_full_page_process.py	python-doctr, easyocr
stave	2_staff_box.py, 2_1_staff_box_plot.py, 3_*.py, 4_*.py, 5_*.py, 7_*.py, b_*.py, f.py	ultralytics, openai, omrdatasettools
intern	6_1_gpt_classify_reformat.py, 8_gpt_staffinfo.py	tensorflow, openai, transformers
omr10	2_cnt_staff_omr.py	numpy 1.26.4, onnxruntime-gpu, local omr/ package

---

2. OpenAI API Key

Steps 6 and 8 call the OpenAI API. Create a .env file next to the scripts:

echo "OPENAI_API_KEY=sk-..." > /workspace/run_full/.env

The scripts load it automatically via python-dotenv.

---

3. Model Weights

The YOLO staff-detection model is already included:

run_full/ola-layout-analysis-2.0-2025-03-09.pt

The OMR segmentation model checkpoints are in:

run_full/omr/checkpoints/
    seg_net/
    unet_big/

No additional downloads are required.

---

4. Running the Pipeline

bash run_pipeline.sh <input_folder>

<input_folder> must contain at least one image (.png, .jpg, or .jpeg).
All outputs are written into the same folder.

Example using the included test image:

bash run_pipeline.sh example_img

---

5. Pipeline Steps

Step	Script	Env	Input	Output
1	1_doctr_full_page_process.py	omr_all	*.png/jpg	*_ocr.json
2a	2_staff_box.py	omr_all	*.png/jpg	*.txt (YOLO boxes)
2b	2_1_staff_box_plot.py	omr_all	image + *.txt	*_box_plot.png
2c	2_cnt_staff_omr.py	omr_all	*.png/jpg	*_stafflist.pkl
3a	3_count_staff2json.py	omr_all	*.txt	*_yolostaff.json
3b	3_check_staff_cnt_pk.py	omr_all	*_stafflist.pkl	*_yolostaff_gt.json
4	4_ocr_filter.py	omr_all	*_ocr.json + staff JSONs	*_ocr_filtered.json
5	5_ocr_filtered_plot.py	omr_all	image + *_ocr_filtered.json	*_ocr_filtered.png
6	6_1_gpt_classify_reformat.py	omr_all	*_ocr_filtered.json	*_ocr_filtered_classified_normalized.json
8	8_gpt_staffinfo.py	omr_all	image + classified JSON	*_staffgroup.json
b	b_trans_gpt.py	omr_all	*_staffgroup.json	*_trans_gpt.csv
7	7_matching_reformat_standard.py	omr_all	*_trans_gpt.csv + staff JSONs	*_staff_instruments.json, *_trans_gpt_modify.csv
f	f.py	omr_all	*_trans_gpt_modify.csv	*_trans_gpt_final.csv

---

6. Output Files

For an input image named page001.png the pipeline produces:

File	Description
page001_ocr.json	Raw DocTR OCR tokens with bounding boxes
page001.txt	YOLO staff bounding boxes (raw format)
page001_box_plot.png	YOLO detection visualisation
page001_stafflist.pkl	OMR staff segmentation objects
page001_yolostaff.json	Staff count from YOLO
page001_yolostaff_gt.json	Staff count from OMR segmentation
page001_ocr_filtered.json	OCR tokens filtered to staff regions
page001_ocr_filtered.png	Filtered OCR visualisation
page001_ocr_filtered_classified_normalized.json	GPT instrument labels (normalised)
page001_staffgroup.json	GPT staff-group / ensemble structure
page001_trans_gpt.csv	Raw GPT transposition CSV
page001_staff_instruments.json	Staff-to-instrument mapping
page001_trans_gpt_modify.csv	Reformatted transposition CSV
page001_trans_gpt_final.csv	Final output — one row per staff

The final CSV columns: page, staff, system, staffgroup, num_ins, ens, ins1, part1, tone1, ins2, part2, tone2, ins3, part3, tone3, ocr, page_norm, family

---