PedNYC deterministic pipeline

This repository currently implements Checkpoint 1 only: deterministic raw-file discovery and Unity base64 float-array decoding. It intentionally does not reconstruct time, calculate features, analyze conflicts or gestures, or generate graphs.

Setup

python -m pip install -r requirements.txt

Run Checkpoint 1

The full decoder discovers all 374 raw trials, writes semicolon-delimited decoded files under data/processed/decoded/<study>/, and resumes only after validating existing outputs.

python run_checkpoint_01b.py --dry-run --workers 2
python run_checkpoint_01b.py --limit 5 --resume --workers 2
python run_checkpoint_01b.py --resume --workers 2

Manifests are written under outputs/summary/, and the full report is outputs/checkpoints/checkpoint_01_full_decoding.md. Decoded filenames retain scenario and session identity. Raw files are opened read-only and never modified.

Test

python -m pytest -q

Checkpoint 2 recommended command (after its implementation):

python run_pipeline.py checkpoint-02 --manifest outputs/checkpoints/checkpoint_01_decoding_manifest.csv

Driver signal analysis

Parse the raw left/right Boolean indicator states, update trial summaries, and regenerate the PedNYC1 pilot driver-signal timelines with:

python run_driver_signals.py --pilot-study PedNYC1

Generate the same graph for every successfully analyzed trial with:

python run_driver_signals.py --pilot-study all --graphs-only

Short one-frame pulses are rejected. Valid turn-signal pulses separated by no more than 0.75 seconds are merged as one blinking event; the raw Boolean trace remains in the frame-level feature CSV.