SewingStudy

Wizard-of-Oz console for studying proactive vs. reactive machine assistance during novice sewing. Used in a within-subjects study (N=20) comparing two levels of machine initiative while participants sewed pillows on a Brother XM3700 sewing machine.

To be published, likely as: Proactive vs. Reactive Machine Assistance during Fabrication: A Wizard-of-Oz Study of Machine Initiative during Novice Sewing.

What this does

A hidden wizard observes participants through four synchronized camera streams and delivers real-time interventions that appear to come from the sewing machine itself. The system provides two output modalities:

• Speech: keyboard hotkeys trigger pre-scripted TTS utterances (task guidance, encouragement, error correction). Free-text input is also supported for unanticipated situations.
• Spatial pointing: a pygame-controlled white dot projected onto the workspace, allowing the wizard to point at physical locations (e.g., the presser foot lever, a fabric edge).

Experimental conditions

Two scripts implement the two experimental conditions:

• code_active.py (proactive): the wizard intervenes based on anticipated needs — hesitation, incorrect positioning, gaze suggesting confusion, or approach to common error points.
• code_passive.py (reactive): the wizard intervenes only in response to explicit requests for help or clearly observed errors that would prevent task completion.

Both scripts share the same interaction modalities and utterance set, but the utterances differ in framing (e.g., proactive: "Make sure to put the right sides together" vs. reactive: "You did not put the right sides together").

Controls

Key	Function
Mapped keys (arrows, letters, etc.)	Trigger pre-scripted TTS utterances
1	Enter free-text typing mode
2 / Esc	Exit typing mode
Enter (in typing mode)	Speak typed text
Up arrow	Repeat last spoken phrase
q	Quit
Mouse	Control projected pointing dot

Requirements

• Python 3
• pygame
• macOS (uses the say command for TTS)

pip install pygame

Study setup

The wizard sat behind a screen in the same room as the participant, with access to:

• A computer running OBS (streaming and recording two webcams: a handwheel-focused camera and a fisheye needle-area camera)
• An iPad streaming 360-degree footage from a machine-mounted Insta360 camera through the Insta360 app
• A phone streaming an egocentric view from participant-worn pinhole camera glasses through the Ti2Cam app
• A projector for the spatial pointing dot

Key findings

• Proactive assistance was perceived as significantly more timely (W=0.0, p=.001, r=.89) without increasing perceived annoyance
• Participants engaged socially with the machine (naming it, thanking it, seeking encouragement)
• Participants anticipated their preferences would shift from proactive to reactive assistance as expertise develops

Citation

@inproceedings{bremers2026proactive,
  title={Proactive vs. Reactive Machine Assistance during Fabrication: A Wizard-of-Oz Study of Machine Initiative during Novice Sewing},
  author={Bremers, Alexandra and Guimbretiere, Francois and Ju, Wendy},
  booktitle={Under Review},
  year={2026}
}