Agent audio prompts should emit PROSE captions, not comma-separated tags (ACE-Step 1.5)

[mabry1985]

Agent audio prompts should emit PROSE captions, not comma-separated tags (ACE-Step 1.5)

TL;DR: the caption guidance in PR #23 optimizes for the format that loses. We A/B'd it on the
ML node and prose beat tags across 3 genres and 2 seeds. The guidance came from our docstring,
not from ACE-Step — we were wrong, and we propagated it to you. Sorry about that. The fix is
less work than what's currently shipped: drop the reduction constraint and let the agent write
prose.

What PR #23 currently tells the model

Three places, all explicit:

AgentInstructions.swift   "CAPTION, not prose: comma-separated tags, genre first, then
                           instruments, mood, an explicit BPM and vocal style
                           ("indie folk, acoustic guitar, piano, warm, 95 BPM, female
                           vocal"); 3–7 tags, no contradictory genres."

ToolDefinitions.swift     "...for ACE-Step write a CAPTION of comma-separated tags (genre
  (generate prompt)        first, then instruments, mood, an explicit BPM, vocal style),
                           not prose; 3–7 tags, no contradictory genres."

ToolDefinitions.swift     "Caption for the new audio — comma-separated tags (genre,
  (edit prompt)            instruments, mood, explicit BPM, vocal style), not prose."

The evidence

ComfyUI workflow, 3 tracks × 2 formats × 2 seeds = 12 generations, one Run:

track	genre	bpm	prose	tags
Big K.R.I.T. — Higher Calling	neo-soul	103	471 ch	208 ch
Bas — Passport Bros	afrobeats	152	436 ch	218 ch
J. Cole — c l o s e	boom-bap	172	382 ch	199 ch

Controlled: verified programmatically that per track+seed the two encoder nodes differ in
exactly one field — the caption. Lyrics, seed, BPM, duration, keyscale, time signature,
language, sampler settings identical, and both arms share the same empty latent. Turbo config
copied from the known-working workflow (shift 3, 8 steps, cfg 1, euler/simple).

Prose won across all three genres at both seeds.

Why — this isn't a preference, it's distribution

acestep-captioner is, per its HF card, "the annotation model used by ACE-Step v1.5 for training
data labeling" — and it emits prose. ACE-Step labeled 1.5's own training corpus with prose
captions. So prose isn't merely tolerated; it's what the model saw in training.

The prose arm above is literally that captioner's verbatim output on real tracks. Our 428-track
corpus is labeled this way and it's what our LoRA training uses.

The tags rule appears to be v1-era/community lore. A likely reason it keeps getting re-derived:
ComfyUI's core node is TextEncodeAceStepAudio1.5 with a field named tags. The field name is
not evidence about the format.

Caveats (please weigh these — this is not a scored eval)

• Subjective listening, n=12, one listener, one corpus (hip-hop / soul / afrobeats). No metric,
  no blind protocol. Directionally strong and consistent, but not a benchmark.
• The tags arm was a hand reduction of each prose caption, not the output of our live
  Qwen2-Audio tag pipeline. So "prose > tags" is proven; "prose > our tag pipeline" is inferred.
• Untested on rock / electronic / classical / metal.

Suggested change

Drop the "not prose / 3–7 tags / comma-separated" constraint from the three strings above and let
the agent write a descriptive prose caption — genre, instruments, vocal character, production
texture, mood, ending with an explicit BPM. Roughly the shape of what the captioner emits:

A smooth, soulful neo-soul track built on a steady, laid-back hip-hop drum loop and a warm,
melodic bassline. A male rapper delivers a brief, conversational verse before giving way to a
powerful female lead vocal... The production is clean and warm, with a touch of reverb on the
vocals, creating an intimate yet groovy atmosphere reminiscent of late '90s R&B. 103 BPM.

Keep unchanged — these were right and are unaffected:

• explicit BPM in the caption (the captioner does this too, ending with "103 BPM.")
• section-tagged lyrics ([intro]/[verse]/[chorus]/... at ~2–3 words/sec)
• seed reuse for consistent takes

Nothing changes on the adapter — /v1/audio/generations takes the caption as a free-text prompt
either way. This is purely prompt craft on your side.

Refs: protoLab#22 (the audio contract) · write-up with full method and caveats in
experiments/acestep-lora/RESULTS.md on the lab node.