Allow for customizing the prompt

[bartekpacia]

User feedback on assertWithAI

I like assertWithAI for detecting defects. It helps to limit the unexpected changes but I miss something to tell the model what defects it should ignore. For example, localization changes, I have one situation where I test the app in a spanish locale but since the email for the login has english names, the model reported a defect that the locale doesn't match.

To be honest, we expected this feedback (see #1222 (comment)), but preferred to not implement it until users actually request it.

Ideas

(ordererd from most basic to most powerful)

1. Per-command list of false positives (injected into the final prompt)

- assertWithAI:
    assertion: There is a login screen visible
    falsePositives:
      - Both English and Spanish text being visible
      - <something-else>

Cons:

• why limit the flexibility? Feels wrong to me. Better just give users the control of the whole prompt.

2. Per-command list customizable prompt

- assertWithAI:
    assertion: There is a login screen visible
    prompt: Ignore problems related to both English and Spanish being visible

Cons:

• our current prompt is quite long and including it all every time, on the commands, would be tedious.

3. Customize the default prompts for AI-powered commands

Allow for customization on per-workspace level, with possible override on per-command level.

Define prompts

We'd let you configure the prompt template in .maestro/config.yaml:

ai:
  defaultPrompts:
    assertNoDefectsWithAI: ./prompts/my_assert_no_defects_ai_prompt_template.txt
    assertWithAI: ./prompts/my_assert_with_ai_prompt_template.txt
  
  customPrompts:
    noBadWords: ./prompts/badwords_filter.txt

That's what the workspace directory hierarchy would look like in this example:

.
├── config.yaml
└── prompts
    ├── badwords_filter.txt
    ├── my_assert_no_defects_ai_prompt_template.txt
    └── my_assert_with_ai_prompt_template.txt

Of course, users would be free to structure it however they want.

Use prompts

We would introduce some syntax to refer to the prompts prefefined on the per-workspace level in the per-command level. For example:

- assertWithAI:
    prompt: noBadWords # refers to the contents of the "noBadWords" prompt

Or, to not introduce new arguments, we could re-use JavaScript for this:

- assertWithAI:
    prompt: ${prompts.noBadWords} # "prompts" would be the JS object, set by Maestro