Run-Rules.mdx

title	Run Rules
description	Order and co-schedule the flows in a run with a rules.yaml file in your flows repository.
keywords	rules.yaml file based rules run-with run-without co-scheduling sequencing rules

By default all flows run in parallel on the QA Wolf run infrastructure, but there are times when you want to sequence some flows to run before others.

• Quick sanity checks before longer or more complex flows.
• Multi-user flows where state is shared between flows.
• When the environment can't handle the load of an entire test suite.

You define Run Rules in a rules.yaml file in your flows repository. There is no Run Rules screen.

Run Rules act on the flows inside a run. They apply whether a trigger started the run or you did, and they never start a run themselves.

How Run Rules work

Configuring groups

A rule names flows or tags. A tag stands for every flow carrying it, so one rule can order or group many flows at once.

A rule never adds a draft or in-maintenance flow to a run, and triggers skip them. A draft runs only when you select it yourself.

Ordering

run-after and run-before put flows in sequence. To run flows in a sequence like A → B → C, give B both: run-after: [A] and run-before: [C].

By default, when a flow in the earlier group fails, the flows in the later group are marked failed without running. Set fail-if-priors-fail: false on the subject to let them run anyway. The setting covers both run-after and run-before on that subject.

Runs you start yourself

Runs a trigger starts always follow the rules. Runs you start yourself follow them by default, and you can choose to skip them.

If you select a flow that a rule runs after other flows, the rule requires those earlier flows too. When you start a run in the platform and your selection leaves any out, QA Wolf asks whether to add them. Choose Yes, apply rules to add them, or No, skip rules to run your selection without the rules. If a required flow is a draft, in maintenance or bugged, only No, skip rules is available.

Rerunning a run keeps its original choice.

From the CLI, qawolf run create adds the required flows automatically. Pass --ignore-rules to skip the rules.

Rules file

Add a rules.yaml file to the root of your repository, alongside package.json. QA Wolf reads every run rule from it.

Where the file lives and what it contains

QA Wolf looks for a file named exactly rules.yaml at the root of your repository. A rules.yml file with the same content is reported as a violation telling you to rename it — QA Wolf never reads it silently.

The file has a version key and a rules map. Each key inside rules is a subject — the name of a flow or a tag — and its value sets one or more rules for that subject.

Key	Type	Default	Sets
version	number	required	Must be 1.
rules	map	required	Maps each subject to its rules.
run-after	list of names	—	The subject runs after every flow or tag listed.
run-before	list of names	—	The subject runs before every flow or tag listed.
run-with	list of names	—	The subject starts at the same time as every flow or tag listed.
run-without	list of names	—	The subject never starts at the same time as any flow or tag listed.
fail-if-priors-fail	boolean	true	Only valid next to run-after or run-before. See The two modifiers.
fail-together	boolean	false	Only valid next to run-with. See The two modifiers.

A subject needs at least one of run-after, run-before, run-with, or run-without — a subject with none of these is a violation, not a no-op.

Example:

# checkout combines two rule kinds on one subject, and both modifiers appear elsewhere in the file.
version: 1
rules:
  checkout:
    run-after: [login]
    run-without: [flaky-upload]
  smoke:
    run-before: [regression]
    fail-if-priors-fail: false
  billing:
    run-with: [invoicing]
    fail-together: true

A branch with no rules.yaml file runs with no run rules. So does a file whose rules map is empty (rules: {}).

A `rules.yaml` file larger than 10 MB can't be read, and is treated the same as an invalid file. See [When the file can't be used](#when-the-file-cant-be-used).

Naming a flow or a tag

Every name — a subject, or a name inside run-after, run-before, run-with, or run-without — is matched against both your flow names and your tag names.

If a name matches both a flow and a tag, QA Wolf can't tell which you mean. Write flow:<name> or tag:<name> instead of the bare name. Until you do, the whole file is invalid.

Example:

# flow:checkout and tag:checkout name different things when a flow and a tag share a name.
version: 1
rules:
  "flow:checkout":
    run-after: [login]
  "tag:checkout":
    run-after: [smoke]

If a flow or tag is genuinely named `flow:something` or `tag:something`, you can't reference it in `rules.yaml` at all — QA Wolf always reads a `flow:` or `tag:` prefix as the escape hatch above. Rename the flow or tag instead.

The two modifiers

Two boolean keys fine-tune how a rule behaves. Each is only valid next to the rule kind it modifies — setting one without a matching rule makes the file invalid.

• fail-if-priors-fail (next to run-after or run-before, default true) — controls whether flows in the later group are marked failed, without running, when a flow in the earlier group fails. See Ordering for what this looks like in a run. Set it to false to let the later flows run anyway.
• fail-together (next to run-with, default false) — when true, a flow in the group failing its final attempt stops every flow in the group that is still running. See What isn't visible in the file for how groups are built.

Rules apply per branch

Each environment is tied to one branch of your repository, and a rules.yaml file governs only the branch it's committed to. There's no way to write one set of rules that covers every environment — production and staging each read whichever rules.yaml exists on their own branch, independently of each other.

This means a rules change for production only takes effect once it reaches your production branch. Adding or editing rules on a different branch — staging, or a pull request — has no effect on production until that same change merges into production's branch too.

When the file can't be used

QA Wolf checks rules.yaml for problems as you write it. Invalid YAML, an unrecognized key, a modifier without its matching rule kind, and a few other mistakes all show up as violations against the file, the same way any other problem in your project would.

One problem makes the whole file invalid, and every rule on that branch stops applying. That includes a name that matches no flow or tag, a name that matches both, a flow name two flows share, and ordering rules that form a loop.

A run on a branch whose rules.yaml can't be used is refused. There is no fallback:

• The file is invalid or larger than 10 MB: the run is refused right away and isn't retried. In the platform, the Run button is disabled until you fix the file.
• The file couldn't be read from GitHub: a run a trigger starts is retried for up to 30 minutes before it's refused. A run you start yourself is refused right away, so try again.

The platform won't let you delete or rename a flow that rules.yaml names. Renaming or deleting a tag the file names isn't blocked, and leaves the file invalid. Update rules.yaml first.

What isn't visible in the file

run-with and run-without describe one pair of names at a time, but QA Wolf builds groups from every rule in the file together, and checks those groups against each run before it starts. Several behaviors follow from that which the file itself doesn't show.

• Groups grow transitively. If checkout is run-with billing, and billing is run-with invoicing, all three start together — even though nothing in the file names checkout and invoicing together directly.
• A run-without rule only needs to be written once. It keeps two flows apart no matter which one is picked first, so you don't need to add the rule to both flows.
• fail-together covers the whole group. Set it on any one flow in a run-with group, and it applies to every flow in that group — you don't need to repeat it on each flow. It takes effect when a flow in the group fails its final attempt, after any retries: every flow in the group that is still running is then stopped. Flows that already finished keep their result.
• A run applies none of the file's run-with and run-without rules if even one of them can't be honored. Before a run starts, QA Wolf checks every run-with and run-without rule against the flows in that run and against the most flows the run could ever have going at once. A rule can't be honored when it would combine two flows that a run-without rule keeps apart, when it would combine two flows an ordering rule (run-after/run-before) already keeps in sequence, when the group it forms is bigger than the run could ever have going at the same time, or when two groups would end up waiting on each other and neither could ever start. If any single rule fails one of these checks, the whole run applies none of the file's run-with and run-without rules — not only the one that failed. Ordering rules are unaffected and keep applying. QA Wolf is alerted whenever this happens, so it shouldn't go unnoticed for long — but it does mean a rule you're expecting to apply may quietly not, for that run.
• Too many possible pairings has the same effect. A tag-to-tag rule expands into every flow carrying the first tag paired with every flow carrying the second. If a run's run-with and run-without rules add up to more than 20,000 such pairings, QA Wolf applies none of the file's grouping rules to that run either.
• This is checked per run, so the same file can behave differently from one run to the next. A run with a lower concurrency limit, or one that happens to include a flow another run leaves out, can end up with none of its grouping rules applied, while another run built from the exact same file applies them without issue.

Keep `run-with` groups small, and avoid combining `run-with` with `run-without` or an ordering rule between the same two flows. Those are the most common reasons a run ends up applying none of its grouping rules — check them first if co-scheduling doesn't seem to be working.