Validation rules cannot depend on field values (no predicate on cardinality/uniqueness, no value test in conditional)

[sorenwacker]

Problem

Validation rules cannot depend on field values. Two shapes are missing, and between them they account for every constraint I could not encode while modelling the SEEK definition format as a profile (seek-template-model).

1. A conditional rule tests presence, not value.

ConditionalRule parses its condition by splitting on whitespace and discarding the tokens AND, OR, NOT; whatever remains is treated as a field name and checked for presence (validators/rules.py:446-453). So "A OR B" and "(NOT A) OR B" work, but there is no way to write "required when this other field holds a particular value".

The SEEK case: an attribute whose data_type is Controlled Vocabulary must carry cv_terms. With any other data type, cv_terms is meaningless. Not expressible.

2. Cardinality and uniqueness cannot be scoped to a subset.

cardinality constrains the length of a list (min_items/max_items). uniqueness constrains all values of a field within a scope. Neither takes a predicate, so a rule about some of the children cannot be written.

The SEEK cases, all constraints the server enforces and rejects uploads for:

• Exactly one attribute per sample type sets is_display_column. It is the row's identity, so zero or two is a defect. Needs a count over the children that match a predicate, not a count of the children.
• At most one attribute per sample type has data_type: Registered Sample List (the Input attribute).
• The tags source, protocol, sample, data_file and other_material may each be used at most once per sample type — but attributes named Input are exempt from that count. So it is a uniqueness constraint scoped both by value set and by an exclusion.

Evidence this matters

I round-tripped 11 sample types, 8 extended metadata types, 146 sample attributes, 51 extended metadata attributes and 456 vocabulary terms through seek-template-model 0.2. The enum and required-field constraints work well — validate_extracted correctly rejected a bad data_type, a bad isa_tag and a missing isa_tag, the last being an error that previously only surfaced as a 422 from SEEK at upload time.

But the four constraints above had to be checked by a Python script sitting outside metaseed. That script immediately found a real defect the profile reported as valid:

CropXR extended metadata controlled vocabulary upload: 0 display columns, expected exactly 1

A template with no display column passes validate_extracted cleanly today. The whole point of moving this model into metaseed was to stop maintaining a separate checker, and these four rules are what keeps it alive.

Proposal

Add a where predicate to cardinality and uniqueness, and when/require to conditional.

- name: cv_terms_required_for_controlled_vocabulary
  type: conditional
  applies_to: [SampleAttribute]
  when: "data_type == 'Controlled Vocabulary'"
  require: [cv_terms]

- name: exactly_one_display_column
  type: cardinality
  applies_to: [SampleType]
  field: attributes
  where: "is_display_column == true"
  min_items: 1
  max_items: 1

- name: singleton_isa_tags
  type: uniqueness
  applies_to: [SampleAttribute]
  field: isa_tag
  unique_within: parent
  where: "isa_tag in ['source', 'protocol', 'sample', 'data_file', 'other_material'] and name != 'Input'"

A deliberately small expression grammar is enough for all of these: field op literal, with op in == != in "not in" > >= < <=, combined with and / or / not and parentheses. No function calls, no arithmetic, no traversal into other entities.

Definition of done

• where is accepted on rules of type cardinality and uniqueness; when absent, behaviour is byte-for-byte what it is today.
• when and require are accepted on rules of type conditional; the existing presence-only condition keeps working unchanged, so no existing profile changes behaviour.
• ValidationRuleSpec gains the three fields, keeping extra="forbid".
• The expression grammar is implemented for the operators listed above and documented in the profile-spec reference with one worked example per rule type.
• An expression naming a field that the entity does not declare is a spec error, raised at load, in the same loud style as _VALID_RULE_TYPES in validators/engine.py:138-151 — not silently skipped, since a rule that never runs lets invalid data pass.
• An expression that fails to parse is likewise a spec error at load, quoting the offending expression.
• Predicated rules run in both validate_dataset and validate_extracted, and report the offending entity plus the predicate that selected it.
• spec_add_rule in the Hub MCP takes where, when and require.
• spec_validate reports these rules in its issue list like any other.
• Tests, each proved red before green:
  • a Controlled Vocabulary attribute with no cv_terms fails; the same attribute with data_type: String and no cv_terms passes;
  • a sample type with zero display columns fails, one passes, two fails;
  • a sample type with two Registered Sample List attributes fails, one passes;
  • two attributes tagged protocol fail, but a second attribute tagged input and named Input passes;
  • a rule whose where names an unknown field raises at load;
  • an existing rule with no where/when produces identical results before and after the change.
• seek-template-model is updated to carry all four SEEK rules, and re-running the round-trip over the CropXR definitions reproduces the 0 display columns finding from metaseed rather than from an external script.

Context

Found while building seek-template-model 0.1/0.2, a profile whose instances are SEEK template definitions rather than research metadata. Related to #210, which is about identity being inferred rather than declared; this one is about constraints being inexpressible rather than misinferred.

[sorenwacker]

Design written before any code: docs/architecture/decisions/003-value-dependent-validation-rules.md (ADR 003, f7da2ba).

Three findings in the code that change the scope of this issue. Each was verified, not inferred:

1. validate_extracted runs no validation rules at all. It calls ExtractionContext.validate_instance (agent/core.py:376-407), which iterates the entity spec's fields and applies field-level constraints only — it never reads ProfileSpec.validation_rules and never builds a ValidationEngine (grep for validation_rules in that module returns nothing). So the display-column defect this issue reports was not primarily caused by the absence of predicates: a plain non-predicated cardinality rule would have been ignored on that path too. The DoD item 'predicated rules run in both validate_dataset and validate_extracted' needs a prerequisite change that is not about predicates.

2. UniquenessRule in the engine never fires, by its own docstring (validators/rules.py:677-697): create_engine_for_entity builds a fresh instance per record so it never sees a second value. Uniqueness is actually enforced by DatasetValidator._load_uniqueness_rules (validators/dataset.py:159) and _validate_uniqueness (dataset.py:358). Adding to UniquenessRule alone would be dead code.

3. The facade/MCP validate_dataset path cannot see a child collection. Children are separate EntityNodes linked by parent_id and each node is validated from node.instance.model_dump(), so SampleType.attributes is empty there. The display-column rule can only fire on the nested-dict path (DatasetValidator, validators/api.py).

Also: spec_add_rule/spec_update_rule accept only name, type, message, applies_to, field, reference — not condition, min_items, max_items or unique_within — so none of the affected rule types is authorable end-to-end over MCP today; and SpecBuilder.validate inspects no validation rules at all.

Where the design differs from the proposal: it recommends a structured predicate ({field, op, value} with all/any/not) rather than the string form, because rules are part of content_hash and canonical_json sorts keys — so a mapping is canonical for free, whereas 'a==b' and 'a == b' are two different hashes and the comparator would report a whitespace edit as BREAKING. The string survives as the rendering, and can be reintroduced later as an input syntax compiled to the stored form.

Everything else is in the ADR: safe evaluation without a parser and its structural bounds, predicate scope, spec_version 0.7 and its failure mode on older metaseed, comparator classification per case with a required_bump table, concrete error-message shapes for all four SEEK constraints, the guided rule-editor UI, and a first slice (cardinality first).