Skip to content

Latest commit

 

History

History
261 lines (224 loc) · 23.3 KB

conformance.md

File metadata and controls

261 lines (224 loc) · 23.3 KB

SBOL 3.1.0 Conformance Report

This file is generated by cargo run -p sbol3 --bin generate-conformance-report. It is committed and CI runs git diff --exit-code docs/conformance.md to enforce freshness after every change to validation_rule_statuses().

Source spec: spec/SBOL3.1.0.md (version 3.1.0).

Headline coverage

109 of 109 machine-checkable spec rules are fully implemented (100.0%).

The catalog tracks all 149 SBOL 3.1.0 validation rules. Appendix B (p.2837–2840) marks 40 of them with the ▲ symbol: weak-REQUIRED conditions whose violations are NOT to be machine-reported. Those are tracked separately from the headline percentage: the runtime signals them as RuleCoverage::not_applied { MachineUncheckable }, and the validator emits at most warnings (never errors) when its local subset can decide a sub-case.

Count
Total rules in spec 149
▲ machine-uncheckable per Appendix B 40
Machine-checkable 109
  → fully implemented 109
  → not yet implemented 0

Status taxonomy

Every rule classifies into exactly one of five statuses. The Blocker column carries the secondary axis where it applies.

Status Count Algorithm Diagnostics Blocker meaning
Error 55 Complete Errors (none)
Warning 17 Complete Warnings (none)
Configurable 37 Complete; scope or severity varies by config Errors / Warnings per config Which configuration axis: Resolver, Ontology snapshot, Policy, or External
MachineUncheckable 40 May have local subset Warnings only, on positively-decidable cases Why the spec marks it ▲: Ontology, Policy, or External
Unimplemented 0 None Never emits What's needed: Ontology data, Resolver protocol, Policy decision
Total 149

Machine-checkable gaps

None. Every machine-checkable rule is fully implemented.

Runtime coverage signal

Every ValidationReport carries a RuleCoverage value with three buckets:

  • fully_applied: every spec case for this rule is decidable with the current configuration and was evaluated against this document.
  • partially_applied: the local subset ran but the rule's full coverage is configuration- or snapshot-bounded. Each entry carries a coverage_kind: OntologyKnownTermsOnly, LocalReferencesOnly, LexicalShapeOnly, or PolicyDefaultUndecided.
  • not_applied: either the rule is Unimplemented (no local algorithm; reason: Deferred(blocker)) or it is a ▲ rule whose violations the spec asks tools not to report (reason: MachineUncheckable).

Callers wanting a strict CI gate use Document::check_complete (or the CLI's --treat-partial-as-errors) which returns an error whenever partially_applied is non-empty. The default Document::check returns errors only on hard diagnostics, leaving coverage gaps observable but non-fatal.

Error

55 rules. Algorithm complete; MUST violations emit as Severity::Error.

Rule Note
sbol3-10102 TopLevel URL pattern is checked for local URL identities
sbol3-10103 TopLevel URL prefix collisions are checked within the document
sbol3-10104 child URL pattern is checked for composite child references
sbol3-10106 disjoint SBOL class combinations are rejected
sbol3-10109 allowed SBOL properties are table-driven
sbol3-10110 Table 23 cardinality is table-driven
sbol3-10111 Table 23 value kinds are enforced with strict XSD datatype + lexical-form checks
sbol3-10112 local child references are checked
sbol3-10201 displayId lexical form is checked
sbol3-10202 self derivation is checked
sbol3-10203 wasDerivedFrom cycles are checked within the document
sbol3-10204 wasGeneratedBy / Usage.entity cycles are checked within the document; cross-document detection is deferred
sbol3-10301 http and https TopLevel namespace prefix is checked
sbol3-10501 Sequence elements requires encoding
sbol3-10601 Component type values from Table 2 are counted
sbol3-10702 orientation vocabulary is checked
sbol3-10801 roleIntegration vocabulary is checked
sbol3-10802 SubComponent roles require roleIntegration
sbol3-10803 SubComponent cannot instanceOf containing Component
sbol3-10804 component instanceOf cycles are checked
sbol3-10805 SubComponent location overlap is checked for ranges
sbol3-10806 SubComponent location and sourceLocation lengths are compared when known
sbol3-10807 SubComponent location coverage of instanceOf sequence is checked when known
sbol3-10901 ComponentReference inChildOf containment is checked for Component parents
sbol3-10902 nested ComponentReference inChildOf containment is checked locally
sbol3-10903 ComponentReference refersTo ComponentReference containment is checked locally
sbol3-10904 ComponentReference refersTo Feature containment is checked locally
sbol3-11001 LocalSubComponent type values from Table 2 are counted
sbol3-11013 LocalSubComponent location overlap is checked for ranges
sbol3-11101 ExternallyDefined type values from Table 2 are counted
sbol3-11201 SequenceFeature location overlap is checked for ranges
sbol3-11301 Location orientation vocabulary is checked
sbol3-11302 Feature Location sequence membership is checked locally
sbol3-11303 SubComponent sourceLocation sequence membership is checked locally
sbol3-11401 Range start bounds are checked when sequence elements are local
sbol3-11402 Range end bounds are checked when sequence elements are local
sbol3-11403 Range end must be greater than or equal to start
sbol3-11501 Cut bounds are checked when sequence elements are local
sbol3-11701 Constraint subject containment is checked
sbol3-11702 Constraint object containment is checked
sbol3-11703 Constraint subject and object cannot be identical
sbol3-11901 Participation must contain exactly one participant kind
sbol3-11902 Participation participant containment is checked
sbol3-11903 higherOrderParticipant containment is checked
sbol3-12001 Interface input containment is checked
sbol3-12002 Interface output containment is checked
sbol3-12003 Interface nondirectional containment is checked
sbol3-12101 CombinatorialDerivation strategy vocabulary is checked
sbol3-12102 enumerate strategy disallows unbounded cardinalities
sbol3-12103 duplicate variable features by variable URI are rejected
sbol3-12201 VariableFeature cardinality vocabulary is checked
sbol3-12202 VariableFeature variable containment in template is checked
sbol3-12203 variantCollection members are checked recursively when present locally
sbol3-12204 variantDerivation cycles are checked locally
sbol3-12808 Attachment hash requires hashAlgorithm

Warning

17 rules. Algorithm complete; SHOULD violations emit as Severity::Warning.

Rule Note
sbol3-10107 multiple concrete SBOL rdf:type values are warnings when not disjoint
sbol3-10108 SBOL properties without SBOL rdf:type produce warnings
sbol3-10205 workflow role recommendation requires Table 20 and 21 policy
sbol3-10604 Component type IRIs that resolve to known SBO terms outside the SBO:0000236 physical entity representation branch produce warnings; unknown IRIs remain undecided
sbol3-11004 LocalSubComponent type IRIs that resolve to known SBO physical-entity terms outside SBOL Table 2 produce warnings; unknown IRIs remain undecided
sbol3-11104 ExternallyDefined type IRIs that resolve to known SBO physical-entity terms outside SBOL Table 2 produce warnings; unknown IRIs remain undecided
sbol3-11704 Constraint restriction is checked against Tables 8, 9, and 10
sbol3-12507 Model framework IRIs that resolve to known SBO terms outside the SBO:0000004 modelling-framework branch produce warnings; unknown IRIs remain undecided
sbol3-12807 local hashAlgorithm literals warn unless they use the recommended sha3-256 value
sbol3-12901 workflow stage recommendation policy is deferred
sbol3-12902 workflow association recommendation policy is deferred
sbol3-13001 workflow usage recommendation policy is deferred
sbol3-13401 om:Measure type IRIs that resolve to known SBO terms outside the SBO:0000545 Systems Description Parameter branch produce warnings; unknown IRIs remain undecided
sbol3-13501 local om:Unit sbol:name and om:label literal values are compared
sbol3-13502 local om:Unit sbol:description and om:comment literal values are compared
sbol3-14201 local om:Prefix sbol:name and om:label literal values are compared
sbol3-14202 local om:Prefix sbol:description and om:comment literal values are compared

Configurable

37 rules. Behavior depends on configuration. The Blocker column names the axis:

  • Resolver: algorithm needs --external-mode provided|allowed for full scope. Without one, coverage records LocalReferencesOnly.
  • Ontology: algorithm correct for every snapshot-known term; out-of-snapshot terms left undecided by design.
  • Policy: algorithm runs at the Conservative PolicyOptions default; Strict and Lenient modes alter emit behavior. See docs/policies/<id>.md.
  • External: local-only mode is the implementation. The spec scope is structurally unreachable without external infrastructure (a global registry); the validator does what it can locally and documents the boundary.

By configuration axis

Axis Count
Resolver 3
Ontology 18
Policy 15
External 1
Rule Axis Note
sbol3-10101 External local Identified object identities must be IRIs; global URI uniqueness outside the document is not proven
sbol3-10105 Policy unknown SBOL predicates and rdf:type IRIs are checked locally
sbol3-10113 Resolver non-local references are checked when targets are present in the document
sbol3-10114 Resolver resolver-aware validation can warn on unresolved external TopLevel references and reject resolved wrong target classes; default validation remains offline
sbol3-10503 Ontology IUPAC nucleotide and protein alphabets are checked for Table 1 EDAM encodings
sbol3-10505 Ontology bundled ontology facts warn when a known encoding term is outside the EDAM textual format branch; unknown/custom terms remain undecided
sbol3-10607 Ontology bundled SO branch facts warn when explicit DNA/RNA type values include more than one known topology term
sbol3-10608 Ontology bundled SO branch facts warn when known topology or strand terms appear without an explicit DNA/RNA type
sbol3-10612 Ontology bundled SO sequence-feature branch facts warn when known SO feature roles appear without an explicit DNA/RNA type
sbol3-10613 Ontology bundled SO sequence-feature branch facts warn when explicit DNA/RNA types do not have exactly one known SO feature role; unknown/custom roles remain undecided
sbol3-10616 Ontology known local Component types with bundled cross-listing facts require a compatible known local Sequence encoding when all local sequence encodings are known
sbol3-10617 Policy local Sequence objects with the same exact encoding warn when literal elements differ
sbol3-11007 Ontology bundled SO branch facts warn when explicit DNA/RNA type values include more than one known topology term
sbol3-11008 Ontology bundled SO branch facts warn when known topology or strand terms appear without an explicit DNA/RNA type
sbol3-11011 Ontology bundled SO sequence-feature branch facts warn when known SO feature roles appear without an explicit DNA/RNA type
sbol3-11012 Ontology bundled SO sequence-feature branch facts warn when explicit DNA/RNA types do not have exactly one known SO feature role; unknown/custom roles remain undecided
sbol3-11107 Ontology bundled SO branch facts warn when explicit DNA/RNA type values include more than one known topology term
sbol3-11108 Ontology bundled SO branch facts warn when known topology or strand terms appear without an explicit DNA/RNA type
sbol3-11705 Policy direct sameOrientationAs and oppositeOrientationAs contradictions are checked for local Feature orientations
sbol3-11706 Policy Range and Cut locations on the same Sequence are checked for Table 10 relations
sbol3-11803 Ontology bundled Table 11/SBO facts warn when the known local Interaction type count is not exactly one; unknown/custom terms remain undecided
sbol3-11804 Ontology bundled Table 11/Table 12 cross-list facts warn when known local Participation roles do not match known local Interaction types
sbol3-11906 Ontology bundled Table 12/SBO facts warn when the known local Participation role count is not exactly one; unknown/custom terms remain undecided
sbol3-12104 Policy local templates with no Features produce a warning
sbol3-12105 Policy local derived Component child Features are checked for template Feature provenance
sbol3-12106 Policy local derived Collection members are checked for derivation provenance
sbol3-12107 Policy local derived Component types are checked against template Component types
sbol3-12108 Policy local derived Component roles are checked against template Component roles
sbol3-12109 Policy local static Feature derivations compare core Feature properties
sbol3-12110 Policy local static template Features are checked for exactly one derived Feature
sbol3-12111 Policy local variable Feature derivation counts are checked against cardinality
sbol3-12112 Policy local derived SubComponent variants are resolved through variants, variantCollections, and variantDerivations
sbol3-12113 Resolver local derived Features are checked against template Constraint relations when resolver facts are sufficient
sbol3-12114 Policy local derived variable Feature roles are checked against template Feature roles
sbol3-12115 Policy local type-determining referent types are resolved for Feature derivation checks
sbol3-12504 Ontology Model language values from known non-EDAM bundled ontologies warn; unknown/custom IRIs remain undecided
sbol3-12803 Ontology known non-EDAM format terms warn; unknown/custom terms remain undecided

▲ MachineUncheckable

40 rules. Appendix B ▲: violations are NOT to be machine-reported. The validator runs a local subset for some of these and emits warnings on positively-decidable cases; the broader spec rule is always recorded in RuleCoverage::not_applied { MachineUncheckable }.

Local subset? Count
Yes, emits warnings on positively-decidable cases 31
No, coverage signal only 9
Rule Local subset? Blocker Note
sbol3-10502 Yes Ontology bundled ontology facts reject known terms that are not Sequence encoding terms; unknown/custom terms remain undecided
sbol3-10504 Yes Ontology known aliases of exact Table 1 sequence encodings require the preferred SBOL Table 1 URI; broader well-described-by reasoning is deferred
sbol3-10602 Yes Ontology bundled ontology facts classify known Component type and type-modifier terms; unknown/custom terms remain undecided
sbol3-10603 No Ontology well-described-by ontology rule is not locally decidable
sbol3-10605 Yes Ontology bundled ontology facts detect conflicts between known Component type families; unknown/custom terms remain undecided
sbol3-10606 Yes Policy default validation remains undecided, but topology-complete validation rejects DNA/RNA objects without a known SO topology term
sbol3-10609 Yes Ontology bundled ontology branch facts reject known Component roles that are locally decidable as incompatible with known Table 2 types; unknown/custom terms remain undecided
sbol3-10610 Yes Ontology bundled ontology facts classify known Component role terms; unknown/custom terms remain undecided
sbol3-10611 No Ontology well-described-by role rule is not locally decidable
sbol3-10614 Yes Ontology local Sequence encodings and Component types present in bundled ontology facts are checked for compatibility; unknown/custom terms remain undecided
sbol3-10615 Yes Ontology known local Sequence encoding families from bundled ontology facts are checked for conflicts
sbol3-10701 Yes Ontology bundled ontology facts classify known Feature role terms; unknown/custom terms remain undecided
sbol3-11002 Yes Ontology bundled ontology facts classify known Component type and type-modifier terms; unknown/custom terms remain undecided
sbol3-11003 No Ontology well-described-by ontology rule is not locally decidable
sbol3-11005 Yes Ontology bundled ontology facts detect conflicts between known Component type families; unknown/custom terms remain undecided
sbol3-11006 Yes Policy default validation remains undecided, but topology-complete validation rejects DNA/RNA objects without a known SO topology term
sbol3-11009 Yes Ontology bundled ontology branch facts reject known LocalSubComponent roles that are locally decidable as incompatible with known Table 2 types; unknown/custom terms remain undecided
sbol3-11010 No Ontology well-described-by role rule is not locally decidable
sbol3-11102 Yes Ontology bundled ontology facts classify known Component type and type-modifier terms; unknown/custom terms remain undecided
sbol3-11103 No Ontology well-described-by ontology rule is not locally decidable
sbol3-11105 Yes Ontology bundled ontology facts detect conflicts between known Component type families; unknown/custom terms remain undecided
sbol3-11106 Yes Policy default validation remains undecided, but topology-complete validation rejects DNA/RNA objects without a known SO topology term
sbol3-11109 Yes Ontology known ExternallyDefined protein and simple-chemical definition resource prefixes are checked against Table 17 recommendations; unknown resources remain undecided
sbol3-11801 Yes Ontology bundled ontology facts classify known Interaction type terms; unknown/custom terms remain undecided
sbol3-11802 Yes Ontology bundled ontology facts detect conflicts between known Interaction type terms; unknown/custom terms remain undecided
sbol3-11904 Yes Ontology bundled ontology facts classify known Participation role terms; unknown/custom terms remain undecided
sbol3-11905 Yes Ontology bundled ontology facts detect conflicts between known Participation role terms; unknown/custom terms remain undecided
sbol3-12301 Yes Resolver Implementation prov:wasDerivedFrom IRIs that resolve in-document must refer to a Component; external derivations remain unchecked
sbol3-12302 Yes Resolver in-document Components referenced by Implementation prov:wasDerivedFrom are checked for type-term conflicts via the bundled ontology; external Components remain unchecked
sbol3-12303 No Policy faithful built Component check is not locally decidable
sbol3-12501 Yes Resolver resolver-aware validation can check Model source availability with caller-provided content providers; default validation remains offline
sbol3-12502 No Resolver Model language IRI must specify a language; requires resolving the IRI
sbol3-12503 Yes Ontology known aliases of Table 15 Model language URIs require the preferred Table 15 URI; broader well-described-by reasoning is deferred
sbol3-12505 No Resolver Model framework IRI must specify a framework; requires resolving the IRI
sbol3-12506 Yes Ontology known aliases of Table 16 Model framework URIs require the preferred Table 16 URI; broader well-described-by reasoning is deferred
sbol3-12801 Yes Resolver resolver-aware validation can check Attachment source availability with caller-provided content providers; default validation remains offline
sbol3-12802 No Ontology Attachment format semantic check is deferred
sbol3-12804 Yes Resolver local size literals are checked for nonnegative byte counts; resolver-aware validation compares size against resolved content bytes
sbol3-12805 Yes Resolver local hash literals are checked for hexadecimal digest shape; resolver-aware validation verifies sha3-256 against resolved content bytes
sbol3-12806 Yes Policy Appendix B ▲: spec asks tools not to report. Local subset emits a warning when the hashAlgorithm literal is not a registry-style token; PolicyOptions::hash_algorithm_registry can opt into Strict (error on unknown algorithm) or Lenient (skip).

Compliance profile claims

These claims are provisional and describe what is currently exercised by the test suite. Release-grade profile boundaries will be finalized before the first crates.io publish.

  • Structural import / export: Turtle parse and serialize for the SBOLTestSuite fixture corpus is exercised by representative_fixtures_round_trip_as_normalized_rdf in crates/sbol3/tests/sbol3_fixtures.rs.
  • Lossless RDF round-trip: Parse → write → reparse yields equal normalized triple sets for every representative fixture.
  • Typed round-trip with extension preservation: typed_round_trip_preserves_extension_triples_on_typed_objects verifies that non-SBOL/PROV/OM predicates survive a typed rebuild.
  • Cross-implementation RDF I/O equivalence vs libSBOLj3: 33 SBOLTestSuite SBOL3 fixtures (the full valid-input set in tests/sbol3_fixtures_manifest.tsv) parse and serialize to normalized triple sets identical to libSBOLj3 1.0.5.2 reference output. CI runs cargo test --test cross_impl against committed *.libSBOLj3.expected.ttl files in tests/fixtures/cross-impl/. This compares parser/serializer output only; the two libraries' validator outputs have not been systematically compared.
  • Property-based testing: crates/sbol3/tests/properties.rs exercises eight spec-derived properties under proptest at the default case count, including catalog coverage partitioning and override-builder semantics.
  • Fuzz coverage: cargo fuzz run read_turtle, cargo fuzz run round_trip, cargo fuzz run validate, and cargo fuzz run round_trip_validate (see fuzz/) exercise the parser and validator under libFuzzer. CI smoke runs the parser targets for 60s each; the validator targets are local-only until corpus and budget are tuned.

Per-rule regression cases

Every rule with an algorithm has at least one regression case in crates/sbol3/tests/rule_cases/, organized by spec section. The catalog is enforced by implemented_validation_rules_have_regression_cases and validation_rule_regression_cases_report_expected_rule_ids in crates/sbol3/tests/validation_rules.rs.

Selected high-value clusters also have positive cases (the rule must NOT fire on a valid example), verified by positive_rule_cases_do_not_report_their_rule.