Field Type Validation of NBS Orders

[JohnNKing]

Story

As a hospital, so that NBS orders with invalid information can be fixed more quickly, I need validation of orders that identifies if any data is of the wrong type (e.g. date, string, number).

Context

Are there key values that, if of the wrong format, would cause things to break when received by the LIMS? For example:

• What if birth weight is provided in ounces instead of grams? Or vice versa? What if "1100g" is provided instead of "1100"?
• Gestational age not provided in weeks?
• What if the bar code is not provided in an expected format?
• What if yes/no question values are not in the desired format? (e.g. 0 vs 1, or "yes" vs "no")
• Utilize LOINC code value sets?

Pre-conditions

• A single set of validation rules are used (i.e. they are not state or hospital specific).

Acceptance Criteria

• A list of shared validation rules, based upon data gap analysis work to date, have been documented.
• The shared validations rules are used to validate incoming orders.
• We provide some means of capturing & persisting validation warnings (so they can be communicated back to the sender in the future).
• None of the validation rules added prevent the order from being delivered.

Tasks

Research

• Document list of standard validation rules
  • Review data gap for consistency across states
  • Test ReportStream's validation that may reject a message before it reached the ETOR/TI service (e.g. OBX NM without numeric in OBX.5; invalid OBX.6)

Engineering

• Document what RS is validating
• For each validation rule, document fhirpath for resource that needs validation (related to Document field-level transformations #636)
• Look into HapiFHIR validation library
• Look into fhirpath validations
• Are these validations applicable? Taking into account the hapifhir marshaling
• Determine how we want to implement validation rules
• Implement validation rules

Proposed Solution

Use a variation on the Rules Engine Design Pattern (read more here and here and here)

The rules would be defined in a json file with this shape:

Version 1

{
  "rules": [
    {
      "fieldName": "patientName",
      "fhirPath": "Bundle.entry.resource.ofType(Patient)",
      "required": true,
      "conditions": ["fieldExists('Patient.name')"],
      "validations": ["stringLength('Patient.name.family', min=1, max=100)"]
    }
  ]
}

Version 2

{
  "patienName": {
    "conditions": [
      "Patient.name.exists()"
    ],
    "validations": [
      "Patient.name.where(use='usual').given.exists()",
    ],
  }
}

Final version?

{
  "rules":
  [ {
      "name": "patienName",
      "conditions": [
        "Patient.name.exists()"
      ],
      "validations": [
        "Patient.name.where(use='usual').given.exists()",
      ]
    },
    {
      "name": "routingFields",
      "conditions": [
        // conditions
      ],
      "validations": [
        // validation rules
      ]
    }
  ]
}

And we'd have these classes/interfaces for the implementation:

• ValidationEngine
  • void addRule(ValidationRule rule)
  • void validate(FhirResource resource)
• ValidationRule
  • void validate(FhirResource resource)
  • boolean appliesTo(FhirResource resource)
• RuleLoader
  • List<ValidationRule> loadRules(String configPath)
• ExpressionInterpreter
  • boolean evaluateBooleanExpression(FhirContext context, String expression)

Universal Validations

• ETOR Intermediary Requirements
  • Routing: MSH-5 and/or MSH-6
• NBS Use Case
  • Card Number (though this will not work until OBX-3 translation is fixed via ReportStream ORM/OML Transformations Losing Data #900)

Findings for RS validation

• Field type errors happen in RS when converting HL7 to FHIR
• RS throws an error if there is a numeric type field with a non-numeric value

Possible categories of validation

• HL7 Data type validation
  • What if "1100g" is provided instead of "1100" for birth weight?
  • What if the bar code is not provided in an expected format?
  • Date validation (TS - Time Stamp)
• Unit type validation
  • Gestational age not provided in weeks?
  • What if birth weight is provided in ounces instead of grams? Or vice versa?
• Business / field value validation
  • What if yes/no question values are not in the desired format? (e.g. 0 vs 1, or "yes" vs "no", or "Y" vs "N")
  • Utilize LOINC code value sets?
  • value is in ["Y", "N"] => Patient.name.where(use='usual').exists(given = 'Jim' or given = 'Peter')
• Required fields/segments
  • Required segments
    • MSH, PID, etc
  • Required fields for parsing and routing
    • MSH-5 and/or MSH-6 present
  • Required fields for partners
• Combined-fields validation?
  • Card Number
  • Infant’s Last Name
  • Mother’s Last Name
  • Date/Time of Birth
  • Date/Time of Collection
  • Card Number is an AOE. ADPH need this data element to process the order

Definition of Done

• Threat model updated
• Code refactored for clarity and no design/technical debt
• Adhere to separation of concerns; code is not tightly coupled, especially to 3rd party dependencies.
• Source code is merged to the main branch.
• Unit test coverage of our code >= 90%
• Code is reviewed or developed by pair; 1 approval is needed but consider requiring an outside-the-pair reviewer.
• Build process updated
• Feature toggles created and/or deleted. Document the feature toggle.
• Source code documentation created when the code is not self-documenting.
• API documentation generated
• Code quality checks passed
• Security & privacy gates passed
• Load tests passed
• Documentation and diagrams created or updated
• API(s) are versioned
• Debug logging

Research Questions

• Optional: Any initial questions for research

Decisions

• Optional: Any decisions we've made while working on this story

Notes

• Optional: Any reference material or thoughts we may need for later reference

[scleary1cs]

-What validation is needed in the TI?
-Who needs to know if we are validating internally?
-Who should we notify if validation fails?
-How will we get asynchronous error messages bac to (RS?)?

[scleary1cs]

This Story helps complete the requirement:
MVP-REQ-04 - Each intermediary will create an MVP with the capability to support when the facility receives the entire order, and the intermediary is responsible for validating and transforming the order for PHL

[scleary1cs]

This Story helps complete the requirement:
MVP-REQ-05 - Each intermediary will create an MVP wherein the system accepts all order-related information from the facility. The intermediary is responsible for receiving the order, performing validation, and carrying out transformations for PHL LIMS

[scleary1cs]

This Story helps complete the requirement:
MVP-REQ-15 - Intermediaries will establish, utilize, and validate a common set of data elements

[scleary1cs]

This Story helps complete the requirement:
MVP-REQ-36 - Each intermediary will have the same validations performed for all data exchange formats (e.g, NIST)

[cjterdi]

Emily will provide a list of data types to test to confirm what RS is currently validating and inform us on future validation rules (date, coded data, strings etc.)