README.md

FHIR Pipes 🔥

• Pieces of FHIR data pipelines
• Split up data processing in time and space using:
  • Event-Driven ETL (Extract, Transform, Load)
  • Scalable transformation functions
• More focused on comparing mapping and messaging implementations
• Less focused on orchestrating full pipelines
• Demonstrates examples of event-driven architecture and infrastructure-as-code
  • Applicable to other kinds of data processing not specific to FHIR

Architecture

Loading

graph LR
FHIRSource((FHIR \nSource Servers))
DataSource[(Data Sources)]

%%% EventSource[Incoming Events]
MessageSource([Incoming \nMessages/Events])

Function{{Transformation\nFunctions}}

%%% EventSink[Outgoing Events]
MessageSink([Outgoing \nMessages/Events])
DataSink[(Data Sinks)]
FHIRSink((FHIR \nSink Servers))

%%% 
FHIRSource --> |produce/publish/trigger| MessageSource
%%% FHIRSource --> |publish| EventSource
DataSource --> |produce/publish| MessageSource
%%% DataSource --> |publish| EventSource

MessageSource -.-> |trigger| Function
Function -.-> |consume/subscribe| MessageSource
%%% EventSource --> Function

%%% Function --> |publish| EventSink
Function --> |write/publish| MessageSink
Function --> |write| DataSink
Function --> |write| FHIRSink

ETL Implementations

FHIR as Sink

• HL7v2 to FHIR with AWS Lambda & SQS (Java)

FHIR as Source

• FHIR to Synthea with Azure Function & Service Bus Queue (C#)

Declarative Mappings

Data transformations can be more concise, readable, and easier to reason about when expressed declaratively. Here are some options worth exploring for encoding these transformations in data.

• Microsoft
  • Liquid Templates
• HL7
  • ConceptMaps
• Google
  • Whistle Data Transformation Language - transpiled to Protocol Buffers
• RDF ?
  • RDF data can easily be mapped using predicates like owl:equivalentClass and owl:equivalentProperty