Enable FHIR subscriptions to the HAPI FHIR starter project during deployment

[dubdabasoduba]

Context

• We have been testing out this HAPI FHIR <> DHIS2 integration tool. For it to function correctly it needs FHIR subscriptions enabled.

Work to do

• Update the configurations to enable FHIR subscriptions during deployment.
• Test out the creation of the subscription to validate that the subscriptions functionality works correctly.
  • This link will be quite helpful https://github.com/hapifhir/hapi-fhir-jpaserver-starter#enabling-subscriptions

[rehammuzzamil]

On quick RnD, it seems like we only need to add configurations related to the enabling of subscription in the application.yml file.
To test out that subscriptions work fine, we need to test against a particular use case. @dubdabasoduba do you have any particular use-case in mind?

• I can test creating a subscription on my local using http://localhost:8080/hapi-fhir-jpaserver/fhir/Subscription

[dubdabasoduba]

On quick RnD, it seems like we only need to add configurations related to the enabling of subscription in the application.yml file. To test out that subscriptions work fine, we need to test against a particular use case. @dubdabasoduba do you have any particular use-case in mind?

• I can test creating a subscription on my local using http://localhost:8080/hapi-fhir-jpaserver/fhir/Subscription

Please test this locally then write up some documentation on the process you take.

[pld]

@rehammuzzamil you can collaborate with @lincmba on the production use case after it is working locally

CC @ageryck

[lincmba]

@rehammuzzamil , I believe the rest endpoints for fhir work fine. What we are having trouble with is making the dhis2 fhir adapter work.

Documentation can be found here on the readme https://github.com/ITINordic/dhis2-fhir-adapter.
Api guide can be found here http://localhost:8081/docs/api-guide.html once the adapter is up and running.

Currently the configs are set to point to the demo dhis server https://play.dhis2.org/2.37.3/ and the payload to create the subscription contains a fhir config set to any the fhir serves, I set mine to http://hapi.fhir.org/.

We have two problems:

1. We cant create subscriptions on either our stage of prod fhir servers using the adapter, both fail with the following error:

2022-02-14 14:39:36.792 ERROR 60231 --- [http-nio-8081-exec-9] o.a.c.c.C.[.[.[/].[dispatcherServlet]    :  - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.dhis2.fhir.adapter.rest.RestBadRequestException: The subscription could not be created on https://turn-fhir.smartregister.org/fhir: Failed to retrieve the server metadata statement during client initialization. URL used was https://turn-fhir.smartregister.org/fhir/metadata] with root cause

ca.uhn.fhir.parser.DataFormatException: Invalid attribute value "special": Unknown SearchParamType code 'special'
	at ca.uhn.fhir.parser.StrictErrorHandler.invalidValue(StrictErrorHandler.java:49)

2. Subscription can be created on the public site from the adapter. However, when a resource we have a subscription on (Patient) is created or updated on fhir, we don't get a notification on the adapter, neither is it created on the dhis server we have set.

Seeing this is a POC, we are trying to address the second problem before we later make our fhir sites match the public one

[rehammuzzamil]

As per our pairing sessions with @lincmba , here are my thoughts:

• We should use R4 as the fhirVersion, as our current JPA Server deployment is based on the R4 version and not on DSTU3.
• Simba has also tried using R4 with our stage FHIR server, but it seems like the Adapter does not recognize this version. It throws an exception of "ca.uhn.fhir.parser.DataFormatException: Invalid attribute value "4.0.1": Unknown FHIRVersion code '4.0.1'
  -- I'm not sure what's causing that, but it could be due to an FHIR version mismatch; for example, our current deployment is running on 5.7.0, and the adapter may be pointing to an older version. We may need to look at the adapter repo code for this.
• I managed to run the adapter locally by downgrading the JDK version to 1.8
• Need to complete the initial setup of adapter with the public dhis.

I will resume R&D on this in the coming week.
cc : @ageryck @dubdabasoduba @lincmba

[rehammuzzamil]

Updates:

• Subscriptions do not work fine when a public HAPI instance is used.
• To further debug, we planned to set up dhis-fhir-adapter locally and use HAPI FHIR JPA Locally.
• On debugging, we have identified it fails when it reads the Conformance statement it gets fhirVersion as 4.0.1 but the enum does not have the corresponding mapping. Thus, it fails while parsing.
• Enumerations.class inside package org.hl7.fhir.r4.model; does not have mapping for 4.0.1 version code.
• Currently it uses the 3.7.0version.
• Seems like the fhir-dhis-adapter is pointing to quite an older version of the fhir-structures-r4 library.

To make it work, a quick solution looks like upgrading the library fhir-structures-r4 version to the latest one.

• It can be found here: https://mvnrepository.com/artifact/ca.uhn.hapi.fhir/hapi-fhir-structures-r4

cc : @dubdabasoduba @ageryck @f-odhiambo

[dubdabasoduba]

Updates:

• Subscriptions do not work fine when a public HAPI instance is used.
• To further debug, we planned to set up dhis-fhir-adapter locally and use HAPI FHIR JPA Locally.
• On debugging, we have identified it fails when it reads the Conformance statement it gets fhirVersion as 4.0.1 but the enum does not have the corresponding mapping. Thus, it fails while parsing.
• Enumerations.class inside package org.hl7.fhir.r4.model; does not have mapping for 4.0.1 version code.
• Currently it uses the 3.7.0version.
• Seems like the fhir-dhis-adapter is pointing to quite an older version of the fhir-structures-r4 library.

To make it work, a quick solution looks like upgrading the library fhir-structures-r4 version to the latest one.

• It can be found here: https://mvnrepository.com/artifact/ca.uhn.hapi.fhir/hapi-fhir-structures-r4

cc : @dubdabasoduba @ageryck @f-odhiambo

@rehammuzzamil thanks for this feedback. Please go ahead and test the lib upgrade. Let us know incase you need any help.

[rehammuzzamil]

During the in-depth Rnd, I have figured out that:

• Only updating the 'fhir-structures-r4' library version is not the right option; it is preferable to upgrade all of HAPI FHIR's modules.
• After updating the HAPI FHIR version to 5.7.0 which is the latest stable build, several compilation issues occured.
• All compilation issues were fixed, but right now we are facing the Jackson library version mismatch issue.
• Tried to skip building an app without tests for a while but it throws following exception via run-time as well:
  java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.json.JsonMapper
• I have generated the Maven Dependency Tree and will try to resolve the above-mentioned issue.

Maven dependency tree.pdf

cc: @dubdabasoduba @ageryck

[rehammuzzamil]

Updates:
On the HAPI JPA Server side:

• I was struggling to post a Subscription via the HAPI JPA server directly on my local, I was facing SQLGrammarExceptions on almost each and every API.
• It was resolved by setting the HibernateDialect to the correct one i.e.
  hibernate.dialect: ca.uhn.fhir.jpa.model.dialect.HapiFhirPostgres94Dialect
  It was added in a recent HAPI Release and by default it was pointing to the H2 one.

On the Adapter side:

• Issue of Jackson mismatch dependencies has been fixed by adding all Jackson sub-module JARs in to the adapter classpath directly (via Parent Pom.xml)
• Few of the module's failing unit tests are ignored by commenting out for testing purpose.
• The request http://localhost:8080/api/fhirClients triggers on the DHIS FHIR Adapter and passes the request to the local HAPI instance as per the configuration.
• It fails with the following exception:

[Request processing failed; nested exception is org.dhis2.fhir.adapter.rest.RestBadRequestException: The subscription could not be created on http://localhost:8090/fhir: HTTP 302 ] with root case

ca.uhn.fhir.rest.server.exceptions.UnclassifiedServerFailureException: HTTP 302  

• Since, our JPA Server APIs are protected via Keycloak as the Authentication Server, therefore it redirects the call to the default keycloak login page i.e./sso/loginbut the parser on the DHIS FHIR Adapter fails to recognize it.
• Attached are the request and response debugging snapshots.
  

• For a while, I have removed the dependency of Keycloak, the intention was to test without the Authentication piece. And with this approach, subscription was created successfully on the HAPI local instance, though I have observed following error in the logs:
  `
• ERROR c.u.f.j.s.m.r.SubscriptionLoader [SubscriptionLoader.java:230] Subscription 52 could not be activated. This will not prevent startup, but it could lead to undesirable outcomes! null`

But this seems to be a vague comment as subscription was already enabled at the time of creation. This requires a bit more debugging.

• There is a sync method inside **SubscriptionLoader** class that ensures the activation of subscriptions.
• On testing, I did not get a notification on DHIS Adapter after posting the Patient into the HAPI FHIR server.

I will continue debugging and share my updates here.

cc : @dubdabasoduba @ageryck

[rehammuzzamil]

@rehammuzzamil , I believe the rest endpoints for fhir work fine. What we are having trouble with is making the dhis2 fhir adapter work.

Documentation can be found here on the readme https://github.com/ITINordic/dhis2-fhir-adapter. Api guide can be found here http://localhost:8081/docs/api-guide.html once the adapter is up and running.

Currently the configs are set to point to the demo dhis server https://play.dhis2.org/2.37.3/ and the payload to create the subscription contains a fhir config set to any the fhir serves, I set mine to http://hapi.fhir.org/.

We have two problems:

1. We cant create subscriptions on either our stage of prod fhir servers using the adapter, both fail with the following error:

2022-02-14 14:39:36.792 ERROR 60231 --- [http-nio-8081-exec-9] o.a.c.c.C.[.[.[/].[dispatcherServlet]    :  - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.dhis2.fhir.adapter.rest.RestBadRequestException: The subscription could not be created on https://turn-fhir.smartregister.org/fhir: Failed to retrieve the server metadata statement during client initialization. URL used was https://turn-fhir.smartregister.org/fhir/metadata] with root cause

ca.uhn.fhir.parser.DataFormatException: Invalid attribute value "special": Unknown SearchParamType code 'special'
	at ca.uhn.fhir.parser.StrictErrorHandler.invalidValue(StrictErrorHandler.java:49)

2. Subscription can be created on the public site from the adapter. However, when a resource we have a subscription on (Patient) is created or updated on fhir, we don't get a notification on the adapter, neither is it created on the dhis server we have set.

Seeing this is a POC, we are trying to address the second problem before we later make our fhir sites match the public one

Problem # 1 has been addressed and now we can create a subscription on HAPI FHIR JPA instance. The work is on local machine for now.
cc : @lincmba @dubdabasoduba @ageryck

[dubdabasoduba]

Updates: On the HAPI JPA Server side:

• I was struggling to post a Subscription via the HAPI JPA server directly on my local, I was facing SQLGrammarExceptions on almost each and every API.
• It was resolved by setting the HibernateDialect to the correct one i.e.
  hibernate.dialect: ca.uhn.fhir.jpa.model.dialect.HapiFhirPostgres94Dialect
  It was added in a recent HAPI Release and by default it was pointing to the H2 one.

On the Adapter side:

• Issue of Jackson mismatch dependencies has been fixed by adding all Jackson sub-module JARs in to the adapter classpath directly (via Parent Pom.xml)
• Few of the module's failing unit tests are ignored by commenting out for testing purpose.
• The request http://localhost:8080/api/fhirClients triggers on the DHIS FHIR Adapter and passes the request to the local HAPI instance as per the configuration.
• It fails with the following exception:

[Request processing failed; nested exception is org.dhis2.fhir.adapter.rest.RestBadRequestException: The subscription could not be created on http://localhost:8090/fhir: HTTP 302 ] with root case

ca.uhn.fhir.rest.server.exceptions.UnclassifiedServerFailureException: HTTP 302  

• Since, our JPA Server APIs are protected via Keycloak as the Authentication Server, therefore it redirects the call to the default keycloak login page i.e./sso/loginbut the parser on the DHIS FHIR Adapter fails to recognize it.
• Attached are the request and response debugging snapshots.
  

• For a while, I have removed the dependency of Keycloak, the intention was to test without the Authentication piece. And with this approach, subscription was created successfully on the HAPI local instance, though I have observed following error in the logs:
  `
• ERROR c.u.f.j.s.m.r.SubscriptionLoader [SubscriptionLoader.java:230] Subscription 52 could not be activated. This will not prevent startup, but it could lead to undesirable outcomes! null`

But this seems to be a vague comment as subscription was already enabled at the time of creation. This requires a bit more debugging.

• There is a sync method inside **SubscriptionLoader** class that ensures the activation of subscriptions.
• On testing, I did not get a notification on DHIS Adapter after posting the Patient into the HAPI FHIR server.

I will continue debugging and share my updates here.

cc : @dubdabasoduba @ageryck

@rehammuzzamil Please try using this endpoint. It points to the same database but has auth disabled. https://turn-fhir.smartregister.org/fhir/