test: use ContextConfiguration instead of profiles for DB DHIS2-17821

[teleivo]

Our production code has logic in it that changes its behavior on whether its running inside a test or with a given test profile.

dhis2-core/dhis-2/dhis-web-api/src/main/java/org/hisp/dhis/webapi/controller/security/ImpersonateUserController.java

Line 88 in 9df1984

@Conditional(value = UserImpersonationEnabledCondition.class)

This is risky as we might not run some crucial logic if we don't test this well. Spring has other ways for us to override part of our application during testing that does not compromise production code.

We configure the Spring (web) ApplicationContext using @ContextConfiguration in our tests. These can be inherited, merged and overridden. The inheritance can also be opted-out of. Using this we can override specific beans like the dhisConfigurationProvider so that we test against H2 or Postgres with Minio.

Our production code does a @ComponentScan so Spring creates all components (config, service, repo).

So far the IntegrationTestBaseConfig did a @ComponentScan like we do in production code of org.hisp.dhis code. This includes all our test @Configuration classes during integration testing.

I found an issue when I made following improvement:

I moved H2 related beans into the H2Configuration class. H2 does not support JSON, so we need to register fake functions with the data source. We also use H2 auto DDL feature instead of running flyway.

When I ran a PostgresIntegrationTest it was not running the flyway migrations. The reason was that it used the NoOpFlyway from the H2Configuration.

Summarized version of Spring logs during a PostgresIntegrationTest

Overriding bean definition for bean 'dhisConfigurationProvider' with a different definition:
replacing `org.hisp.dhis.external.conf.DefaultDhisConfigurationProvider`
with `org/hisp/dhis/test/config/H2DhisConfiguration.class`

Overriding bean definition for bean 'flyway' with a different definition:
replacing flyway `org/hisp/dhis/db/migration/config/FlywayConfig.class`
with `org/hisp/dhis/test/config/H2DhisConfiguration.class`

Overriding bean definition for bean 'namedParameterJdbcTemplate' with a different definition:
replacing `org/hisp/dhis/config/DataSourceConfig.class`
with `org/hisp/dhis/test/config/H2DhisConfiguration.class`

Overriding bean definition for bean 'dataSource' with a different definition:
replacing `org/hisp/dhis/config/DataSourceConfig.class`
with `org/hisp/dhis/test/config/H2DhisConfiguration.class`

Overriding bean definition for bean 'dhisConfigurationProvider' with a different definition:
replacing `org/hisp/dhis/test/config/H2DhisConfiguration.class`
with `org/hisp/dhis/test/config/MinIOConfiguration.class`

Overriding bean definition for bean 'dhisConfigurationProvider' with a different definition:
replacing `org/hisp/dhis/test/config/MinIOConfiguration.class`
with `org.hisp.dhis.test.config.PostgresDhisConfiguration`

So you can see that individual beans from the prod config are overridden by our test configs. You can also see that even though its a Postgres test the H2Configuration does override the flyway bean. That explains why no migrations are run even though we do setup a Postgres DB in the end.

To be really explicit and in control we can filter out all test related @Configurations with excludeFilters = @ComponentScan.Filter(type = FilterType.REGEX, pattern = "org\\.hisp\\.dhis\\.test\\..*"). Then include only the ones we need using a @ContextConfiguration.

After this filter is applied the bean creation is much simpler

Overriding bean definition for bean 'dhisConfigurationProvider' with a different definition:
replacing `org.hisp.dhis.external.conf.DefaultDhisConfigurationProvider`
with `org.hisp.dhis.test.config.PostgresDhisConfiguration`

Spring Boot has a https://reflectoring.io/spring-boot-testconfiguration/
which is a @Configuration that is not picked up by @ComponentScan. It has to be explicitly be imported in a test. The approach we take here is similar.

Related articles

https://reflectoring.io/dont-use-spring-profile-annotation/
https://reflectoring.io/spring-boot-testconfiguration/

Debugging Bean Creation

If you ever need to debug what beans get created enable this one

        <Logger name="org.springframework.beans.factory.support.DefaultListableBeanFactory" level="debug" additivity="false">
            <AppenderRef ref="console"/>
        </Logger>

in dhis-2/dhis-test-integration/src/test/resources/log4j2-test.xml for integration tests
in dhis-2/dhis-support/dhis-support-test/src/main/resources/log4j2-test.xml for other tests

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[enricocolasante]

Is this profile still needed?

[teleivo]

Yes, we still have production code that is aware of profiles like

dhis2-core/dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/database/HibernateDatabaseInfoProvider.java

Line 164 in 6971c94

if (SystemUtils.isH2(environment.getActiveProfiles())) {

. We might be able to solve this particular case. I have not tried.

I tried to get rid of prod code knowing about test profiles in general in #18282 but am blocked. The issue is that some tests need to set specific dhis.conf properties. We do this via @ContextConfiguration overriding beans. We use @Conditional like

dhis2-core/dhis-2/dhis-support/dhis-support-cache-invalidation/src/main/java/org/hisp/dhis/cacheinvalidation/redis/CacheInvalidationListener.java

Line 64 in db09c45

@Conditional(value = CacheInvalidationEnabledCondition.class)

to create certain components only if a feature is active. So there is a chicken and egg situation 😅 Spring Boot has automatic support for what we would like to do with application.properties and ConditionalOnProperty. It would be amazing if we can come up with something similar in https://dhis2.atlassian.net/browse/DHIS2-17797. @jbee let's talk about this when you are ready to look into https://dhis2.atlassian.net/browse/DHIS2-17797.