Skip to content

Document ES7 indexing type and add condition regression test#929

Open
bulgarian-beast wants to merge 5 commits intoconductor-oss:mainfrom
bulgarian-beast:fix/es7-docker-config-indexing-type
Open

Document ES7 indexing type and add condition regression test#929
bulgarian-beast wants to merge 5 commits intoconductor-oss:mainfrom
bulgarian-beast:fix/es7-docker-config-indexing-type

Conversation

@bulgarian-beast
Copy link
Copy Markdown
Contributor

@bulgarian-beast bulgarian-beast commented Mar 26, 2026

The ES7 Docker examples configured Elasticsearch but did not explicitly select the Elasticsearch indexing backend, which prevent IndexDAO from being created and break startup.

Summary

Fix ES7 Docker example configs and docs to explicitly select Elasticsearch indexing by setting:

conductor.indexing.type=elasticsearch

Without this property, the ES7 IndexDAO condition does not match, which can cause server startup to fail because no IndexDAO bean is created.

Fixes #928.


Pull Request type

  • Bugfix
  • Feature
  • Refactoring (no functional changes, no API changes)
  • Build related changes
  • WHOSUSING.md
  • Other

What was wrong

The ES7-backed Docker configs already enabled indexing and set the Elasticsearch version/URL, but a property was not set:

conductor.indexing.type=elasticsearch

That property is required for the ES7 IndexDAO configuration to activate.

Because of that, all Docker setups fail during startup with a missing IndexDAO bean, for example:

Parameter 2 of constructor in com.netflix.conductor.core.dal.ExecutionDAOFacade required a bean of type 'com.netflix.conductor.dao.IndexDAO' that could not be found.

I reproduced this with:

docker compose -f docker/docker-compose-mysql.yaml up --build

Although I hit the issue through the MySQL compose path, the same missing property existed in the ES7-backed Docker configs more broadly, so this PR fixes the configuration consistently across those examples.


What this PR changes

Config fixes

Adds:

conductor.indexing.type=elasticsearch

to:

  • docker/server/config/config-mysql.properties
  • docker/server/config/config-redis.properties
  • docker/server/config/config-postgres-es7.properties

Documentation fixes

Updates ES7 examples in:

  • docker/server/config/config.properties
  • docs/devguide/running/deploy.md

so the documented configuration matches the actual runtime requirement.

Regression coverage

Adds a regression test for the ES7 conditional activation:

  • es7-persistence/src/test/java/com/netflix/conductor/es7/config/ElasticSearchConditionsTest.java

Why this fix is needed

The ES7 persistence module does not activate its IndexDAO based only on:

  • conductor.indexing.enabled=true
  • conductor.elasticsearch.version=7

It also requires the indexing backend to be explicitly selected with:

conductor.indexing.type=elasticsearch

Without that property, Spring does not create the ES7 IndexDAO, and server startup can fail before initialization completes.


Verification

Ran:

./gradlew spotlessApply
./gradlew :conductor-es7-persistence:test --tests com.netflix.conductor.es7.config.ElasticSearchConditionsTest
docker compose -f docker/docker-compose-mysql.yaml up --build
docker compose -f docker/docker-compose-mysql.yaml ps

Verified:

  • spotlessApply completed successfully
  • the targeted ES7 condition test passed
  • the MySQL + Redis + Elasticsearch 7 Docker setup started successfully
  • conductor-server reached healthy state
NAME                               IMAGE                                                   COMMAND                                                        SERVICE                   CREATED             STATUS                        PORTS
conductor-server                   conductor:server                                        "/bin/sh /app/startup.sh"                                      conductor-server          45 minutes ago      Up About a minute (healthy)   0.0.0.0:8080->8080/tcp, 0.0.0.0:8127->5000/tcp
docker-conductor-elasticsearch-1   docker.elastic.co/elasticsearch/elasticsearch:7.17.11   "/bin/tini -- /usr/local/bin/docker-entrypoint.sh eswrapper"   conductor-elasticsearch   About an hour ago   Up About a minute (healthy)   9300/tcp, 0.0.0.0:9201->9200/tcp
docker-conductor-mysql-1           mysql:latest                                            "docker-entrypoint.sh mysqld"                                  conductor-mysql           About an hour ago   Up About a minute (healthy)   0.0.0.0:3306->3306/tcp, 33060/tcp
docker-conductor-redis-1           redis:6.2.3-alpine                                      "docker-entrypoint.sh redis-server"                            conductor-redis           About an hour ago   Up About a minute (healthy)   0.0.0.0:7379->6379/tcp

Scope

This PR is intentionally narrow:

  • no behavior changes outside ES7 configuration selection
  • no API changes
  • no dependency or build changes
  • only config/docs alignment plus regression coverage

Docker ES7-backed configs enabled indexing and set Elasticsearch version/url, but did not set conductor.indexing.type=elasticsearch.

Because the ES7 IndexDAO condition requires that property, the MySQL docker-compose example could fail at startup with a missing IndexDAO bean.

This change:
- adds conductor.indexing.type=elasticsearch to ES7 docker configs
- updates the docs to show the required property
- adds a regression test for the ES7 condition
@bulgarian-beast bulgarian-beast force-pushed the fix/es7-docker-config-indexing-type branch from da624f6 to 2896bc3 Compare April 8, 2026 07:18
@bulgarian-beast
Copy link
Copy Markdown
Contributor Author

Hi @v1r3n and @mp-orkes,

Would you mind taking a look at this PR when you have a moment?

This ES7 Docker config issue has been pretty disruptive on my side because it breaks the MySQL/Redis/ES7 local stack and I end up hitting it again whenever I branch from main before the fix is merged.

I kept the scope intentionally small:

  • adds the missing conductor.indexing.type=elasticsearch in the ES7 Docker configs
  • aligns the docs/examples
  • adds a regression test for the ES7 condition
  • verified with docker compose -f docker/docker-compose-mysql.yaml up --build and healthy startup

PR: #929
Issue: #928

Thanks a lot.

@nthmost-orkes nthmost-orkes self-assigned this Apr 14, 2026
@nthmost-orkes nthmost-orkes self-requested a review April 14, 2026 21:50
Copy link
Copy Markdown
Contributor

@nthmost-orkes nthmost-orkes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix — the doc change and the regression test are both clearly in scope, but the PR as it stands is confusing to review because the description and the diff don't match. Requesting changes purely to reconcile that before merge.

What's actually in the diff

Only two files:

  • docs/devguide/running/deploy.md — adds conductor.indexing.type=elasticsearch to the ES7 example.
  • es7-persistence/src/test/java/com/netflix/conductor/es7/config/ElasticSearchConditionsTest.java — new regression test.

What the description claims, but isn't in the diff

The PR body says it modifies four Docker config files:

  • docker/server/config/config-mysql.properties
  • docker/server/config/config-redis.properties
  • docker/server/config/config-postgres-es7.properties
  • docker/server/config/config.properties

On current main, the three non-comment configs above already contain conductor.indexing.type=elasticsearch — they were added via #900 (e2e test suite). So those changes aren't needed here, but the description still advertises them as part of this PR.

Could you either:

  1. Rebase onto main, update the title/body to reflect the real scope (docs fix + regression test), and confirm whether #928 is still open against the Docker configs or only against the documented example, or
  2. If you intended additional config changes beyond what's on main, push them — right now they're not in the branch.

Test quality — LGTM once scope is reconciled

ElasticSearchConditionsTest is well-designed:

  • Uses Spring Boot's ApplicationContextRunner, which is the idiomatic way to test @Conditional classes without spinning up a full context.
  • Five cases cover the meaningful branches of ElasticSearchV7Enabled: happy path, implicit version (relies on matchIfMissing=true), disabled indexing, missing conductor.indexing.type (the exact regression this PR is preventing), and wrong-type selector (elasticsearch8).
  • Follows the JUnit 4 convention of the sibling ElasticSearchPropertiesTest in the same package.
  • @Configuration(proxyBeanMethods = false) is the right call for a lightweight test config.

Minor nits (non-blocking)

  • shouldActivateWhenVersionIsImplicit locks in the current matchIfMissing=true behavior for conductor.elasticsearch.version. Worth a one-line comment so a future maintainer who tightens that condition doesn't have to reverse-engineer the expectation.
  • static class Marker {}Es7Marker would read slightly better alongside the "es7Marker" bean name. Trivial.

Doc change — LGTM

The deploy.md addition is correct: conductor.indexing.type has no matchIfMissing on its @ConditionalOnProperty in ElasticSearchConditions.java, so users who follow the documented snippet verbatim really do need this line.

@bulgarian-beast
Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review.

I rebased/fast-forwarded the branch and reconciled the scope with current main. The Docker config files already contain conductor.indexing.type=elasticsearch via #900, so this PR is now intentionally limited to:

  • docs/devguide/running/deploy.md
  • es7-persistence/src/test/java/com/netflix/conductor/es7/config/ElasticSearchConditionsTest.java

I also addressed the non-blocking test nits by adding a short comment for the implicit ES7 version behavior and renaming Marker to Es7Marker.

Verified again with:

./gradlew spotlessApply
./gradlew :conductor-es7-persistence:test --tests com.netflix.conductor.es7.config.ElasticSearchConditionsTest

@bulgarian-beast bulgarian-beast changed the title Fix missing Elasticsearch indexing type in Docker ES7 configs Fix Document ES7 indexing type and add condition regression test Apr 24, 2026
@bulgarian-beast bulgarian-beast changed the title Fix Document ES7 indexing type and add condition regression test Document ES7 indexing type and add condition regression test Apr 24, 2026
@bulgarian-beast
Copy link
Copy Markdown
Contributor Author

Hello @nthmost-orkes,

The failing redis-es8 e2e job appears unrelated to this PR. This branch only changes the ES7 deployment doc snippet and adds ES7 conditional activation coverage. The failed job is in the Redis + ES8 e2e matrix and the visible logs point to sync workflow timing/e2e behavior.

I re-ran the focused verification locally:

./gradlew spotlessApply
./gradlew :conductor-es7-persistence:test --tests com.netflix.conductor.es7.config.ElasticSearchConditionsTest

@nthmost-orkes
Copy link
Copy Markdown
Contributor

Hello @nthmost-orkes,

Thanks, revisiting this today!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ES7 Docker configs are missing conductor.indexing.type, causing startup failure with no IndexDAO

2 participants