Refactor Evaluator Images, Pipeline "Run" type image specification#826
Draft
JersyJ wants to merge 2 commits intomrlvsb:masterfrom
Draft
Refactor Evaluator Images, Pipeline "Run" type image specification#826JersyJ wants to merge 2 commits intomrlvsb:masterfrom
JersyJ wants to merge 2 commits intomrlvsb:masterfrom
Conversation
Contributor
JersyJ
commented
Feb 9, 2026
- Reworked Evaluator Docker images (base/run/java/dotnet/clang-tidy/cargo/pythonrun)
- Kelvin: Updated pipeline configuration/validation to allow selecting a Docker image for the "tests" type (pipe).
…e if required for different language
There was a problem hiding this comment.
Pull request overview
This PR refactors several evaluator Docker images (base/run/java/dotnet/clang-tidy/cargo/pythonrun) and extends pipeline configuration/validation to allow selecting a Docker image for the tests pipeline action, with accompanying documentation updates.
Changes:
- Add an
imageoption to thetestspipeline action (frontend validation + evaluator runtime support). - Refactor evaluator Docker images (package installs, cleanup, pinned tooling versions, updated bleach).
- Update teacher documentation for pipeline/tests, including clearer code block formatting.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| frontend/src/PipelineValidation.js | Allows pipeline: - type: tests to accept an image field in config validation/help text. |
| evaluator/testsets.py | Minor warning formatting change (still within pipeline parsing error path). |
| evaluator/pipelines.py | Adds configurable image for TestsPipe and uses it to start the container. |
| evaluator/images/run/Dockerfile | Refactors apt install pattern and removes Java from the run image. |
| evaluator/images/pythonrun/Dockerfile | Re-bases pythonrun to kelvin/base and installs pytest/flake8 (but currently loses the run entrypoint behavior). |
| evaluator/images/java/entry.py | Removes runtime env-var injection logic (relies on Dockerfile env now). |
| evaluator/images/java/Dockerfile | Refactors installs and hardcodes Java/Maven env vars. |
| evaluator/images/dotnet/Dockerfile | Refactors installs; switches base reference to kelvin/base:latest. |
| evaluator/images/clang-tidy/Dockerfile | Switches from Alpine to kelvin/gcc and installs clang-tidy via apt. |
| evaluator/images/cargo/entry.py | Minor string formatting change in warning output. |
| evaluator/images/cargo/Dockerfile | Refactors apt install; bumps bleach version. |
| evaluator/images/base/Dockerfile | Consolidates packages, sets locale env, installs bleach 6.3.0 in base. |
| docs/docs/03-teachers-guide/02-task-configuration/05-pipeline.mdx | Documents the new tests.image option. |
| docs/docs/03-teachers-guide/02-task-configuration/03-tests.mdx | Documents tests.image and improves code block language annotations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
Author
|
Based on newest feedback from @Kobzol , I must postpone this, so I am putting this into Draft |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.