Skip to content

Refactor Evaluator Images, Pipeline "Run" type image specification#826

Draft
JersyJ wants to merge 2 commits intomrlvsb:masterfrom
JersyJ:refactor-evaluator-images
Draft

Refactor Evaluator Images, Pipeline "Run" type image specification#826
JersyJ wants to merge 2 commits intomrlvsb:masterfrom
JersyJ:refactor-evaluator-images

Conversation

@JersyJ
Copy link
Contributor

@JersyJ JersyJ commented Feb 9, 2026

  • Reworked Evaluator Docker images (base/run/java/dotnet/clang-tidy/cargo/pythonrun)
  • Kelvin: Updated pipeline configuration/validation to allow selecting a Docker image for the "tests" type (pipe).

Copilot AI review requested due to automatic review settings February 9, 2026 15:16
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors several evaluator Docker images (base/run/java/dotnet/clang-tidy/cargo/pythonrun) and extends pipeline configuration/validation to allow selecting a Docker image for the tests pipeline action, with accompanying documentation updates.

Changes:

  • Add an image option to the tests pipeline action (frontend validation + evaluator runtime support).
  • Refactor evaluator Docker images (package installs, cleanup, pinned tooling versions, updated bleach).
  • Update teacher documentation for pipeline/tests, including clearer code block formatting.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
frontend/src/PipelineValidation.js Allows pipeline: - type: tests to accept an image field in config validation/help text.
evaluator/testsets.py Minor warning formatting change (still within pipeline parsing error path).
evaluator/pipelines.py Adds configurable image for TestsPipe and uses it to start the container.
evaluator/images/run/Dockerfile Refactors apt install pattern and removes Java from the run image.
evaluator/images/pythonrun/Dockerfile Re-bases pythonrun to kelvin/base and installs pytest/flake8 (but currently loses the run entrypoint behavior).
evaluator/images/java/entry.py Removes runtime env-var injection logic (relies on Dockerfile env now).
evaluator/images/java/Dockerfile Refactors installs and hardcodes Java/Maven env vars.
evaluator/images/dotnet/Dockerfile Refactors installs; switches base reference to kelvin/base:latest.
evaluator/images/clang-tidy/Dockerfile Switches from Alpine to kelvin/gcc and installs clang-tidy via apt.
evaluator/images/cargo/entry.py Minor string formatting change in warning output.
evaluator/images/cargo/Dockerfile Refactors apt install; bumps bleach version.
evaluator/images/base/Dockerfile Consolidates packages, sets locale env, installs bleach 6.3.0 in base.
docs/docs/03-teachers-guide/02-task-configuration/05-pipeline.mdx Documents the new tests.image option.
docs/docs/03-teachers-guide/02-task-configuration/03-tests.mdx Documents tests.image and improves code block language annotations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@JersyJ
Copy link
Contributor Author

JersyJ commented Feb 17, 2026

Based on newest feedback from @Kobzol , I must postpone this, so I am putting this into Draft

@JersyJ JersyJ marked this pull request as draft February 17, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants