Skip to content

Conversation

@eap
Copy link
Collaborator

@eap eap commented Aug 12, 2025

Performance improvements for jedi-ci

  1. Part of the test-launcher container is pre-built and pushed to the github container repo. This pre-building improves the speed of the action init. We do not pre-build the full image because it would prevent testing of unsubmitted changes.

  2. Use a thread pool to cancel batch jobs and GitHub check runs. The functions used to to cancel prior tests runs are network bound and somewhat slow. Rather than running them in serial, this change runs them in parallel with a thread pool executor (Python's GIL doesn't matter here because the latency comes from non-blocking network calls).

Why was this needed?

The JEDI-CI launcher is somewhat interactive and a failure here causes a great deal of downstream developer pain. I want this action to execute as fast as possible in order to improve the chances that a failure is noticed before the user "walks away". Additionally, this will be JCSDA's most widely used GitHub-hosted action and this will prevent it from consuming unnecessary compute time (which is "expensive" due to our limited GitHub-earmarked funding).

Checklist

  • I have performed a self-review of my own code
  • N/A I have made corresponding changes to the documentation
  • I have run the unit tests before creating the PR

@eap eap marked this pull request as ready for review August 12, 2025 20:40
@eap eap changed the title draft JEDI-ci performance fixes Aug 12, 2025
Copy link
Collaborator

@ashley314 ashley314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

@eap eap changed the title JEDI-ci performance fixes JEDI-CI performance fixes Aug 13, 2025
@eap eap merged commit daa0412 into develop Aug 13, 2025
1 check passed
@eap eap deleted the feature/oops-fixes branch August 13, 2025 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants