Skip to content

feat: add task chunking#294

Open
npmacl wants to merge 2 commits intoaws-deadline:mainlinefrom
npmacl:task-chunking
Open

feat: add task chunking#294
npmacl wants to merge 2 commits intoaws-deadline:mainlinefrom
npmacl:task-chunking

Conversation

@npmacl
Copy link
Copy Markdown
Contributor

@npmacl npmacl commented Mar 26, 2026

Note:

This is a continuation of an earlier PR from @rickamas: #293.
The changes are the same, except for adding some updates to the Job Bundle Output Tests.

What is the change?

Add opt-in task chunking using the OpenJD TASK_CHUNKING extension. When enabled, frames are grouped into contiguous chunks instead of being dispatched individually, reducing per-task overhead. A future update could add non-contiguous chunking but for now this seems useful as is.

Changes:

New chunked job template with CHUNK[INT] Frame parameter, ChunkSize, and TargetRuntimeSeconds job parameters
RenderSettings: use_chunking, chunk_size, target_runtime_seconds fields (sticky, default off)
Submitter selects chunked template when use_chunking is enabled
UI: Enable task chunking checkbox with chunk size and target chunk duration controls

No adaptor changes needed - the existing NukeHandler.start_render() already handles the contiguous chunk frame range format (e.g. 1-10).

How was this change tested?

  • Local testing of some 500 frame scenes in Nuke 15.
  • Tested submitting jobs with ChunkSize of 1 and TargetRuntimeSeconds set to 0. Tested with ChunkSize of 1 and TargetRuntimeSeconds set to a non-zero value. tested ChunkSize >1 and TargetRuntimeSeconds set to 0. Tested TargetRuntimeSeconds > 0 and ChunkSize >1. All worked as expected.

Please run the integration tests and paste the results below

=== 155 passed, 8 skipped in 13.94s ===

If installer/ was modified or a file was added/removed from src/, then update the installer tests and post the test results below

No modified

Did you run the "Job Bundle Output Tests"? If not, why not? If so, paste the test results here.

Ran the tests, all 6 passed.

Was this change documented?

Yes

Is this a breaking change?

No

rickrams and others added 2 commits March 3, 2026 22:08
Add task chunking using the OpenJD TASK_CHUNKING extension. The existing
job template now uses CHUNK[INT] for the Frame parameter with contiguous
chunks. Chunk size defaults to 1 (one frame per task, same as before).
Users increase chunk size to group frames and reduce per-task overhead.

Changes:
- Updated default job template with TASK_CHUNKING extension, CHUNK[INT]
  Frame parameter, ChunkSize (default 1), and TargetChunkDuration params
- RenderSettings: chunk_size and target_chunk_duration fields (sticky)
- UI: Chunk size and target chunk duration spinners in job settings
- Docs: Updated user guide, README, and CHANGELOG

No adaptor changes needed - the existing NukeHandler.start_render()
already handles the contiguous chunk frame range format (e.g. 1-10).

Note: Requires a worker agent that supports the TASK_CHUNKING extension.
Service-managed fleets always use a compatible version.

Signed-off-by: Rick Ramsay <49293857+rickrams@users.noreply.github.com>
Signed-off-by: Nathan MacLeod <142927985+npmacl@users.noreply.github.com>
@sonarqubecloud
Copy link
Copy Markdown

@github-actions github-actions bot added the waiting-on-maintainers Waiting on the maintainers to review. label Mar 26, 2026
@npmacl npmacl marked this pull request as ready for review March 26, 2026 20:02
@npmacl npmacl requested a review from a team as a code owner March 26, 2026 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-on-maintainers Waiting on the maintainers to review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants