[rhaiis] add custom dspark spec-decoding preset by aas008 · Pull Request #180 · openshift-psap/forge

aas008 · 2026-08-19T22:28:33Z

Add four H200 Zeus comparison presets: baseline, native MTP, DSpark fixed-k, and DSpark with dynamic SD. Adaptive verification is omitted because DeepSeek-V4 varlen full graphs are SM100-only.

Add four H200 Zeus comparison presets: baseline, native MTP, DSpark fixed-k, and DSpark with dynamic SD. Adaptive verification is omitted because DeepSeek-V4 varlen full graphs are SM100-only. Co-authored-by: Cursor <cursoragent@cursor.com>

openshift-ci · 2026-08-19T22:28:38Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign tosokin for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

coderabbitai · 2026-08-19T22:28:45Z

Important

🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e38dec0d-bc59-4642-9a43-f94b5d729d7b

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Run profile1-4 and enable the vLLM profiler so spec-decoding jobs land on the staging dashboard with traces uploaded to S3. Co-authored-by: Cursor <cursoragent@cursor.com>

aas008 · 2026-08-19T22:36:52Z

/test fournos rhaiis deepseek-v4-dspark profile1
/pipeline forge-full
/cluster zeus
/var rhaiis.engines.vllm.images.nvidia: vllm/vllm-openai:v0.27.0
/var tests.rhaiis.version: vLLM-0.27.0

psap-forge-bot · 2026-08-19T23:46:03Z

🔴 Execution of `rhaiis` deepseek-v4-dspark profile1 🔴

Execution Engine Configuration

forge:
  args:
  - deepseek-v4-dspark
  - profile1
  configOverrides:
    rhaiis.engines.vllm.images.nvidia: vllm/vllm-openai:v0.27.0
    tests.rhaiis.version: vLLM-0.27.0
  project: rhaiis

Artifact Links

Test Logs

✅ 00 Pre-Cleanup `1 second`

✅ 01 Prepare `3 seconds`

✅ 02 Preflight `1 second`

❌ 03 Test `1 hour, 5 minutes, 38 seconds`

✅ 04 Post-Cleanup `6 seconds`

🔄 05 Export-Artifacts

Post-processing Status ❌

❌ parse: failed * No records found - parsing completed successfully but no test data was extracted

psap-forge-bot · 2026-08-19T23:46:37Z

🔴 Submission of rhaiis deepseek-v4-dspark profile1 failed after 1 hour, 9 minutes, 18 seconds 🔴

Error: FournosJobFailureError: FOURNOS Job 'forge-rhaiis-20260819-223718' failed: Tasks Completed: 6 (Failed: 1, Cancelled 0), Skipped: 0

/test fournos rhaiis deepseek-v4-dspark profile1
/var rhaiis.engines.vllm.images.nvidia: vllm/vllm-openai:v0.27.0
/var tests.rhaiis.version: vLLM-0.27.0
/pipeline forge-full
/cluster zeus

Harshith-umesh · 2026-08-20T16:01:03Z

+  engine_args:
    tensor-parallel-size: 8

+deepseek-v4-pro-0813: &deepseek-v4-pro-0813


what do the &, * < symbols here mean?

YAML anchors and aliases. I learnt its a way to write the shared model block once and reuse it. I used them here to keep the file succinct; & names the block, * copies it, and <<: merges it so the MTP/DSpark variants only add speculative-config.

In current architecture of this models.yaml, ever config would need a new entry. This will bloat up very very fast.

This will bloat up very very fast.

if you look how the presets work, something like:

common_values: abc: true specific_values: extends: [common_values] efg: true

you can build a mechanism to avoid the config file size explosion. That may be more flexible than YAML anchors

or alternatively, if applicable, some default values that apply to all the models

default: engine_args: tensor-parallel-size: 8

Harshith-umesh · 2026-08-20T16:04:17Z

+    - profile2
+    - profile3
+    - profile4
+  rhaiis.profiler.enabled: true


do we need profiling for spec decoding?

This is a one time use preset @Harshith-umesh , should I add a tag to not merge it to main?

you can comment /hold in the main thread to prevent the merge and/or add Draft flag in the PR to convey that

yes if it's a one time preset then you can add a do not merge tag to the PR

I don't have permissions to add the label.

aas008 · 2026-08-20T17:33:29Z

I'm trying to understand why my failed. Is it because ready_timeout: 3600, and the download + loading the weigths could not be done in 1 hr? @kpouget @Harshith-umesh ?
It will be common for RHAIIS users to run new models from PRs, can we extend this timeout? Actually I see I can set it myself - rhaiis.deploy.ready_timeout

I think the bigger thing was rhaiis.deploy.image_pull_secrets: ["npalaska-image-pull"] was not added by default.

Harshith-umesh · 2026-08-20T17:38:51Z

nope the timeout is not this issue, please see this file on mlflow: mlflow-artifacts:/workspaces/forge-rhaiis/264/da8ad9a31b4e49e0ac57957ff6e02a8b/artifacts/03__test/001__test_deepseek-v4-pro-0813-dspark_profile1_profile2_profile3_profile4/002__capture_isvc_state/artifacts/inferenceservice.pods.logs

aas008 · 2026-08-20T17:42:21Z

Ah thank you I could download and see the memory issue.
Any reason I could not preview it on the web UI?

aas008 · 2026-08-20T19:57:14Z

/test fournos rhaiis deepseek-v4-dspark
/pipeline forge-full
/cluster zeus
/exclusive true
/var rhaiis.engines.vllm.images.nvidia: vllm/vllm-openai:v0.27.0
/var tests.rhaiis.version: vLLM-0.27.0

aas008 · 2026-08-20T20:04:05Z

/test cancel

aas008 · 2026-08-20T20:09:15Z

/test cancel

@Harshith-umesh Can I cancel a test?

aas008 · 2026-08-20T20:29:46Z

/test fournos rhaiis deepseek-v4-dspark
/pipeline forge-full
/cluster zeus
/exclusive true
/var rhaiis.engines.vllm.images.nvidia: vllm/vllm-openai:v0.27.0
/var tests.rhaiis.version: vLLM-0.27.0

psap-forge-bot · 2026-08-20T20:30:57Z

🛑 Execution of `rhaiis` deepseek-v4-dspark 🛑
🛑 JOB ABORTED - `spec.shutdown=Stop`

Execution Engine Configuration

forge:
  args:
  - deepseek-v4-dspark
  configOverrides:
    rhaiis.engines.vllm.images.nvidia: vllm/vllm-openai:v0.27.0
    tests.rhaiis.version: vLLM-0.27.0
  project: rhaiis

Artifact Links

Test Logs

✅ 00 Pre-Cleanup `1 second`

✅ 01 Prepare `3 seconds`

✅ 02 Preflight `1 second`

❓ 03 Test

✅ 04 Post-Cleanup `10 seconds`

🔄 05 Export-Artifacts

kpouget · 2026-08-20T20:41:28Z

/test cancel

fyi, in the github workflow (/test ...) you abort a test by pushing a commit to the PR. On purpose or because you forget you had a test running (talking from experience 😅 )

psap-forge-bot · 2026-08-20T21:39:36Z

🔴 Execution of `rhaiis` deepseek-v4-dspark 🔴

Execution Engine Configuration

forge:
  args:
  - deepseek-v4-dspark
  configOverrides:
    rhaiis.engines.vllm.images.nvidia: vllm/vllm-openai:v0.27.0
    tests.rhaiis.version: vLLM-0.27.0
  project: rhaiis

Artifact Links

Test Logs

✅ 00 Pre-Cleanup `1 second`

✅ 01 Prepare `3 seconds`

✅ 02 Preflight `1 second`

❌ 03 Test `1 hour, 5 minutes, 32 seconds`

✅ 04 Post-Cleanup `7 seconds`

🔄 05 Export-Artifacts

Post-processing Status ❌

❌ parse: failed * No records found - parsing completed successfully but no test data was extracted

psap-forge-bot · 2026-08-20T21:39:54Z

🔴 Submission of rhaiis deepseek-v4-dspark failed after 1 hour, 9 minutes, 36 seconds 🔴

Error: FournosJobFailureError: FOURNOS Job 'forge-rhaiis-20260820-203017' failed: Tasks Completed: 6 (Failed: 1, Cancelled 0), Skipped: 0

/test fournos rhaiis deepseek-v4-dspark
/var rhaiis.engines.vllm.images.nvidia: vllm/vllm-openai:v0.27.0
/var tests.rhaiis.version: vLLM-0.27.0
/pipeline forge-full
/cluster zeus
/exclusive true

openshift-ci · 2026-08-20T21:40:02Z

@aas008: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/fournos	`be9efff`	link	true	`/test fournos`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[rhaiis] presets: collect all profiles and vLLM profiler traces

be9efff

Run profile1-4 and enable the vLLM profiler so spec-decoding jobs land on the staging dashboard with traces uploaded to S3. Co-authored-by: Cursor <cursoragent@cursor.com>

Harshith-umesh reviewed Aug 20, 2026

View reviewed changes

Uh oh!

Conversation

aas008 commented Aug 19, 2026

Uh oh!

openshift-ci Bot commented Aug 19, 2026

Uh oh!

coderabbitai Bot commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aas008 commented Aug 19, 2026

Uh oh!

psap-forge-bot Bot commented Aug 19, 2026

🔴 Execution of rhaiis deepseek-v4-dspark profile1 🔴

✅ 00 Pre-Cleanup 1 second

✅ 01 Prepare 3 seconds

✅ 02 Preflight 1 second

❌ 03 Test 1 hour, 5 minutes, 38 seconds

✅ 04 Post-Cleanup 6 seconds

🔄 05 Export-Artifacts

Uh oh!

psap-forge-bot Bot commented Aug 19, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aas008 commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Harshith-umesh commented Aug 20, 2026

Uh oh!

aas008 commented Aug 20, 2026

Uh oh!

aas008 commented Aug 20, 2026

Uh oh!

aas008 commented Aug 20, 2026

Uh oh!

aas008 commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aas008 commented Aug 20, 2026

Uh oh!

psap-forge-bot Bot commented Aug 20, 2026

🛑 Execution of rhaiis deepseek-v4-dspark 🛑 🛑 JOB ABORTED - spec.shutdown=Stop

✅ 00 Pre-Cleanup 1 second

✅ 01 Prepare 3 seconds

✅ 02 Preflight 1 second

❓ 03 Test

✅ 04 Post-Cleanup 10 seconds

🔄 05 Export-Artifacts

Uh oh!

kpouget commented Aug 20, 2026

Uh oh!

psap-forge-bot Bot commented Aug 20, 2026

🔴 Execution of rhaiis deepseek-v4-dspark 🔴

✅ 00 Pre-Cleanup 1 second

✅ 01 Prepare 3 seconds

✅ 02 Preflight 1 second

❌ 03 Test 1 hour, 5 minutes, 32 seconds

✅ 04 Post-Cleanup 7 seconds

🔄 05 Export-Artifacts

Uh oh!

psap-forge-bot Bot commented Aug 20, 2026

Uh oh!

openshift-ci Bot commented Aug 20, 2026

Uh oh!

coderabbitai Bot commented Aug 19, 2026 •

edited

Loading

🔴 Execution of `rhaiis` deepseek-v4-dspark profile1 🔴

✅ 00 Pre-Cleanup `1 second`

✅ 01 Prepare `3 seconds`

✅ 02 Preflight `1 second`

❌ 03 Test `1 hour, 5 minutes, 38 seconds`

✅ 04 Post-Cleanup `6 seconds`

aas008 commented Aug 20, 2026 •

edited

Loading

aas008 commented Aug 20, 2026 •

edited

Loading

🛑 Execution of `rhaiis` deepseek-v4-dspark 🛑
🛑 JOB ABORTED - `spec.shutdown=Stop`

✅ 00 Pre-Cleanup `1 second`

✅ 01 Prepare `3 seconds`

✅ 02 Preflight `1 second`

✅ 04 Post-Cleanup `10 seconds`

🔴 Execution of `rhaiis` deepseek-v4-dspark 🔴

✅ 00 Pre-Cleanup `1 second`

✅ 01 Prepare `3 seconds`

✅ 02 Preflight `1 second`

❌ 03 Test `1 hour, 5 minutes, 32 seconds`

✅ 04 Post-Cleanup `7 seconds`