【No.3】feat(inference): unify Rollout, GenRM, and Teacher service by Arazyu · Pull Request #356 · redai-studio/Relax

Arazyu · 2026-09-22T14:49:30Z

Summary

Implements the RFC #71 unified inference service acceptance scope for Rollout, GenRM, and Teacher.

Add the CPU-only InferenceGateway, shared InferenceManager/lifecycle, placement planner, routing schema, and compatibility client.
Validate decoupled, split, and defer placement before startup, with shared drain/offload/onload and deferred Teacher writeback.
Preserve static-model protection, logical-head discovery, legacy interfaces, PG ownership, and rollback semantics.
Add the required CPU/GPU tests, launchers, and bilingual acceptance documentation.

Acceptance evidence

Contract suite: 91 passed, 1 skipped; the skip is the Ray-only port allocator check on a workstation without Ray.
GPU allocation 6654867: PP/PD routing and complete Megatron data flow passed.
GPU allocation 6772050: four-node split training passed, including Teacher writeback before GenRM and completed Megatron steps.
GPU allocation 6780723: four-node decoupled training passed, including Teacher results before training and dynamic Rollout updates.

The public API pages document the unified inference interfaces and usage:

docs/en/api/inference.md
docs/zh/api/inference.md

Validation

python3 -m compileall for changed Python modules: passed
bash -n for validation launchers: passed
git diff --check: passed
pre-commit: unavailable in the local shell (pre-commit executable not installed)

Refs #71

# ⭐ Feature ## Share the inference control plane - Add the CPU-only gateway, common manager, lifecycle coordinator, placement planner, routing schema, and compatibility client for Rollout, GenRM, and Teacher. - Validate split, defer, decoupled placement and reject same-phase co-residency before startup. - Coordinate drain, offload, onload, deferred Teacher publication, and dynamic Rollout updates without changing role workloads. --- # ✅ Tests ## Cover RFC redai-studio#71 contracts and GPU workflows - Add CPU tests for routing, discovery, placement, lifecycle rollback, PG ownership, static-weight protection, legacy clients, and deferred publication. - Add opt-in PP/PD topology and three-role GPU integration tests and validation launchers. - Record passing split and decoupled multi-node training evidence in the bilingual validation guide. --- # 📝 Documentation ## Document the acceptance evidence - Add English and Chinese API and validation guides with reproducible commands and acceptance mapping. - Keep the scope limited to the explicit RFC redai-studio#71 acceptance checks.

rai-studio-bot · 2026-09-22T14:52:09Z

Nyanpasu 审查看板

审查状态： ✅ 已通过

审查版本： 375ed00a6a867372022d091a2759e2bab27e0222

已复查合并 main 后的算法注册、奖励处理、critic 放置及设备/权重同步交互，未发现新增问题；F1–F3 仍已解决。推理契约测试 94 通过、1 跳过，另有 6 项隔离握手检查通过。Pre-commit 已通过；Python/GPU CI 仍在运行，本机未复跑多节点 GPU 验证。

编号	问题	状态	规则来源
F1	首次异步权重同步后未发布网关就绪状态	✅ 已解决	—
F2	同步更新旧握手测试的 Manager 替身	✅ 已解决	—
F3	更新 PR 描述中的验收资料引用	✅ 已解决	—

^{Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.}

rai-studio-bot

已完成审查，有一项非阻塞的启动就绪问题，详见行级评论。

本地 CPU 契约测试为 91 通过、1 跳过；未发现需要阻塞合并的问题。GitHub 暂无 CI 检查记录，完整训练及多节点 GPU 验证未在本机复跑。

^{Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.}

rai-studio-bot · 2026-09-22T14:59:03Z

+                        complete
+                        and group.lifecycle_status == EngineGroupLifecycle.ACTIVE
+                        and not self._is_weight_updating
+                        and self._policy_weights_ready.get(name, False)


优先级：P2

请在纯 fully-async 的首次权重同步成功后发布就绪状态。包含 Actor 的默认模型初始为 False，但 Controller 启动时调用的 Actor.update_weights_fully_async → MegatronTrainRayActor.update_weights_fully_async 只完成 DCS 传输，没有更新此标志；目前异步路径要等首次训练后的 /end_update_weight 才设为 True。因此首次同步已成功、引擎可用时，/rollout/generate、/rollout/v1/chat/completions 和新客户端仍会返回 503。建议在初始同步确认成功后解除门控，并补充“首次训练前即可通过网关生成”的回归测试；失败同步仍应保持不可用。

已确认在 4335eab 修复。Actor 在等待全部训练 worker 的初始异步同步成功后发布 Rollout 就绪状态；同步异常和 actor_fwd-only 分支不会发布。已核对启动调用链，并复跑 CPU 契约测试：94 通过、1 跳过。F1 已解决。

rai-studio-bot

复查通过：此前的启动就绪问题已修复，本轮增量未发现新的审查问题。CPU 契约测试 94 通过、1 跳过（缺少 Ray）；GitHub 暂无 CI 检查记录，本机未复跑多节点 GPU 验证。

^{Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.}

Arazyu · 2026-09-22T15:15:35Z

@SigureMo 任务已完成，请求 review。

rai-studio-bot

新到达的 CI 结果确认了一项测试替身兼容性回归，详见行级评论；此前 F1 仍已解决。本次补充意见不要求阻塞合并。

Pre-commit 检查也因 ruff-format、docformatter 修改文件而失败，需要应用格式化结果。GPU 检查仍在运行。

^{Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.}

rai-studio-bot · 2026-09-22T15:22:06Z

        await self._weight_update_ready.wait()

        self.status = "running"
+        await self.rollout_manager.set_policy_weights_ready.remote(True)


优先级：P2

新增 RPC 后需要同步更新 tests/components/test_rollout_weight_update_handshake.py 的 _ManagerStub。它目前仅提供 health_monitoring_pause 和 set_weight_updating，因此现有 test_end_update_weight_does_not_block_after_failed_can_do 在此处抛出 AttributeError: '_ManagerStub' object has no attribute 'set_policy_weights_ready'。此提交的 Python 3.11 CI 已复现该失败（458 通过、47 跳过后停止）。请给替身补充可等待的就绪 RPC，并保留故障后恢复握手的原有断言；新增的 inference 契约测试没有覆盖这个旧测试调用链。

已确认 c519b19 修复 F2：测试替身补充了可等待的就绪 RPC，原有故障恢复断言保留。由于本机缺少 Ray，我提取生产方法并隔离 Ray 导入边界，执行该文件的 6 项握手测试，全部通过；CPU inference 契约测试另有 94 通过、1 跳过。完整 CI 仍需以新提交结果为准。

# 🐛 Bug Fix ## Keep weight readiness test doubles compatible - Add the rollout readiness RPC to the handshake manager stub. - Preserve failed synchronization behavior while allowing successful resume. --- # 🎨 Style ## Apply required CI formatting - Apply the repository's ruff and docformatter output to touched Python files.

Arazyu · 2026-09-22T15:26:49Z

@rai-studio-bot review

Arazyu · 2026-09-22T15:27:15Z

@SigureMo 不好意思，只验证了rai-studio-bot，忘记检查CI了，我先修复下CI错误

rai-studio-bot

复查通过：F1、F2 均已解决，本轮其余变更仅为格式化，未发现新增问题。

CPU 契约测试 94 通过、1 跳过；隔离 Ray 导入边界后的 6 项握手测试通过。新提交暂未显示 CI 结果，本机未复跑多节点 GPU 验证。

^{Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.}

SigureMo · 2026-09-22T15:29:03Z

                { text: 'Distributed Checkpoint', link: '/zh/guide/distributed-checkpoint' },
-                { text: '健康检查管理器', link: '/zh/guide/health-check-manager' }
+                { text: '健康检查管理器', link: '/zh/guide/health-check-manager' },
+                { text: '统一推理服务验收记录', link: '/zh/guide/unified-inference-validation' }


先自己检查一下代码吧，比如这个应该是要提交上来的吗？

rai-studio-bot

已复核当前提交 c519b19：与上次审查相同，F1、F2 仍已解决，未发现新增代码问题。

Pre-commit 已通过，Python 3.10–3.12 及 GPU 检查仍在运行。此前本地结果为 94 通过、1 跳过，以及隔离 Ray 导入后的 6 项握手测试通过，不能替代完整 CI。

还需回应维护者关于验收记录进入正式文档的意见，确认文档范围；此前代码审查通过不代表该意见已解决或 PR 已满足合并条件。

^{Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.}

# 🐛 Bug Fix ## Preserve legacy engine discovery and startup fixtures - Keep `get_engines_info()` compatible with managers created without the unified inference manager. - Update the startup cleanup fixture for the fully async weight-update arguments and metrics state. --- # 📝 Documentation ## Keep the public inference API focused - Remove internal acceptance reports and experiment details from the public API docs. - Move shared inference links into the related lifecycle and placement sections. --- # 🔩 Chore ## Remove private validation wrappers - Delete one-off cluster scripts containing site-specific paths and runtime settings.

Arazyu · 2026-09-22T15:59:01Z

@rai-studio-bot

rai-studio-bot

本轮代码兼容性修复及文档清理未发现新增运行时问题；F1、F2 仍已解决。CPU 契约测试 94 通过、1 跳过，另有 4 项隔离的引擎信息兼容检查通过；当前提交 Pre-commit 已通过，Python/GPU CI 仍在运行，本机未复跑多节点 GPU 验证。

优先级：P3 — 非行级：PR 描述。请同步更新 Acceptance evidence：其中仍列出已删除的中英文 guide/unified-inference-validation.md，并称 API 页面包含验收记录和复现信息，但这些段落也已移除。建议把验收结果及复现信息放在 PR 描述或可访问的附件中，删除失效的文件引用，并将剩余 API 页面说明改为接口与用法文档。

^{Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.}

rai-studio-bot

复查通过：F3 已解决，PR 描述已移除失效引用，并准确说明 API 页面的用途。此前 F1、F2 仍已解决。

代码提交未变化，沿用此前验证结果；Pre-commit 已通过，Python/GPU CI 仍在运行。本审查结论不替代 CI 结果或维护者的合并决定。

^{Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.}

Arazyu · 2026-09-22T16:10:34Z

@rai-studio-bot

rai-studio-bot

复查 375ed00a 通过：合并 main 后，算法注册与奖励处理、critic 放置及设备/权重同步接口未发现新增回归；此前 F1–F3 仍已解决。

推理契约测试 94 通过、1 跳过，另有 6 项隔离握手检查通过。Pre-commit 已通过，Python/GPU CI 仍在运行；本机缺少相应运行环境，未复跑多节点 GPU 验证。

^{Powered by Nyanpasu with gpt-6-astra medium, please check the suggestions carefully.}

Arazyu · 2026-09-24T00:13:18Z

@SigureMo 任务已完成，请求 review。

Arazyu requested review from Aurelius84, NINGBENZHE and Yangruipis as code owners September 22, 2026 14:49

rai-studio-bot reviewed Sep 22, 2026

View reviewed changes

fix(inference): publish readiness after initial async sync

4335eab

rai-studio-bot approved these changes Sep 22, 2026

View reviewed changes

SigureMo changed the title ~~feat(inference): unify Rollout, GenRM, and Teacher service~~ 【No.3】feat(inference): unify Rollout, GenRM, and Teacher service Sep 22, 2026

SigureMo added the Hackathon 2nd label Sep 22, 2026

rai-studio-bot reviewed Sep 22, 2026

View reviewed changes

SigureMo mentioned this pull request Sep 22, 2026

Relax 开源贡献者计划第二期｜活动流程指南 🚀 #321

Open

SigureMo assigned Yangruipis and SigureMo Sep 22, 2026

rai-studio-bot approved these changes Sep 22, 2026

View reviewed changes

SigureMo requested changes Sep 22, 2026

View reviewed changes

rai-studio-bot reviewed Sep 22, 2026

View reviewed changes

rai-studio-bot approved these changes Sep 22, 2026

View reviewed changes

Merge branch 'main' into codex/rfc71-unified-inference-service

375ed00

rai-studio-bot approved these changes Sep 22, 2026

View reviewed changes

shanyulu mentioned this pull request Sep 24, 2026

【No.4】GenRM 弹性扩缩容设计（RFC） #351

Open

Conversation

Arazyu commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Acceptance evidence

Validation

Uh oh!

rai-studio-bot commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Nyanpasu 审查看板

Uh oh!

rai-studio-bot left a comment

Choose a reason for hiding this comment

Uh oh!

rai-studio-bot Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

rai-studio-bot Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

rai-studio-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Arazyu commented Sep 22, 2026

Uh oh!

rai-studio-bot left a comment

Choose a reason for hiding this comment

Uh oh!

rai-studio-bot Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

rai-studio-bot Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

Arazyu commented Sep 22, 2026

Uh oh!

Arazyu commented Sep 22, 2026

Uh oh!

rai-studio-bot left a comment

Choose a reason for hiding this comment

Uh oh!

SigureMo Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

rai-studio-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Arazyu commented Sep 22, 2026

Uh oh!

rai-studio-bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rai-studio-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Arazyu commented Sep 22, 2026

Uh oh!

rai-studio-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Arazyu commented Sep 24, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Arazyu commented Sep 22, 2026 •

edited

Loading

rai-studio-bot commented Sep 22, 2026 •

edited

Loading

rai-studio-bot left a comment •

edited

Loading