[roadmap] RL-Insight Q3 Roadmap

[tardis-key]

Online Monitoring of Reasoning Trajectories Based on Uni-Agent

verl-project/verl#7509
Overall design: RFC doc

• New data reporting interface in RL-Insight, aligning sample data format with uni-agent [monitor-api] feat: add direct and async trace span reporting #132
• Grafana-based rendering for trajectory visualization [monitor-config] feat: Agent Loop trajectory nested Repeat static dashboard #135
• Uni-agent data reporting integration [tools, agents] feat: Instrument Uni-Agent with RL-Insight Agent Loop Observability uni-agent#124

Support for Additional RL Frameworks

• Slime framework integration
• AReal framework integration
• ROLL framework integration

Support for service-oriented deployment

• Single-server, multi-client deployment

Experimentation and refinement of online monitoring capabilities

• Docker support
• RL-Insight performance tuning guide
• Multi-hardware metrics collection and dashboards (CPU, NPU, GPU)
• Update and optimize readthedocs [doc] refactor: optim readdocs #146

[ETOgaosion]

Thinking that we can provide a metrics list that we currently supported collection and prepare to support? And then we will be clear that if there are other requirements. And for fully async or agentic scenario, current verl's metrics and timeline may not provide enough insights.

[mengchengTang]

Thinking that we can provide a metrics list that we currently supported collection and prepare to support? And then we will be clear that if there are other requirements. And for fully async or agentic scenario, current verl's metrics and timeline may not provide enough insights.

I will provide a full set of metrics before verl-project/verl#6680 is merged.

[12138flash]

I can take ownership of Grafana-based HTML rendering for trajectory visualization.
My initial analysis and implementation work are documented in #106: #106
I’ll continue advancing this based on that approach and post progress updates in this roadmap issue.

[mengchengTang]

Roadmap update:

Docker support
Single-server, multi-client deployment
RL-Insight performance tuning guide
Multi-hardware metrics collection and dashboards (CPU, NPU, GPU)

[mengchengTang]

Roadmap update:

Update and optimize readthedocs

[cyyecao-lappland]

I would like to take the unchecked Slime framework integration roadmap item.

Proposed scope:

1. Map Slime's rollout/training records onto RL-Insight's existing sample-reporting and run/experiment identity contracts, rather than creating a second visualization schema.
2. Add a thin optional reporter/adapter so the initial integration does not require invasive changes to Slime's trainer core.
3. Cover trajectory/sample identity, reward, turn count, rollout and training timing, failure state, and policy/training-step metadata needed to interpret asynchronous runs.
4. Add schema/serialization unit tests and a small end-to-end run that verifies data reaches RL-Insight and appears in the existing dashboards.
5. Document a reproducible local setup. I can validate the Slime side with a small model on up to 4×RTX 3090 GPUs; this would be functional integration validation, not a large-cluster performance claim.

Before implementation, would the maintainers prefer the adapter to live primarily in rl-insight, or as a Slime-side reporting hook with only the shared schema/client in this repository? Once that ownership boundary is confirmed, I am happy to create a dedicated tracking issue/PR and own this roadmap item.

[tardis-key]

I would like to take the unchecked Slime framework integration roadmap item.

@cyyecao-lappland weclome! You can refer to this PR: verl-project/verl#6680. The overall approach is that rl-insight provides the underlying data ingestion interfaces and corresponding dashboards, while the RL framework side (e.g., slime) implements the corresponding Adapter and manages lifecycle concerns such as initialization and environment variable propagation.

You can submit an RFC to track the Slime adaptation work, and use it to sync with us on the design plan and progress. Also, you can join our Lark group via the QR code in the repository's README for further discussion.