Skip to content

Conversation

@pseudo-rnd-thoughts
Copy link
Member

Description

For the FlattenObservation connector, in the mutli-agent case, if any of the agent's had a nested space, if it was ignored then we used the base_struct version of the observation_spaces rather than the actual spaces which for nested spaces is a problem.
FYI, I believe that get_base_struct_from_space can be removed as Gymnasium has added the ability to treat Dict and Tuple spaces as like a dict or tuple for iterating, calling .keys(), .values() and .items(). For a future PR possibly

Related issues

Fixes #59849

@pseudo-rnd-thoughts pseudo-rnd-thoughts changed the title [rllib] Update flatten_observations.py for nested spaces for multi-ag… [rllib] Update flatten_observations.py for nested spaces for ignored multi-agent Jan 7, 2026
@pseudo-rnd-thoughts pseudo-rnd-thoughts marked this pull request as ready for review January 7, 2026 12:26
@pseudo-rnd-thoughts pseudo-rnd-thoughts requested a review from a team as a code owner January 7, 2026 12:26
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses a bug in the FlattenObservations connector for multi-agent scenarios with nested observation spaces. The core fix correctly preserves the original observation space for agents whose observations are not flattened, which is crucial for nested spaces. The changes are well-implemented and include several improvements: a new multi-agent example in the docstring that clearly demonstrates the fix, added type hints for better readability, and new assertions that enhance the robustness of the code by validating space types. The docstring for the __init__ method has also been updated for better clarity. Overall, this is a high-quality contribution that not only fixes the bug but also improves the code's maintainability and documentation.

@ray-gardener ray-gardener bot added the rllib RLlib related issues label Jan 7, 2026
@ArturNiederfahrenhorst ArturNiederfahrenhorst enabled auto-merge (squash) January 7, 2026 15:02
@github-actions github-actions bot disabled auto-merge January 7, 2026 15:02
@github-actions github-actions bot added the go add ONLY when ready to merge, run all tests label Jan 7, 2026
@ArturNiederfahrenhorst ArturNiederfahrenhorst merged commit 8e131f4 into ray-project:master Jan 9, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

go add ONLY when ready to merge, run all tests rllib RLlib related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RLlib] Nested spaces (tuple or dict) break FlattenObservations in partially active multi-agent setups

2 participants