Gymnasium

Presentation : Gymnasium is an open-source Python library that provides a standard API for reinforcement learning (RL) environments. It is the maintained fork of OpenAI Gym and serves as the bridge between AI agent (trained with PPO from Stable Baselines3) and the game logic.

Core Methods and Spaces

Every Gymnasium environment relies on these fundamental building blocks:

• Observation Space: Defines the structure of the data the agent "sees" (e.g., ball position, player position).
• Action Space: Defines the allowed moves (e.g., 0: Stay, 1: Up, 2: Down).
• reset(): Resets the environment to an initial state and returns the first observation.
• step(action): Processes the action, updates the game state, and returns:
  • Observation: The new state of the game.
  • Reward: A numerical value (e.g., +1 for scoring, -1 for conceding).
  • Terminated: Boolean indicating if the game ended (e.g., someone reached 11 points).
  • Truncated: Boolean indicating if the episode was cut short (e.g., time limit).
  • Info: A dictionary for auxiliary debugging data.

---

Do's & Don'ts

✅ Do	❌ Don't
Use float32: Ensure your Box spaces use np.float32 for compatibility with PyTorch/Stable Baselines3.	Heavy loops: Avoid complex calculations inside step(); keep the physics engine efficient.
Check Compliance: Use gymnasium.utils.env_checker.check_env to verify your environment follows the API.	Global variables: Don't store game state in global variables; keep everything inside the class instance.
Reward Shaping: Provide small intermediate rewards (e.g., for hitting the ball) to help the agent learn faster.	Ignore Terminated: Don't forget to stop the episode; otherwise, the agent will learn "infinite" loops.

---

Resources

Type	Resource	Notes
📄	Official Docs	Complete API reference
💻	Gymnasium Github	Source code and issues
📄	Environment Creation Guide	Step-by-step tutorial

Legend: 📄 Doc, 📘 Book, 🎥 Video, 💻 GitHub, 📦 Package, 💡 Blog