The Arcade Learning Environment

The Arcade Learning Environment (ALE) is a simple framework that allows researchers and hobbyists to develop AI agents for Atari 2600 games. It is built on top of the Atari 2600 emulator Stella and separates the details of emulation from agent design. This video depicts over 50 games currently supported in the ALE.

For an overview of our goals for the ALE read The Arcade Learning Environment: An Evaluation Platform for General Agents. If you use ALE in your research, we ask that you please cite this paper in reference to the environment. See the Citing section for BibTeX entries.

Features

• Object-oriented framework with support to add agents and games.
• Emulation core uncoupled from rendering and sound generation modules for fast emulation with minimal library dependencies.
• Automatic extraction of game score and end-of-game signal for more than 100 Atari 2600 games.
• Multi-platform code (compiled and tested under macOS, Windows, and several Linux distributions).
• Python bindings through pybind11.
• Native support for OpenAI Gym.
• Visualization tools.

Quick Start

The ALE currently supports three different interfaces: C++, Python, and OpenAI Gym.

Python

You simply need to install the ale-py package distributed via PyPI:

pip install ale-py

Note: Make sure you're using an up to date version of pip or the install may fail.

You can now import the ALE in your Python projects with

from ale_py import ALEInterface

ale = ALEInterface()

ROM Management

The ALE doesn't distribute ROMs but we do provide a couple tools for managing your ROMs. First is the command line tool ale-import-roms. You can simply specify a directory as the first argument to this tool and we'll import all supported ROMs by the ALE.

ale-import-roms roms/

[SUPPORTED]       breakout   roms/breakout.bin
[SUPPORTED]       freeway    roms/freeway.bin

[NOT SUPPORTED]              roms/custom.bin

Imported 2/3 ROMs

Furthermore, Python packages can expose ROMs for discovery using the special ale-py.roms entry point. For more details check out the example python-rom-package.

Once you've imported a supported ROM you can simply import the path from the ale-py.roms package and load the ROM in the ALE:

from ale_py.roms import Breakout

ale.loadROM(Breakout)

OpenAI Gym

Gym support is included in ale-py. Simply install the Python package using the instructions above. You can also install gym[atari] which also installs ale-py with Gym.

As of Gym v0.20 and onwards all Atari environments are provided via ale-py. We do recommend using the new v5 environments in the ALE namespace:

import gym

env = gym.make('ALE/Breakout-v5')

The v5 environments follow the latest methodology set out in Revisiting the Arcade Learning Environment by Machado et al..

The only major change difference from Gym's AtariEnv is that we'd recommend not using the env.render() method in favour of supplying the render_mode keyword argument during environment initialization. The human render mode will give you the advantage of: frame perfect rendering, audio support, and proper resolution scaling. For more information check out docs/gym-interface.md.

For more information on changes to the Atari environments in OpenAI Gym please check out the following blog post.

C++

The following instructions will assume you have a valid C++17 compiler and vcpkg installed.

We use CMake as a first class citizen, and you can use the ALE directly with any CMake project. To compile and install the ALE you can run

mkdir build && cd build
cmake ../ -DCMAKE_BUILD_TYPE=Release
cmake --build . --target install

There are optional flags -DSDL_SUPPORT=ON/OFF to toggle SDL support (i.e., display_screen and sound support; OFF by default), -DBUILD_CPP_LIB=ON/OFF to build the ale-lib C++ target (ON by default), and -DBUILD_PYTHON_LIB=ON/OFF to build the pybind11 wrapper (ON by default).

Finally, you can link agaisnt the ALE in your own CMake project as follows

find_package(ale REQUIRED)
target_link_libraries(YourTarget ale::ale-lib)

Citing

If you use the ALE in your research, we ask that you please cite the following.

M. G. Bellemare, Y. Naddaf, J. Veness and M. Bowling. The Arcade Learning Environment: An Evaluation Platform for General Agents, Journal of Artificial Intelligence Research, Volume 47, pages 253-279, 2013.

In BibTeX format:

@Article{bellemare13arcade,
    author = {{Bellemare}, M.~G. and {Naddaf}, Y. and {Veness}, J. and {Bowling}, M.},
    title = {The Arcade Learning Environment: An Evaluation Platform for General Agents},
    journal = {Journal of Artificial Intelligence Research},
    year = "2013",
    month = "jun",
    volume = "47",
    pages = "253--279",
}

If you use the ALE with sticky actions (flag repeat_action_probability), or if you use the different game flavours (mode and difficulty switches), we ask you that you also cite the following:

M. C. Machado, M. G. Bellemare, E. Talvitie, J. Veness, M. J. Hausknecht, M. Bowling. Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents, Journal of Artificial Intelligence Research, Volume 61, pages 523-562, 2018.

In BibTex format:

@Article{machado18arcade,
    author = {Marlos C. Machado and Marc G. Bellemare and Erik Talvitie and Joel Veness and Matthew J. Hausknecht and Michael Bowling},
    title = {Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents},
    journal = {Journal of Artificial Intelligence Research},
    volume = {61},
    pages = {523--562},
    year = {2018}
}