index.md

layout	title
default	Modeling Agents with Probabilistic Programs

Modeling Agents with Probabilistic Programs

This book describes and implements models of rational agents for (PO)MDPs and Reinforcement Learning. One motivation is to create richer models of human planning, which capture human biases and bounded rationality.

Agents are implemented as differentiable functional programs in a probabilistic programming language based on Javascript. Agents plan by recursively simulating their future selves or by simulating their opponents in multi-agent games. Our agents and environments run directly in the browser and are easy to modify and extend.

The book assumes basic programming experience but is otherwise self-contained. It includes short introductions to "planning as inference", MDPs, POMDPs, inverse reinforcement learning, hyperbolic discounting, myopic planning, and multi-agent planning.

For more information about this project, contact Owain Evans.

Table of contents

{% assign sorted_pages = site.pages | sort:"name" %}

{% for p in sorted_pages %} {% if p.hidden %} {% else %} {% if p.layout == 'chapter' %} {% if p.is_section %} {% if p.status == 'stub' %}

1. {{ p.title }}
   {% else %}
2. {{ p.title }}
   {% endif %} {{ p.description }} {% else %} {% if p.status == 'stub' %}
   1. {{ p.title }}
      {% else %}
   2. {{ p.title }}
      {% endif %} {{ p.description }}
      {% endif %} {% endif %} {% endif %} {% endfor %}

Citation

Please cite this book as:

Owain Evans, Andreas Stuhlmüller, John Salvatier, and Daniel Filan (electronic). Modeling Agents with Probabilistic Programs. Retrieved from http://agentmodels.org. [bibtex]

@misc{agentmodels,
  title = {% raw %}{{Modeling Agents with Probabilistic Programs}}{% endraw %},
  author = {Evans, Owain and Stuhlm\"{u}ller, Andreas and Salvatier, John and Filan, Daniel},
  year = {2017},
  howpublished = {\url{http://agentmodels.org}},
  note = {Accessed: }
}

Open source

• Book content
  Markdown code for the book chapters
• WebPPL
  A probabilistic programming language for the web
• WebPPL-Agents
  A library for modeling MDP and POMDP agents in WebPPL
  

Acknowledgments

We thank Noah Goodman for helpful discussions, all WebPPL contributors for their work, and Long Ouyang for webppl-viz. This work was supported by Future of Life Institute grant 2015-144846 and by the Future of Humanity Institute (Oxford).