Cleanup miscellaneous(#119)

* change default agent color to :red * update gifs * rename env file names * update TagBot.yml * update README * add some info about storage of agent and world
JuliaReinforcementLearning · Jan 9, 2021 · f740aff · f740aff
1 parent 2e8975c
commit f740aff
Show file tree

Hide file tree

Showing 35 changed files with 45 additions and 22 deletions.
diff --git a/.github/workflows/TagBot.yml b/.github/workflows/TagBot.yml
@@ -1,10 +1,12 @@
 name: TagBot
 on:
-  schedule:
-    - cron: 0 0 * * *
+  issue_comment:
+    types:
+      - created
   workflow_dispatch:
 jobs:
   TagBot:
+    if: github.event_name == 'workflow_dispatch' || github.actor == 'JuliaTagBot'
     runs-on: ubuntu-latest
     steps:
       - uses: JuliaRegistries/TagBot@v1

diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 # GridWorlds
 
-This package aims to provide grid world environments (like [gym-minigrid](https://github.com/maximecb/gym-minigrid)) for reinforcement learning research in Julia. The focus of this package is on being **lightweight** and **efficient**.
+This package aims to provide grid world environments for reinforcement learning research in Julia. The focus of this package is on being **lightweight** and **efficient**. This package is inspired by [gym-minigrid](https://github.com/maximecb/gym-minigrid)
 
 ### Table of contents:
 
@@ -57,12 +57,17 @@ This package uses the API provided in [`ReinforcementLearningBase.jl`](https://g
 
 #### Representation of a grid-world
 
-A grid-world environment struct contains within it an instance of `GridWorldBase`, which represents the grid-world.
-An instance of `GridWorldBase` contains a 3-D boolean array (`BitArray{3}`) of size `(num_objects, height, width)`. Each tile of the grid can have multiple objects in it, as indicated by a multi-hot encoding along the first dimension of the `BitArray{3}`.
+A grid-world environment instance (often named `env`) contains within it an instance of `GridWorldBase` (often named `world`), which represents the grid-world. A `world` contains a 3-D boolean array (`BitArray{3}`) (often named `grid`) of size `(num_objects, height, width)`. Each tile of the `grid` can have multiple objects in it, indicated by a multi-hot encoding along the first dimension of the `grid`. The objects in the `world` do not contain any fields. Any related information for such objects that is needed is cached separately as fields of `env`.
+
+`env` contains fields called `world` and `agent` (along with some other fields). The point here is to note that an `agent` is stored separately as a field in `env` instead of an object contained in `world`. You __can__ create a custom field-less agent object and store it in the `world` if you want, but we usually store it as a field in the `env`, since an `agent` often has other information that need caching.
 
 #### Customizing an existing environment
 
-The behaviour of environments is easily customizable. For example, the default implementation of the `ReinforcementLearingBase.reset!` method for an environment is appropriately randomized (like the goal position and agent start position in `EmptyRoom`). In case you need some custom behaviour, you can do so by simply override the `ReinforcementLearningBase.reset!` method, and reusing the rest of the behaviour (like what happens upon taking some action) off the shelf.
+The behaviour of environments is easily customizable. Here are some of the things that one may typically want to customize:
+
+1. Keyword arguments allow for enough flexibility in most environments. For example, most environments allow creation of rectangular worlds.
+1. You can set the navigation style trait (for environments where it makes sense) by `GridWorlds.get_navigation_style(::Type{<:SomeEnv}) = GridWorlds.DIRECTED_NAVIGATION` or `GridWorlds.get_navigation_style(::Type{<:SomeEnv}) = GridWorlds.UNDIRECTED_NAVIGATION`.
+1. You can override specific `ReinforcementLearningBase` methods for customization. For example, the default implementation of the `ReinforcementLearingBase.reset!` method for an environment is appropriately randomized (like the goal position and agent start position in `EmptyRoom`). In case you need some custom behaviour, you can do so by simply overriding the `ReinforcementLearningBase.reset!` method, and reusing the rest of the behaviour (like what happens upon taking some action) as it is. You may also want to customize the `ReinforcementLearningBase.state` method to return the entire grid, or only the agent's view, or anything else you wish. See [RLBase API defaults](https://github.com/JuliaReinforcementLearning/GridWorlds.jl/blob/2e8975c85ce3534c2151121a0791be1ec53a8d31/src/abstract_grid_world.jl#L64) in `abstract_grid_world.jl` for examples.
 
 #### Rendering
 
@@ -86,32 +91,48 @@ The behaviour of environments is easily customizable. For example, the default i
 
 1. #### EmptyRoom
 
-    <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/EmptyRoom.gif" width="300px">
+DirectedNavigation | UndirectedNavigation 
+------------ | -------------
+<img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/empty_room_directed.gif" width="300px"> | <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/empty_room_undirected.gif" width="300px">
 
 1. #### GridRooms
 
-    <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/GridRooms.gif" width="300px">
+DirectedNavigation | UndirectedNavigation 
+------------ | -------------
+<img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/grid_rooms_directed.gif" width="300px"> | <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/grid_rooms_undirected.gif" width="300px">
 
 1. #### SequentialRooms
 
-    <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/SequentialRooms.gif" width="300px">
+DirectedNavigation | UndirectedNavigation 
+------------ | -------------
+<img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/sequential_rooms_directed.gif" width="300px"> | <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/sequential_rooms_undirected.gif" width="300px">
 
 1. #### GoToDoor
 
-    <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/GoToDoor.gif" width="300px">
+DirectedNavigation | UndirectedNavigation 
+------------ | -------------
+<img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/go_to_door_directed.gif" width="300px"> | <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/go_to_door_undirected.gif" width="300px">
 
 1. #### DoorKey
 
-    <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/DoorKey.gif" width="300px">
+DirectedNavigation | UndirectedNavigation 
+------------ | -------------
+<img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/door_key_directed.gif" width="300px"> | <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/door_key_undirected.gif" width="300px">
 
 1. #### CollectGems
 
-    <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/CollectGems.gif" width="300px">
+DirectedNavigation | UndirectedNavigation 
+------------ | -------------
+<img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/collect_gems_directed.gif" width="300px"> | <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/collect_gems_undirected.gif" width="300px">
 
 1. #### DynamicObstacles
 
-    <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/DynamicObstacles.gif" width="300px">
+DirectedNavigation | UndirectedNavigation 
+------------ | -------------
+<img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/dynamic_obstacles_directed.gif" width="300px"> | <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/dynamic_obstacles_undirected.gif" width="300px">
 
 1. #### Sokoban
 
-    <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/Sokoban.gif" width="300px">
+DirectedNavigation | UndirectedNavigation 
+------------ | -------------
+<img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/sokoban_directed.gif" width="300px"> | <img src="https://github.com/JuliaReinforcementLearning/GridWorlds.jl/raw/master/docs/src/assets/img/sokoban_undirected.gif" width="300px">
diff --git a/docs/src/assets/img/CollectGems.gif b/docs/src/assets/img/CollectGems.gif
diff --git a/docs/src/assets/img/DoorKey.gif b/docs/src/assets/img/DoorKey.gif
diff --git a/docs/src/assets/img/DynamicObstacles.gif b/docs/src/assets/img/DynamicObstacles.gif
diff --git a/docs/src/assets/img/EmptyRoom.gif b/docs/src/assets/img/EmptyRoom.gif
diff --git a/docs/src/assets/img/GoToDoor.gif b/docs/src/assets/img/GoToDoor.gif
diff --git a/docs/src/assets/img/GridRooms.gif b/docs/src/assets/img/GridRooms.gif
diff --git a/docs/src/assets/img/SequentialRooms.gif b/docs/src/assets/img/SequentialRooms.gif
diff --git a/docs/src/assets/img/Sokoban.gif b/docs/src/assets/img/Sokoban.gif
diff --git a/docs/src/assets/img/collect_gems_directed.gif b/docs/src/assets/img/collect_gems_directed.gif
diff --git a/docs/src/assets/img/collect_gems_undirected.gif b/docs/src/assets/img/collect_gems_undirected.gif
diff --git a/docs/src/assets/img/door_key_directed.gif b/docs/src/assets/img/door_key_directed.gif
diff --git a/docs/src/assets/img/door_key_undirected.gif b/docs/src/assets/img/door_key_undirected.gif
diff --git a/docs/src/assets/img/dynamic_obstacles_directed.gif b/docs/src/assets/img/dynamic_obstacles_directed.gif
diff --git a/docs/src/assets/img/dynamic_obstacles_undirected.gif b/docs/src/assets/img/dynamic_obstacles_undirected.gif
diff --git a/docs/src/assets/img/empty_room_directed.gif b/docs/src/assets/img/empty_room_directed.gif
diff --git a/docs/src/assets/img/empty_room_undirected.gif b/docs/src/assets/img/empty_room_undirected.gif
diff --git a/docs/src/assets/img/go_to_door_directed.gif b/docs/src/assets/img/go_to_door_directed.gif
diff --git a/docs/src/assets/img/go_to_door_undirected.gif b/docs/src/assets/img/go_to_door_undirected.gif
diff --git a/docs/src/assets/img/grid_rooms_directed.gif b/docs/src/assets/img/grid_rooms_directed.gif
diff --git a/docs/src/assets/img/grid_rooms_undirected.gif b/docs/src/assets/img/grid_rooms_undirected.gif
diff --git a/docs/src/assets/img/sequential_rooms_directed.gif b/docs/src/assets/img/sequential_rooms_directed.gif
diff --git a/docs/src/assets/img/sequential_rooms_undirected.gif b/docs/src/assets/img/sequential_rooms_undirected.gif
diff --git a/docs/src/assets/img/sokoban_directed.gif b/docs/src/assets/img/sokoban_directed.gif
diff --git a/docs/src/assets/img/sokoban_undirected.gif b/docs/src/assets/img/sokoban_undirected.gif
diff --git a/src/envs/collectgems.jl → src/envs/collect_gems.jl b/src/envs/collectgems.jl → src/envs/collect_gems.jl
diff --git a/src/envs/doorkey.jl → src/envs/door_key.jl b/src/envs/doorkey.jl → src/envs/door_key.jl
diff --git a/src/envs/dynamicobstacles.jl → src/envs/dynamic_obstacles.jl b/src/envs/dynamicobstacles.jl → src/envs/dynamic_obstacles.jl
diff --git a/src/envs/emptyroom.jl → src/envs/empty_room.jl b/src/envs/emptyroom.jl → src/envs/empty_room.jl
diff --git a/src/envs/envs.jl b/src/envs/envs.jl
@@ -1,9 +1,9 @@
 include("utils.jl")
-include("emptyroom.jl")
-include("gridrooms.jl")
-include("sequentialrooms.jl")
-include("gotodoor.jl")
-include("doorkey.jl")
-include("collectgems.jl")
-include("dynamicobstacles.jl")
+include("empty_room.jl")
+include("grid_rooms.jl")
+include("sequential_rooms.jl")
+include("go_to_door.jl")
+include("door_key.jl")
+include("collect_gems.jl")
+include("dynamic_obstacles.jl")
 include("sokoban/sokoban.jl")
diff --git a/src/envs/gotodoor.jl → src/envs/go_to_door.jl b/src/envs/gotodoor.jl → src/envs/go_to_door.jl
diff --git a/src/envs/gridrooms.jl → src/envs/grid_rooms.jl b/src/envs/gridrooms.jl → src/envs/grid_rooms.jl
diff --git a/src/envs/sequentialrooms.jl → src/envs/sequential_rooms.jl b/src/envs/sequentialrooms.jl → src/envs/sequential_rooms.jl
diff --git a/src/objects.jl b/src/objects.jl
@@ -63,7 +63,7 @@ mutable struct Agent{I, C} <: AbstractObject
     inventory::I
 end
 
-function Agent(; inventory_type = Union{Nothing, AbstractObject}, color = :green, pos = CartesianIndex(2, 2), dir = RIGHT, inventory = nothing)
+function Agent(; inventory_type = Union{Nothing, AbstractObject}, color = :red, pos = CartesianIndex(2, 2), dir = RIGHT, inventory = nothing)
     Agent{inventory_type, color}(pos, dir, inventory)
 end