update

.github/workflows/run-tests.yml

@@ -0,0 +1,44 @@
+name: Pytest unit/integration
+
+on:
+  pull_request:
+  push:
+    branches:
+      - main
+
+# Use bash by default in all jobs
+defaults:
+  run:
+    shell: bash
+
+jobs:
+  build-test:
+    name: Test Run (${{ matrix.python-version }}, ${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: ["ubuntu-latest"]
+#        os: ["ubuntu-latest", "macos-latest", "windows-latest"]
+        python-version: ["3.10", "3.11"]
+
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install nox
+      - name: List installed packages
+        run: pip list
+      - name: Run tests with pytest & nox
+        run: |
+          nox -s test-${{ matrix.python-version }}
+#  Codecov is only free for open source / public repos. Not really needed anyway   
+#      - name: Upload coverage to Codecov
+#        if: ${{ matrix.os == 'ubuntu-latest' &&  matrix.python-version == '3.10'}}
+#        uses: codecov/codecov-action@v3
+

Equations.md

@@ -0,0 +1,32 @@
+**Equations**
+
+$$ dW/dt = W*( \frac{(B^x*aB)}{(1 + M^x*hM* aM + B^x*hB* aB)} + \frac{(u*aM*M^x)}{(
+   1 + M^x*hM* aM + B^x*hB* aB)} - d - \omega)$$
+
+$$ dB/dt = B*(rb - \frac{(rb*\alpha_{bb} *B)}{Kb} - \frac{(B^{x - 1}*W *aB)}{(
+   1 + M^x*hM* aM + B^x*hB* aB)} - \frac{(rb*\alpha_{bm} *M)}{Kb} )$$
+
+$$ dM/dt = M*(rm - \frac{(rm*\alpha_{mm}*M)}{Km} - \frac{(M^{x - 1}*W *aM)}{(
+   1 + B^x*hB* aB + M^x*hM* aM)} - \frac{(rm*\alpha_{mb}*B)}{Km} - \mu$$
+
+**Parameter Definitions**
+
++ W: Wolves
++ M: Moose 
++ B: Caribou
++ rb or rm: per capita intrinsic growth rate for caribou and moose, respectively units - #/indiv
++ $$\alpha_{ik}$$: the per capita impact of interaction of species k on species i  
++ $$\alpha_{ii}$$: the per capita impact of intraspecific competition (equal to one)
++ u: conversion factor - moose or primary prey might be bigger or more preferable than endangered species unitless?
++ x: type of functional response (1 = linear, 2 = logarithmic, 3 = logistic)  unitless
+hM.
++ hB: PER CAPITA handling time of prey (moose, caribou) units - time/1 caribou|moose
++ $$\omega | \mu$$: per capita or percentage cull (wolves, moose) units - proportion
++ a :  predation efficiency = search efficiency * attack efficiency = km^2/time * (# successful kills)/(total # attempts) = units  km^2/time
+
+**Notes**
+
++ alpha_bb ->1 & alpha_mm->1 here because we'll want to keep the meaning of Kb and Km as the carrying capacities in the absence of predation or competition.
+
++ We changed the structure of predation in the equation to reflect the overall time spent for predation depends both moose and caribou, thus if a wolf is currently eating a moose,  it cannot also be eating a caribou. This change was made after Niki had suspicions about the immediate lack of of impact moose culls on caribou. Niki thought that moose culls, while eventually good for caribou, might immediately be bad as the predation pressure on caribou would increase before wolves had a change to re-equilibrate to a decrease in prey (more predators than prey present).
+

README.md

@@ -1,2 +1,9 @@
-# rl4caribou
-RL for Caribou Conservation
+# Caribou RL
+
+A DRL-based approach to Caribou conservation based on the methods of the 
+[approx-model-or-approx-soln](https://github.com/boettiger-lab/approx-model-or-approx-soln) project:
+
+[![DOI](https://zenodo.org/badge/572256056.svg)](https://zenodo.org/badge/latestdoi/572256056) 
+
+A three-species foodweb is considered, including interactions between Caribou, Elk and Wolf populations.
+

Scenarios.md

@@ -0,0 +1,37 @@
+**Equilibrium Points**
+There are two equilibrium points in our system: 
++ moose-wolf system with caribou at zero (unstable for caribou)
++ moose-wolf-caribou system
+
+two-species equilibrium: 
+
+$$ eqM:  \frac{(d + \omega)}{(aM (u - d hM - hM \omega))} $$
+
+$$ eqW: \frac{(u (rm - \mu))}{(aM (u - (d + \omega) hM))} - \frac{(
+ u (d + \omega) rm)}{(aM^2 Km (u - (d + \omega) hM)^2)} $$
+
+*note* $$(d hM-u+hM \omega) < 0 $$ for eqM to be positive
+
+add notes about stability *NIKI*
+
+three-species equilibrium: *NIKI*
+
+**Scenario**
+
+We are trying to assess how to get from a 2 species system to a stable 3 species system. 
+
+Management Strategies
++ wolf cull (proportional harvest or fixed escapement harvest rules?)
++ moose cull (prop. harvest)
++ habitat restoration (longer time scale than immediate strategies above) - implemented every 10th timestep?
+
+Implementation
++ pulsed management
++ gradual decline
++ effectiveness in short vs. long-term strategy
+
+We are trying to maximize caribou growth rate while retaining all 3 species on the landscape and keeping costs low. 
+
++ budget - curriculum learning? 
+
+

hyperpars/ppo-caribou-v0-1.yml

@@ -0,0 +1,11 @@
+# stable-baselines3 configuration
+
+algo: "PPO"
+env_id: "Caribou-v0"
+n_envs: 12
+tensorboard: "/home/rstudio/logs"
+total_timesteps: 6000000
+config: {}
+use_sde: True
+id: "1"
+repo: "cboettig/rl-ecology"

hyperpars/rllib/ppo-asm.yml

@@ -0,0 +1,30 @@
+asm:
+    env: rl4fisheries.asm.Asm
+    run: PPO
+    stop:
+        time_total_s: 24000
+    config:
+        lambda: 0.95
+        kl_coeff: 0.5
+        clip_param: 0.2
+        vf_clip_param: 400.0
+        entropy_coeff: 0.0001
+        rollout_fragment_length: auto
+        num_sgd_iter: 10
+        num_envs_per_worker: 24
+        min_time_s_per_iteration: 360
+        lr: 0.0003
+
+        # Run with Learner- and RLModule API (new stack).
+        _enable_learner_api: true
+        _enable_rl_module_api: true
+        # Use N Learner worker on the GPU
+        num_learner_workers: 2
+        num_gpus_per_learner_worker: 1
+        num_gpus: 0  # No GPU needed for driver.
+        # Since we are using learner workers, the driver process does not need
+        # a CPU in particular.
+        num_cpus_for_local_worker: 1
+        # Need to unset this b/c we are using the RLModule API, which
+        # provides exploration control via the RLModule's `forward_exploration` method.
+        exploration_config: {}

hyperpars/tqc-caribou-v0-1.yml

@@ -0,0 +1,11 @@
+# stable-baselines3 configuration
+
+algo: "TQC"
+env_id: "Caribou-v0"
+n_envs: 12
+tensorboard: "/home/rstudio/logs"
+total_timesteps: 6000000
+config: {}
+use_sde: True
+id: "1"
+repo: "cboettig/rl-ecology"