Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add retrace #972

Closed
wants to merge 13 commits into from
Closed

add retrace #972

wants to merge 13 commits into from

Conversation

HenriDeh
Copy link
Member

@HenriDeh HenriDeh commented Sep 8, 2023

This PR adds a function to compute the retrace bellman operator as described in this paper. It is described as an algorithm so I put it in RLZoo but it can be argued that this is more of a component used by RL algorithms . I'm thus open to move this to RLCore.

The tests are incomplete. For now I only test if the function returns the expected operators given toy networks. This needs to also be tested with actual networks for the API.

PR #966 must also be merged because this uses the new 'target' api.

PR Checklist

  • Update NEWS.md?
  • Unit tests for all structs / functions?
  • Integration and correctness tests using a simple env?
  • PR Review?
  • Add or update documentation?
  • Write docstrings for new methods?

@codecov
Copy link

codecov bot commented Sep 8, 2023

Codecov Report

Merging #972 (7673022) into main (a8ae878) will increase coverage by 23.25%.
Report is 6 commits behind head on main.
The diff coverage is 0.00%.

❗ Current head 7673022 differs from pull request most recent head 8256e76. Consider uploading reports for the commit 8256e76 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##            main     #972       +/-   ##
==========================================
+ Coverage   0.02%   23.28%   +23.25%     
==========================================
  Files        210      224       +14     
  Lines       7437     7718      +281     
==========================================
+ Hits           2     1797     +1795     
+ Misses      7435     5921     -1514     
Files Changed Coverage Δ
...earningZoo/src/algorithms/bootstrapping/retrace.jl 0.00% <0.00%> (ø)

... and 63 files with indirect coverage changes

@HenriDeh
Copy link
Member Author

I'm closing this as I am not planning on finishing this soon, and the upcoming breaking changes mean one might as well start from scratch to implement this.

@HenriDeh HenriDeh closed this Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant