Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving example environments from RLBase to RLEnvs? #167

Closed
Sid-Bhatia-0 opened this issue Dec 16, 2020 · 6 comments
Closed

Moving example environments from RLBase to RLEnvs? #167

Sid-Bhatia-0 opened this issue Dec 16, 2020 · 6 comments
Labels

Comments

@Sid-Bhatia-0
Copy link
Member

I was wondering why do the example environments currently belong in RLBase instead of RLEnvs.

@findmyway
Copy link
Member

Yeah I should have document this in the README.

They are just examples to demonstrate how to write different kinds of environments and serve as test cases of interfaces.

They were not to be exposed. But then I realized that someof them may be reused in RLIntro. So I exported them all.

I intend to make RLenvs a wrapper of many 3-rd party envs, which may have some large dependencies, like gr, makie, CxxWrap

@Sid-Bhatia-0
Copy link
Member Author

I intend to make RLenvs a wrapper of many 3-rd party envs, which may have some large dependencies, like gr, makie, CxxWrap

Are you suggesting that this should be the only purpose of RLEnvs, to provide wrappers for other environments rather than contain implementations of environments?

There are simple environments, classic control ones like CartPole, that are implemented within RLEnvs itself. My question is why can't the example environments currently present in RLBase be moved to a place like RLEnvs or into a package of their own containing simple environments that don't have any large dependencies?

Mainly for the purpose of keeping RLBase.jl as minimal as possible.

@findmyway
Copy link
Member

Are you suggesting that this should be the only purpose of RLEnvs, to provide wrappers for other environments rather than contain implementations of environments?

Yes, see also #123

My question is why can't the example environments currently present in RLBase be moved to a place like RLEnvs or into a package of their own containing simple environments that don't have any large dependencies?

Like I said above, one important role of these environments is to test the expressiveness of APIs defined in RLBase. And I do not plan to add more in RLBase, unless it enriches the APIs. They serve as a golden guideline for how to write customized environments so I want to keep them updated on the master branch and easy for people to find. Splitting them into an independent package is also fine but I think no one else would use it given that they are just toy examples.

@Sid-Bhatia-0
Copy link
Member Author

one important role of these environments is to test the expressiveness of APIs defined in RLBase
They serve as a golden guideline for how to write customized environments so I want to keep them updated on the master branch and easy for people to find.

What if we clearly state in the README to check out these simple environments as examples, and provide a link to this package. RLBase README isn't cluttered with too much info. I don't think people can miss this.

Splitting them into an independent package is also fine but I think no one else would use it given that they are just toy examples.

Why? Don't you think a simple implementation of something like MultiArmedBandits won't be useful to anyone?
Since it has already been implemented, we might as well provide it as a package that a user can use off the shelf.

Also, exporting these environment names through RLBase seems to clash with its minimalism, and the core purpose of only providing an API. Wrapper environments are allowed since we don't have a better place for them right now, but these environments can easily be put in a package of their own.

A package that uses RLBase heavily (like GridWorlds.jl) does not need variables like TicTackToeEnv entering its namespace.
Moreover, if one is to actually implement TicTacToeEnv in GridWorlds.jl (just saying), then there would also be unnecessary name clashing.

@findmyway
Copy link
Member

I see. Let me have a think. Thanks for your suggestions.

@findmyway
Copy link
Member

Examples are now moved into RLEnvs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants