Question: Can we use packages like DifferentialEquations.jl to evolve or model the environment in ReinforcementLearning.jl

[00krishna]

Just a simple question I wanted to check. Can I use packages like DifferentialEquations.jl to model the environment in a Reinforcement learning simulation. Now is the ReinforcementLearning.jl package able to handle that?

What I mean is, I can write a simple differential equation model for the evolution of environment--say some physics based model. But then I push that model into the run() function, I probably just need to make sure that the env() function will take in the current state and action information and then solve() the differential equation system for the subsequent state. I don't think I would need anything special from the ReinforcementLearning.jl package from this.

So just wanted to check if my interpretation is correct in terms of how to implement this integration of DifferentialEquations.jl with the ReinforcementLearning.jl package.
Thanks.

[findmyway]

Yes, almost right. See also #293

[00krishna]

@findmyway I read the #293 post, but it did not really explain how to use DifferentialEquations.jl in the RL experiment. You said that I was almost right. Like do I need to implement a Hook, or something else. If you can give me a hint as to where to implement those methods to updating the environment, I can take it from there. I want to see if I can get at least a simple demo working with a differential equation defining the environment. Then I can write that up as a tutorial, and develop more sophisticated models. But I just need to get the the first step.

[00krishna]

Oh I see, I am reading the associated blog post https://juliareinforcementlearning.org/blog/an_introduction_to_reinforcement_learning_jl_design_implementations_thoughts/#21_the_general_workflow and that seems to give more implementation details. Let me see what I can understand from that, and then I can check back with you to make sure it is correct. Does not seem very hard, but just a matter of knowing where to start.

[findmyway]

• This environment may help you gain a better understanding: AcrobotEnv
• See also https://juliareinforcementlearning.org/docs/How_to_write_a_customized_environment/

[00krishna]

@findmyway Oh yes, this is excellent. Thanks for pointing me in the right direction.