parallel running

[waynezw0618]

Hi Antymon:
Your PPO code is interesting. I am wondering whether it can be used for parallel training, where each episode is a mpi/openmp solver. I would like to perform simulations on a cluster with each episode as a single node openmp based simulation. do you have any suggestion?

Yours Sincerely

Wei

[Antymon]

Hi, this implementation is meant for a single multicore node and even with that respect is rather simple - with multiple environments being called in parallel on each step and centralized network processing resumed once all environments delivered results. TF's graph portability will be useful for you, of course, and perhaps some snippets of my code. I vaguely recall that OpenAI Baselines had MPI implementations in Python that you can look into to port or reuse (there were 2 implementations of PPO initially PPO and PPO2... ). Hope that helps.