About parallel efficiency

[realjoenguyen]

Hi,

Thanks for the nice implementation,

I have one question about parallel threads: in the original implementation, they use Sever-Client framework to have truly asynchronous threads among data collection (envs), policy (parallel_actor) and policy+world model learning (parallel_learner). Link

Meanwhile, in your implementation, if I understand correctly, you use iteration for env in envs (like Dreamerv2) in tools.py - def simulate link)

I wonder if this difference has any significant efficiency in clockwise and FPS?

Looking forward to your answer!

[NM512]

Thank you for your interest in the project and for bringing up this important point about parallel processing.

I'd like to clarify that parallel processing capabilities are indeed incorporated in our codebase. You can find the implementation of the parallel processing logic within the parallel.py file in our repository, which is available here: parallel.py.

Regarding the usage of this parallel functionality, it's currently configured to be utilized specifically with the Minecraft environment. This decision is based on my observations and testing, where I found significant benefits of parallelization in this particular environment. In other environments, the advantages were not as pronounced, and hence the parallel functionality is not activated by default because environmental interactions are much faster than learning loop. However, the framework is flexible, and users can experiment with parallel processing in different environments by adjusting the configurations.

I hope this clarifies your query. If you have further questions or need more details, feel free to ask.