Cloudformation: Environment & Configuration discussion

[Mindgames]

Infrastructure

We have agreed on partnership between Amazon & Deep learning labs.

Did have a phone call with one of Amazon's cloud engineers last week, there suggested solution was using https://aws.amazon.com/cloudformation/

With Cloudformation we should be able to do something like this

• With Master account generate 50 new sub / amazon accounts at once.
• Provide a config file that states access to resources etc
• Have each account spawn exactly the same, config, hardware, software

Once event is over we kill all sub accounts and on new event we spawn new. We are free to use whatever we want, he recommended EC2

Software

• Docker? ( works in cloudformation, AWS engineer said docker is supported, any other solution to take in to account? )
• Jupyter Lab & Mujoco
• OS? Ubuntu should be most stable as i understood
• Screen? There is a docker VNC solution in docker filer that could work, however we should check if there is anything more high performance, for example teamviewer maybe is more suitable ?

How to contribute

• Clone repo, look in dockerfile
• Get involved in discussions here at Github & on Slack
• To get Amazon cloudfront master account login contact Mathias / @Mindgames
• Docker can be tested locally

Plan

• Read documentation & discuss solutions
• Decide on & Test Docker configuration
• Test it on cloudformation

[BonusPlay]

Are the benefits of cloudformation enough to justify additional configuration when compare to telling participants to "install docker"? (I'm guessing by doing that we are giving them additional computational power like in google collab)

As far as I understand cloudformation will spawn us new VMs with ubuntu that participants can interact. If it's already a VM, there's no real reason for docker.

There is a docker VNC solution in docker filer that could work, however we should check if there is anything more high performance, for example teamviewer maybe is more suitable ?

As far as I know we're probably not falling into free terms of use of teamviewer. Also, there shouldn't be any option that will give us more performance.

[Mindgames]

Are the benefits of cloudformation enough to justify additional configuration when compare to telling participants to "install docker"? (I'm guessing by doing that we are giving them additional computational power like in google collab)

Yes computing power is one of the issues +basically need a solution that just will work out of the box. Then if participants wanna run local setup or local docker, they ofc can use same docker file. There is requests for running DLL in additional cities in Poland and also other countries.

As far as I understand cloudformation will spawn us new VMs with ubuntu that participants can interact. If it's already a VM, there's no real reason for docker.

Going to read up on it now myself also. We need to install all apt & pip packages, Jupyter labs and various other drivers and environments like Mujoco.

Anyone that have or can make complete docker setup that include Ubuntu desktop, VNC, Jupyter lab & mujoco and required drivers to make it run out of the box?