Provide an alternative to docker exec to improve performance

[ChrisTimperley]

Each shell and filesystem interaction via dockerblade consumes at least one, and sometimes multiple, docker exec calls. The overhead of making a docker exec call is sometimes as long as 0.5 seconds, even on high-end hardware.

We should come up with an alternative approach that eliminates that bottleneck for most operations. Below are a few options:

• SSH: we spawn a portable SSH server inside the container and issue commands to it via Paramiko. I suspect that this will create various networking and compatibility issues, and that it may be slightly slower than alternatives.
• XMLRPC: we spawn a portable XML RPC server inside the container. This works pretty well for everything except non-blocking calls (i.e., Popen). To handle non-blocking calls, we would need to either: (a) implement our own protocol for creating, inspecting, and killing processes; or (b) fall back on docker exec for those calls (this is the best solution for now).
  • https://docs.python.org/3/library/xmlrpc.server.html
• HTTP: we spawn a simple HTTP server. This works in almost the same was as above.

Methods

• exists
• is_directory
• is_file
• is_link
• check_call
• check_output
• mkdir
• listdir
• access
• patch
• tempfile
• send_signal