recover from timeout

[abh]

If a deployment job "hangs", there doesn't seem to be a way to recover other than restarting the process.

[russellhaering]

Yeah, this is a problem we need to figure out how to deal with. Much of the difficulty stems from the fact that there is no convention in node for canceling/stopping a set of related ongoing operations.

One approach would be to develop our own convention and attach a reference to every ongoing operation to the baton so that if a deployment times out (or is canceled) they can all be canceled or killed if possible.

A second approach, which seems much safer to me, is to run deployments in a subprocess (or maybe use isolates?). There are a few other potential advantages to this as well (throwing an error in a deployment wouldn't crash Dreadnot), so pursuing it is definitely high on my list of priorities.

[abh]

A subprocess would make sense to me. Isolates didn't make it far: http://groups.google.com/group/nodejs/msg/6b8b8a487d2ab817