Graceful recovery on partial failures

[varungandhi-src]

A customer reported that when lsif-clang crashed, it didn't generate partial input. I suspect the parallel executor is using threads instead of processes; if it were using a separate process per file, this wouldn't happen. However, using threads enables easy sharing of data (and likely reduces memory consumption due to that), but it means that one thread going down would crash the whole process.

It would be worthwhile to investigate what it'd take to switch to a multi-process architecture. As a fall-back, it might be worth investigating if we could write a small script to drive lsif-clang externally, i.e., for each entry in compile_commands.json, generate a separate file, and spawn a separate process for indexing, and then stitch all the LSIF dumps together.