Replace UEFS daemon with inline mount/copy code

[hach-que]

Currently in order to mount UEFS packages from a registry, you need to have the UEFS daemon installed. The original reason that the daemon was created was because you can't mount VHDs on Windows unless you're running with elevated permissions.

The problem is that the UEFS daemon is not 100% reliable - it has very complex transactional code to handle multiple incoming pull requests, and there's some weird bug where it can stall on macOS. When the UEFS daemon gets into an inconsistent state, it's very hard to fix or diagnose without just blowing away the whole state and letting it re-pull all packages.

We should remove or minimalize the UEFS daemon and instead do all the work inline inside the UET process. This roughly looks like:

• When pulling packages from a registry on macOS, or on Windows without WinFsp installed:
  • Check the "downloaded packages" directory.
  • Files are named by the package hash.
  • When the package does not exist:
    • Check for any existing files with <hash>.<pid>.<ext>. Attempt to open them - files that are actively still being downloaded by a process will be locked and the open will fail. For any files we can successfully open, delete them. (I'm not sure whether "delete on close" + "rename" will have the expected behaviour here).
    • If any files fail to open because they're currently downloading, for the one with the largest "actual used disk space", monitor the "actual used disk space" until it's using 100% of the file size. Once it reaches that point, wait in a loop until the package exists. If it doesn't appear after a certain time elapses, go back to the start of this process (potentially starting the download ourselves).
    • If there are no files "in-progress", truncate a file to the desired length with a name of <hash>.<pid>.<ext> (i.e. reserve the full space up-front).
    • Periodically throughout the download, check if the desired <hash>.<ext> package appears. If it does, cancel downloading, delete our temporary file and use the package that some other process downloaded first.
    • When the download completes, hash the content to make sure it matches the expected hash. If the hash doesn't match, delete our temporary file and fail. Otherwise, rename our file to <hash>.<ext> and use it as the package.
  • Allocate a folder for the differencing layer (as our client already does) and mount the VHD or sparseimage. If we want to support developers mounting UEFS packages on Windows without admin permissions, we can make the UEFS daemon just be a basic "please mount/unmount this VHD" service.
• When pulling packages on Windows with WinFsp installed:
  • Check the "package backing cache" directory.
  • Backing files and index files are named by the package hash.
  • We need to memory map our backing cache and it's index file. We check our memory mapped index to see if a block is missing. If it is, download that block and write it back to our memory map, write our memory mapped index flag and then serve the block from what we downloaded. If it isn't, we serve the block from our memory mapped backing cache. As long as our memory mapped file is configured to allow multiple processes to read/write at the same time, this should allow processes to collaborate on filling the cache on-demand. The only issue I could see here is if writing the block to the memory mapped file somehow clears the block with zeroes due to some Win32/.NET API weirdness, but I can't see why that would happen. Overwriting a block with the same content shouldn't otherwise cause issues because the block content will be the same even after a partial write.
  • Allocate a folder for the WinFsp proxy filesystem. Each UET process is now mounting it's own WinFsp filesystem. This filesystem can now also be vastly simplified because it just needs to serve one file at the root, which the filesystem mount can be pre-configured for with the desired filename + in-memory backing cache / cache index file handles.
  • Allocate a folder for the differencing layer (as our client already does) and mount the VHD or sparseimage. If we want to support developers mounting UEFS packages on Windows without admin permissions, we can make the UEFS daemon just be a basic "please mount/unmount this VHD" service.