[9.0.0] Fix cycles when checking the local repo contents cache

[bazel-io]

Fixes #27517 by checking Skyframe deps in batches that stop right before any dep that may cause a cycle if checked while previous deps are out-of-date.

This is accompanied by a restructuring of RepoRecordedInput that consolidates all Skyframe logic associated with the computation of the corresponding value exclusively within that class. This will also be helpful in adding support for dynamic inputs to the remote repo contents cache in future work.

Also made the entirety of RepositoryFetchFunction use skyframe workers, so that checking the up-to-dateness of local repo contents cache entries isn't quadratic.

Closes #28206.

Co-authored-by: Xudong Yang [email protected]
PiperOrigin-RevId: 855252657
Change-Id: Ica18760ae79da5155fc0f3d8cd4f24c52a034c86

Commit 72a25a9

[gemini-code-assist]

Code Review

This pull request introduces a robust fix for potential cycles in Skyframe when checking the local repository contents cache. The core change is to check dependencies in batches, separating dependencies that could cause cycles (like files in other external repositories) from those that cannot. This is implemented through a significant and well-executed refactoring of RepoRecordedInput, introducing concepts like canBeRequestedUnconditionally and splitIntoBatches.

Additionally, the pull request improves performance by refactoring RepositoryFetchFunction to use Skyframe workers, addressing a quadratic complexity issue. The code is also made cleaner and more maintainable by consolidating logic within RepoRecordedInput and simplifying related classes like DigestWriter and StarlarkBaseExternalContext.

The changes are well-tested, with new tests specifically targeting the cycle scenario that is now fixed. Overall, this is an excellent set of changes that improves both the correctness and performance of Bazel's repository fetching mechanism.