fix(resolver): redesign resolver cache

Redesign the resolver cache to address multiple issues.

1. The cache now holds all candidates from the provider. Before it only
   stored candidates that also fulfilled the requirement and
   constraints.
2. The cache now takes sdist url, GitHub repo, and GitLab project into
   account. Before a cache for local PyPI and pypi.org were mixed.
3. GenericProvider no longer caches candidates. There is no way to
   construct a good cache key. Bootstrapper is using GenericProvider in
   way that does not benefit from caching either.
4. There is just one global cache object for all providers. The new
   design makes it easier to clear all caches or just the cache for a
   single identifier.
5. The custom logic for each provider class is now in `find_candidates`
   function. The method just has to return an iterable of candidates.
   The caching logic is handled by the rest of the code.
6. Consumers can now opt out of caching with `use_resolver_cache=False`
   argument.
7. All resolver classes now require keyword arguments. The base provider
   no longer takes PyPI-only arguments like `include_sdists`.

Fixes: #766
Signed-off-by: Christian Heimes <[email protected]>

Author: tiran

docs/hooks.rst

@@ -156,14 +156,15 @@ Resolver hooks
 
         from fromager import resolver
 
-        VERSION_MAP = {'1.0': 'first-release', '2.0': 'second-release'}
+        VERSIONS = {
+            "https://pkg.example/pkg-1.0.tar.gz": "1.0",
+            "https://pkg.example/pkg-2.0.tar.gz": "2.0",
+        }
 
         def _version_source(
                 identifier: str,
-                requirements: resolver.RequirementsMap,
-                incompatibilities: resolver.CandidatesMap,
-            ) -> typing.Iterable[Version]:
-            return VERSION_MAP.keys()
+            ) -> typing.Iterable[tuple[str, Version]]:
+            return VERSIONS.items()
 
 
         def get_resolver_provider(ctx, req, include_sdists, include_wheels, sdist_server_url):

src/fromager/bootstrapper.py

@@ -901,9 +901,11 @@ def _resolve_from_version_source(
             return None
         try:
             # no need to pass req type to enable caching since we are already using the graph as our cache
+            # do not cache candidates
             provider = resolver.GenericProvider(
-                version_source=lambda x, y, z: version_source,
+                version_source=lambda identifier: version_source,
                 constraints=self.ctx.constraints,
+                use_resolver_cache=False,
             )
             return resolver.resolve_from_provider(provider, req)
         except Exception as err: