✨ Add multi-repo scanning: --repos, --org, and optional --combined output #4793

gabrielsoltz · 2025-09-16T20:51:07Z

What kind of change does this PR introduce?

This PR introduces support for scanning multiple repositories in a single invocation, while preserving existing behavior by default. The only user-visible change for single-repo usage is a small improvement to the “Starting/Finished” banners (they now include the repo label).

New flags

--repos: comma-separated list of repositories to scan (e.g., --repos=owner1/repo1,github.com/owner2/repo2).
--org: GitHub organization handle (e.g., --org=github.com/ossf or ossf).
--combined: after scanning N repos, print a single combined table with all checks together with two new extra columns, REPO and AGGREGATED SCORE.

Precedence when multiple inputs are provided: --repos ➜ --org ➜ --local ➜ --repo (or repo resolved from package managers).

UX / output changes

Before (single repo):

Starting [License]
Starting [Code-Review]
...
Finished [License]
Finished [Code-Review]
...

Now (single or multi-repo):

Starting (owner/repo) [License]
Starting (owner/repo) [Code-Review]
...
Finished (owner/repo) [License]
Finished (owner/repo) [Code-Review]
...

This makes it obvious which repo each check belongs to—especially important when scanning many repos.
No other output changes occur unless --combined is used.

When --combined is set, a final COMBINED RESULTS section is emitted after all per-repo results, e.g.:

COMBINED RESULTS
----------------
| REPO                                             | AGGREGATED SCORE | SCORE   | NAME                 | REASON                      | DOCUMENTATION/REMEDIATION                                |
|--------------------------------------------------|------------------|---------|----------------------|-----------------------------|----------------------------------------------------------|
| github.com/ossf-tests/scorecard-check-...        | 3.3 / 10         | 10 / 10 | Binary-Artifacts     | no binaries found in repo   | https://github.com/ossf/scorecard/blob/main/docs/...     |
...

Implementation details

Introduce buildRepoURLs(ctx, *options.Options) ([]string, error):
Normalizes the input universe and always returns a slice of repo URIs.
Honors precedence --repos ➜ --org ➜ --local ➜ single --repo/pkg-manager.
Refactor rootCmd to iterate over the returned list and run the same scan pipeline for each repo.
ListOrgRepos for listing repositories from the organization, reusing the githubrepo code and logic.

Examples

Scan a list

scorecard --repos=ossf/scorecard,ossf-tests/scorecard-check-branch-protection-e2e

Scan all non-archived repos in an org

scorecard --org=github.com/ossf

Aggregate across many repos

scorecard --repos=ossf/scorecard,ossf-tests/scorecard-check-branch-protection-e2e --combined

PR title follows the guidelines defined in our pull request documentation
Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

Fixes #4792

Special notes for your reviewer

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)

codecov · 2025-09-17T05:35:24Z

Codecov Report

❌ Patch coverage is 31.64179% with 229 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.48%. Comparing base (353ed60) to head (201ac1d).
⚠️ Report is 238 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4793      +/-   ##
==========================================
+ Coverage   66.80%   67.48%   +0.68%     
==========================================
  Files         230      250      +20     
  Lines       16602    19367    +2765     
==========================================
+ Hits        11091    13070    +1979     
- Misses       4808     5422     +614     
- Partials      703      875     +172

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

spencerschrock

Did a partial review of a few things before I ran out of time, I need to dive more into how you build the repo list, scan repos, and present the results. Feel free to address that feedback now or wait until I have more time for full review

spencerschrock · 2025-09-18T15:19:09Z

clients/githubrepo/roundtripper/roundtripper.go

+// TransportFactory is used to create an http.RoundTripper. It defaults to
+// calling NewTransport but can be overridden in tests to provide a custom
+// RoundTripper that redirects requests to a test server.
+var TransportFactory = NewTransport


I dont think we need this. Most mocking in this repo is done with dependency injection, so whatever uses the round tripper should have one injected in a test.

E.g.

scorecard/clients/githubrepo/webhook_test.go

Lines 28 to 42 in 2864c76

type stubTripper struct {

responsePath string

}

func (s stubTripper) RoundTrip(_ *http.Request) (*http.Response, error) {

f, err := os.Open(s.responsePath)

if err != nil {

return nil, err

}

return &http.Response{

Status: "200 OK",

StatusCode: http.StatusOK,

Body: f,

}, nil

}

scorecard/clients/gitlabrepo/search_test.go

Lines 172 to 176 in 2864c76

httpClient := &http.Client{

Transport: stubTripper{

responsePath: tt.responsePath,

},

}

And then when you make a client you can still call this by default, but if the test gives you a round tripper you use it instead.

scorecard/clients/githubrepo/client.go

Lines 347 to 359 in 44f521d

func NewRepoClient(ctx context.Context, opts ...Option) (clients.RepoClient, error) {

var config repoClientConfig

for _, option := range opts {

if err := option(&config); err != nil {

return nil, err

}

}

if config.rt == nil {

logger := log.NewLogger(log.DefaultLevel)

config.rt = roundtripper.NewTransport(ctx, logger)

}

This was also changed; still not sure if my implementation is the expected one.

spencerschrock · 2025-09-18T15:22:37Z

go.mod

 	github.com/google/go-github/v53 v53.2.0
+	github.com/google/go-github/v60 v60.0.0


why introduce a second version when the function you need is available in the version we already use?
https://pkg.go.dev/github.com/google/go-github/v53/github#RepositoriesService.ListByOrg

Right. Changed. ✅

spencerschrock · 2025-09-18T15:31:41Z

clients/githubrepo/org.go

+// ListOrgRepos lists all non-archived repositories for a GitHub organization.
+func ListOrgRepos(ctx context.Context, org string) ([]string, error) {


can we move org.go and org_test.go to cmd/internal/org/org.go and cmd/internal/org/org_test.go.

Any code we can keep out of the public is good.

I'm not quite sure I understand this. But if you think that's the correct place for it, I'll move it... My logic was to keep all related github code together....

spencerschrock · 2025-09-18T15:36:10Z

cmd/internal/org/org.go

+// parseOrgName extracts the organization name from a GitHub URL or returns the input if already an org name.
+func parseOrgName(input string) string {
+	// Remove "github.com/" prefix if present
+	const prefix = "github.com/"


other Scorecard arguments accept URLs with schmes. So I would want you to test this works with something like:

http://github.com/gabrielsoltz https://github.com/gabrielsoltz

Improved and added test cases ✅

spencerschrock · 2025-09-18T15:39:25Z

cmd/root.go

 // See the License for the specific language governing permissions and
 // limitations under the License.

-// Package cmd implements Scorecard command-line.


please dont delete comments unrelated to your change.

restored ✅

spencerschrock · 2025-09-18T15:41:01Z

cmd/root.go

+	// Shared setup (unchanged)
+	pol, err := policy.ParseFromFile(o.PolicyFile)


unchanged from what?

Based on the previous behavior before my change, I'll remove that comment now because I see it's confusing.

Deleted that part of the comment ✅

spencerschrock · 2025-09-18T15:41:10Z

cmd/root.go


-	// Read docs.
 	checkDocs, err := docs.Read()
 	if err != nil {
 		return fmt.Errorf("cannot read yaml file: %w", err)
 	}

 	var requiredRequestTypes []checker.RequestType
-	// if local option not set add file based
 	if o.Local != "" {
 		requiredRequestTypes = append(requiredRequestTypes, checker.FileBased)
 	}
-	// if commit option set to anything other than HEAD add commit based
 	if !strings.EqualFold(o.Commit, clients.HeadSHA) {
 		requiredRequestTypes = append(requiredRequestTypes, checker.CommitBased)
 	}
-	// this call to policy is different from the one in scorecard.Run
-	// this one is concerned with a policy file, while the scorecard.Run call is
-	// more concerned with the supported request types
+
 	enabledChecks, err := policy.GetEnabled(pol, o.Checks(), requiredRequestTypes)


please dont delete comments unrelated to your change.

Sorry for this, I restored them ✅

spencerschrock · 2025-09-18T15:48:29Z

Also, this repo requires DCO.

scorecard/CONTRIBUTING.md

Lines 10 to 14 in 2864c76

    
           > Additionally the Linux Foundation (LF) requires all contributions include per-commit sign-offs. 
        
           > Ensure you use the `-s` or `--signoff` flag for every commit. 
        
           > 
        
           > For more details, see the [LF DCO wiki](https://wiki.linuxfoundation.org/dco) 
        
           > or [this Pi-hole signoff guide](https://docs.pi-hole.net/guides/github/how-to-signoff/).

For instructions on how to fix it:
https://github.com/ossf/scorecard/pull/4793/checks?check_run_id=50569515731

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

spencerschrock · 2025-09-29T22:12:18Z

Just wanted to update you that this is still on my radar! We are trying to cut a 5.3.0 release this week, so my attention has been on that. But once that's cut this week, I will make time to finish. my review, and happy to cut a 5.4.0 or 5.3.1 shortly after

github-project-automation bot added this to OpenSSF Scorecard Sep 16, 2025

gabrielsoltz changed the title ~~✨ Implement --org and --repos for scanning organizations or multiple repositories~~ ✨ Add multi-repo scanning: --repos, --org, and optional --combined output Sep 16, 2025

gabrielsoltz marked this pull request as ready for review September 16, 2025 21:20

gabrielsoltz requested a review from a team as a code owner September 16, 2025 21:20

gabrielsoltz requested review from spencerschrock and raghavkaul and removed request for a team September 16, 2025 21:20

gabrielsoltz temporarily deployed to gitlab September 17, 2025 05:30 — with GitHub Actions Inactive

gabrielsoltz temporarily deployed to integration-test September 17, 2025 05:30 — with GitHub Actions Inactive

gabrielsoltz temporarily deployed to gitlab September 18, 2025 04:43 — with GitHub Actions Inactive

gabrielsoltz temporarily deployed to integration-test September 18, 2025 04:43 — with GitHub Actions Inactive

spencerschrock reviewed Sep 18, 2025

View reviewed changes

gabrielsoltz added 9 commits September 19, 2025 10:57

First Iteration

7fa7e9f

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

switch-to-client

08010be

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

implement-outputs

4156f6e

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

minor-improvements

8347087

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

tests-rg

6a486b4

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

Fix linter issues

f5aab58

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

add-license

d772403

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

add-license

f1554a1

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

fix linter

75c4984

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

gabrielsoltz force-pushed the feat-support-org-and-repos branch from 3ab90d2 to 75c4984 Compare September 19, 2025 08:57

gabrielsoltz added 5 commits September 19, 2025 11:10

Restore deleted comments

1407a02

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

Restore comments

e2755cb

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

Move org code to cmd/internal and reuse go-github v53

4d80fc9

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

ParseOrgName support for URLs

f496a80

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

Better Mocking

201ac1d

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

gabrielsoltz temporarily deployed to gitlab September 20, 2025 03:08 — with GitHub Actions Inactive

gabrielsoltz temporarily deployed to integration-test September 20, 2025 03:09 — with GitHub Actions Inactive

justaugustus mentioned this pull request Sep 20, 2025

✨ multi-scorecard experiment #4502

Closed

2 tasks

justaugustus moved this to In Progress in OpenSSF Scorecard Sep 20, 2025

justaugustus moved this from In Progress to Review in progress in OpenSSF Scorecard Sep 20, 2025

add-license

2016fc7

Signed-off-by: Gabriel Alejandro Soltz <[email protected]>

	type stubTripper struct {
	responsePath string
	}

	func (s stubTripper) RoundTrip(_ http.Request) (http.Response, error) {
	f, err := os.Open(s.responsePath)
	if err != nil {
	return nil, err
	}
	return &http.Response{
	Status: "200 OK",
	StatusCode: http.StatusOK,
	Body: f,
	}, nil
	}

	httpClient := &http.Client{
	Transport: stubTripper{
	responsePath: tt.responsePath,
	},
	}

	func NewRepoClient(ctx context.Context, opts ...Option) (clients.RepoClient, error) {
	var config repoClientConfig

	for _, option := range opts {
	if err := option(&config); err != nil {
	return nil, err
	}
	}

	if config.rt == nil {
	logger := log.NewLogger(log.DefaultLevel)
	config.rt = roundtripper.NewTransport(ctx, logger)
	}

		github.com/google/go-github/v53 v53.2.0
		github.com/google/go-github/v60 v60.0.0

		// ListOrgRepos lists all non-archived repositories for a GitHub organization.
		func ListOrgRepos(ctx context.Context, org string) ([]string, error) {

		// Shared setup (unchanged)
		pol, err := policy.ParseFromFile(o.PolicyFile)

✨ Add multi-repo scanning: --repos, --org, and optional --combined output #4793

Are you sure you want to change the base?

✨ Add multi-repo scanning: --repos, --org, and optional --combined output #4793

Uh oh!

Conversation

gabrielsoltz commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What kind of change does this PR introduce?

New flags

UX / output changes

Before (single repo):

Now (single or multi-repo):

Implementation details

Examples

Scan a list

Scan all non-archived repos in an org

Aggregate across many repos

Which issue(s) this PR fixes

Special notes for your reviewer

Does this PR introduce a user-facing change?

Uh oh!

codecov bot commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

spencerschrock left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gabrielsoltz Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

spencerschrock commented Sep 18, 2025

Uh oh!

spencerschrock commented Sep 29, 2025

Uh oh!

Uh oh!

gabrielsoltz commented Sep 16, 2025 •

edited

Loading

codecov bot commented Sep 17, 2025 •

edited

Loading

gabrielsoltz Sep 19, 2025 •

edited

Loading