Github Search Module For Github Issues

[alper98]

Pull Request: Add GitHub Search Backend Module and Frontend Plugin

Summary

This pull request introduces a new GitHub search backend module to enhance the search capabilities within the Backstage instance. The additions include:

1. GitHub Search Backend Module: Indexes GitHub issues and makes them searchable inside the Backstage instance.

Changes

Backend

• Module: search-backend-module-github
  • Functionality: Indexes GitHub issues and integrates them into the Backstage search engine.
  • Configuration:
    • Environment Variables:
      • search.collators.github.org: Specifies the organization to search within.
      • search.collators.github.query: Defines the query string used to search GitHub issues.
    • GitHub Credentials:
      • Token: Managed through Backstage integrations configuration.
      • App Authentication: Supported via GitHub apps with the necessary configuration.
    • YAML Configuration Example:

      search:
        collators:
          github:
            org: backstage
            host: github.com
            query: is:issue is:open org:backstage

Frontend

The GithubSearchResultListItem component will appear as follows on the search page:

[backstage-goalie]

Missing Changesets

The following package(s) are changed by this PR but do not have a changeset:

• @backstage-community/plugin-github-issues

See CONTRIBUTING.md for more information about how to add changesets.

Changed Packages

Package Name	Package Path	Changeset Bump	Current Version
@backstage-community/plugin-github-issues	workspaces/github-issues/plugins/github-issues	none	v0.4.6
@backstage-community/plugin-search-backend-module-github-issues	workspaces/github-issues/plugins/search-backend-module-github-issues	minor	v0.1.0

[vinzscam]

thanks, this is amazing! 😻

[vinzscam]

if there are 1000 issues, this will cause 1000 issues to be loaded in memory.
We could lift some logic from the fetchGithubIssuesDocuments, making sure that fetchGithubIssuesDocuments returns just a batch of element. Something like:

  let nextPage = ...
  while(hasNext()) {
    const result = this.fetchGithubIssuesDocuments(nextPage);
    nextPage = result.nextPage;
    for (const document of result.documents) {
      yield { ...document, ...stuff };
    }
  }

so that the collator will be more memory efficient.

[alper98]

Good point! Will change

[vinzscam]

we could move createClient to the parent function, so that we instantiate the client only once

[alper98]

Makes sense 🫡

[alper98]

self review: example should be backstage