feat: enable private GitHub/GitLab repository templates (#1157)

[Kartikayy007]

Description

This PR adds support for using AsyncAPI templates from private GitHub and GitLab repositories, resolving a long-standing feature request.

Changes

• Added isGitSpecifier() helper to detect git-based template URLs
• Modified installTemplate() to skip cache lookup for git specs and delegate authentication to system git configuration
• Added comprehensive test coverage (9 new tests for git URL detection)
• Updated documentation with examples for private git repository usage

How it works

The implementation leverages npm's Arborist (already used for template installation), which natively supports git URLs. Authentication is handled by the system's existing git configuration (SSH keys, HTTPS tokens, .netrc).

Supported formats:

• git+ssh://[email protected]/org/repo.git
• git+https://github.com/org/repo.git
• [email protected]:org/repo.git
• github:org/repo
• gitlab:org/repo

Usage

CLI:

asyncapi generate fromTemplate asyncapi.yaml \
  git+ssh://[email protected]/org/private-template.git \
  -o output

Library:

const generator = new Generator(
  'git+ssh://[email protected]/org/private-template.git',
  'output'
);

Demo

demo-private-git-compressed.mp4

Testing

• All tests passing:

• 120 unit tests (including 9 new tests for git URL detection)
• 10 integration tests
• Total: 130/130 passing

• Manual verification:

• Tested with actual private GitHub repository
• Verified multiple git URL formats work correctly
• Confirmed authentication delegates to system git config

Related issue(s)

Resolves #1157

Summary by CodeRabbit

• New Features

  • Install templates directly from Git specifiers (including private repos) with proper git-auth handling and clearer git-source logging.

• Documentation

  • Added a comprehensive guide for using templates from private Git repositories with CLI examples and authentication guidance.

• Tests

  • Added tests covering Git specifier detection and installation flows.

[changeset-bot]

⚠️ No Changeset found

Latest commit: de9609b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[coderabbitai]

Walkthrough

Adds support for templates sourced from Git specifiers by detecting git-style template identifiers, bypassing local cache for git templates, adding tests and mocks, and documenting usage for private Git repositories.

Changes

Cohort / File(s)	Summary
Documentation apps/generator/docs/using-private-template.md	New doc describing using templates from private Git repositories, CLI examples with various git specifiers, authentication guidance, and a library usage example.
Git specifier utility apps/generator/lib/utils.js, apps/generator/lib/__mocks__/utils.js	Adds isGitSpecifier(str) in production and mock; mock exposes __isGitSpecifierValue override. Detects git+ssh/git+https/ssh/git@host:path and github:/gitlab:/bitbucket: npm-style specifiers.
Generator logic apps/generator/lib/generator.js	Detects git templates in installTemplate, skips local cache lookup for git specifiers, and emits debug logging when installing from git sources.
Test mocks apps/generator/test/__mocks__/requireg.js	Adds mock requireg.resolve() returning 'npm/index.js' to satisfy test expectations.
Tests apps/generator/test/generator.test.js, apps/generator/test/utils.test.js	Adds test for installing from a git specifier and comprehensive tests for isGitSpecifier covering positive and negative git formats.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• Pay attention to regex/validation correctness in isGitSpecifier (edge cases, refs like #branch/#tag).
• Verify the cache-bypass condition in installTemplate (!force && !isGitTemplate) is correctly updated to !isGitTemplate behavior.
• Confirm tests and mocks (especially requireg mock and mock override flag) align with test runner expectations.

Possibly related PRs

• #1329 — Modifies installTemplate (error messages/logging) in the same function; review for logging or behavior overlap/conflicts.

Pre-merge checks and finishing touches

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main feature: enabling private GitHub/GitLab repository templates for the AsyncAPI generator.
Linked Issues check	✅ Passed	The PR successfully implements the core requirement from issue #1157: adding support for private GitHub/GitLab repository templates. New isGitSpecifier() utility detects git URLs, installTemplate() skips cache for git specs, and tests confirm functionality.
Out of Scope Changes check	✅ Passed	All changes are directly aligned with issue #1157: documentation for private git templates, git specifier detection utility, modified installation logic, test cases, and mock modules for testing.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[asyncapi-bot]

What reviewer looks at during PR review

The following are ideal points maintainers look for during review. Reviewing these points yourself beforehand can help streamline the review process and reduce time to merge.

1. PR Title: Use a concise title that follows our Conventional Commits guidelines and clearly summarizes the change using imperative mood (it means spoken or written as if giving a command or instruction, like "add new helper for listing operations")

   Note - In Generator, prepend feat: or fix: in PR title only when PATCH/MINOR release must be triggered.

2. PR Description: Clearly explain the issue being solved, summarize the changes made, and mention the related issue.

   Note - In Generator, we use Maintainers Work board to track progress. Ensure the PR Description includes Resolves #<issue-number> or Fixes #<issue-number> this will automatically close the linked issue when the PR is merged and helps automate the maintainers workflow.

3. Documentation: Update the relevant Generator documentation to accurately reflect the changes introduced in the PR, ensuring users and contributors have up-to-date guidance.

4. Comments and JSDoc: Write clear and consistent JSDoc comments for functions, including parameter types, return values, and error conditions, so others can easily understand and use the code.

5. DRY Code: Ensure the code follows the Don't Repeat Yourself principle. Look out for duplicate logic that can be reused.

6. Test Coverage: Ensure the new code is well-tested with meaningful test cases that pass consistently and cover all relevant edge cases.

7. Commit History: Contributors should avoid force-pushing as much as possible. It makes it harder to track incremental changes and review the latest updates.

8. Template Design Principles Alignment: While reviewing template-related changes in the packages/ directory, ensure they align with the Assumptions and Principles. If any principle feels outdated or no longer applicable, start a discussion these principles are meant to evolve with the project.

9. Reduce Scope When Needed: If an issue or PR feels too large or complex, consider splitting it and creating follow-up issues. Smaller, focused PRs are easier to review and merge.

10. Bot Comments: As reviewers, check that contributors have appropriately addressed comments or suggestions made by automated bots. If there are bot comments the reviewer disagrees with, react to them or mark them as resolved, so the review history remains clear and accurate.

[asyncapi-bot]

🚀 Docs preview deployed
Below link points directly to the generator docs preview. May the force be with you!
→ https://691dbf3bea74840c154b8324--asyncapi-website.netlify.app/docs/tools/generator

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

apps/generator/lib/utils.js (1)

107-141: Add support for git:// protocol and bitbucket: shorthand in git specifier detection.

npm supports git:// protocol (e.g., git://host/user/repo.git#commit-ish) and host shorthands like bitbucket:user/repo. The current implementation is missing these formats. Consider adding checks for git:// and bitbucket: prefixes to ensure complete coverage of npm-supported git specifiers.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 77bf8b5 and 5c9d1a2.

📒 Files selected for processing (7)

• apps/generator/docs/using-private-template.md (1 hunks)
• apps/generator/lib/__mocks__/utils.js (1 hunks)
• apps/generator/lib/generator.js (4 hunks)
• apps/generator/lib/utils.js (1 hunks)
• apps/generator/test/__mocks__/requireg.js (1 hunks)
• apps/generator/test/generator.test.js (1 hunks)
• apps/generator/test/utils.test.js (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (3)

apps/generator/test/generator.test.js (5)

apps/generator/lib/generator.js (2)

• log (7-7)
• logMessage (51-51)

apps/generator/lib/utils.js (3)

• log (9-9)
• utils (15-15)
• logMessage (13-13)

apps/generator/test/utils.test.js (4)

• log (4-4)
• utils (5-5)
• Generator (3-3)
• logMessage (7-7)

apps/generator/test/__mocks__/loglevel.js (1)

• log (1-1)

apps/generator/lib/__mocks__/utils.js (1)

• utils (1-1)

apps/generator/lib/utils.js (2)

apps/generator/lib/__mocks__/utils.js (2)

• utils (1-1)
• trimmed (31-31)

apps/generator/test/utils.test.js (1)

• utils (5-5)

apps/generator/lib/__mocks__/utils.js (2)

apps/generator/lib/utils.js (2)

• utils (15-15)
• trimmed (123-123)

apps/generator/test/utils.test.js (1)

• utils (5-5)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)

• GitHub Check: Test generator as dependency with Node 20
• GitHub Check: Test generator as dependency with Node 18
• GitHub Check: Acceptance tests for generated templates
• GitHub Check: Test NodeJS PR - macos-latest
• GitHub Check: Test NodeJS PR - windows-latest
• GitHub Check: preview

🔇 Additional comments (6)

apps/generator/docs/using-private-template.md (1)

42-80: LGTM! Clear documentation for Git-based templates.

The new section provides clear guidance on using templates from private Git repositories. The examples cover common git specifier formats, and the authentication guidance appropriately delegates to system git configuration.

apps/generator/test/__mocks__/requireg.js (1)

1-10: LGTM! Mock is correctly implemented.

The mock returns a string containing 'index.js' to support the string replacement in generator.js line 11. The implementation is simple and appropriate for test purposes.

apps/generator/test/generator.test.js (1)

431-451: LGTM! Test correctly validates git specifier flow.

The test appropriately validates that git specifiers trigger installation from disk and log the expected git-specific message. The TODO comment acknowledges the known Arborist reification testing limitation.

apps/generator/lib/generator.js (2)

574-603: LGTM! Git specifier handling is correctly implemented.

The changes appropriately:

1. Detect git specifiers using the new utility function
2. Bypass local cache lookup for git-based templates (line 576)
3. Provide clear debug logging for git template installation

The logic ensures git URLs are always fetched fresh, which is the correct behavior since they can't be resolved through the existing local cache mechanism.

---

1121-1121: Verify whether this export is intentional.

This line exports listBakedInTemplates, which doesn't appear to be directly related to the git specifier feature. Please confirm whether this export was intentionally included in this PR or should be part of a separate change.

apps/generator/lib/__mocks__/utils.js (1)

25-43: LGTM! Mock implementation provides test flexibility.

The mock implementation duplicates the production logic while providing an override mechanism via __isGitSpecifierValue. This duplication is acceptable for test mocks and ensures consistent behavior during testing. The override flag allows tests to control the return value when needed.

Note that changes to the production isGitSpecifier function should be synchronized with this mock implementation.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

apps/generator/lib/__mocks__/utils.js (1)

25-43: Consider reducing duplication between mock and real implementation.

The mock includes a fallback implementation (lines 30-42) that duplicates the git specifier detection logic from the real utils.js module. While this pattern is consistent with other mocks in this file and allows tests to work without always setting override values, it does create a maintenance burden—if the real implementation evolves, the mock must be kept in sync.

For future consideration, you could:

• Simplify the mock to rely primarily on __isGitSpecifierValue overrides in tests, OR
• Import and delegate to the real implementation when __isGitSpecifierValue is false (though Jest mocking may complicate this)

Given the established pattern in this codebase and the relatively stable nature of git specifier detection, this approach is acceptable for now.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5c9d1a2 and de9609b.

📒 Files selected for processing (4)

• apps/generator/docs/using-private-template.md (1 hunks)
• apps/generator/lib/__mocks__/utils.js (1 hunks)
• apps/generator/lib/utils.js (1 hunks)
• apps/generator/test/utils.test.js (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (3)

• apps/generator/test/utils.test.js
• apps/generator/lib/utils.js
• apps/generator/docs/using-private-template.md

🧰 Additional context used

🧬 Code graph analysis (1)

apps/generator/lib/__mocks__/utils.js (2)

apps/generator/lib/utils.js (2)

• utils (15-15)
• trimmed (123-123)

apps/generator/test/utils.test.js (1)

• utils (5-5)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)

• GitHub Check: Test generator as dependency with Node 20
• GitHub Check: Test generator as dependency with Node 18
• GitHub Check: Acceptance tests for generated templates
• GitHub Check: Test NodeJS PR - windows-latest
• GitHub Check: Test NodeJS PR - macos-latest
• GitHub Check: Test NodeJS PR - ubuntu-latest
• GitHub Check: preview

[Kartikayy007]

adding it to draft as unplanned