Skip to content

Concurrently clones remote repositories based on a JSON file

License

Notifications You must be signed in to change notification settings

costa86/repo-downloader

Repository files navigation

Repo Cloner

MIT License

Downloads remote repositories (GitHub, GitLab, BitBucket) based on a JSON file.

Providing the repositories

The program takes a JSON file that defines the repositories to be cloned. Use repos.json as a template.

⚠️ Valid repositories must start with "http", "https", "git" or "ssh", and end with ".git".

Here's a template example:

[
    {
        "folder": "autogit",
        "url": "https://github.com/costa86/autogit.git"
    },
    {
        "folder": "coppy",
        "url": "https://github.com/costa86/coppy.git"
    }
]

Installation

No installation required. Just run the executable providing the required arguments.

OS Executable
Windows repo-download-windows.exe
Linux repo-download-linux
MacOS repo-download-darwin

Usage

Run it with -help

./repo-download-linux -help

Output

Usage of ./repo-download-linux:
-file string
        JSON file with repositories (default "repos.json")
-password string
        password (case authentication is required)
-username string
        username (case authentication is required)

Option 1: file

This is an option when no credentials are required

./repo-download-linux -file <json_file>

Option 2: file + username

In this case, the user will be prompted for a password (***)

./repo-download-linux -file <json_file> -username <username>

Option 3: file + username + password

This option is more suitable for CI/CD pipelines, where the password can be passed via some mechanism that manages sensitive variables, such as Jenkins.

./repo-download-linux -file <json_file> -usename <username> -password <password>

It clones each repository into the specified local folder.

Concurrent cloning

The program was built in Go, which takes advantage of the Goroutines to clone repositories concurrently. This maximizes throughput by cloning them simultaneously.

Build from source

Will generate the executables for Windows, Linux and MacOS

Requirements:

  • Go (check version in go.mod)
  • Make

Command:

make build

Links

portfolio linkedin