Node-based Cloud Run Service Template

• What This Repository Is
• What This Repository Is Not
• Who Should Not Use This Repository
• How To Use This Repository
• Step-by-Step Instructions
  • 1. Prerequisites
  • 2. Cloning the repo
  • 3. Bootstrapping Cloud Run
  • 4. Deploying the Initial Service
• Troubleshooting Steps

What This Repository Is

The purpose of this repo is to provide a simple template repository that will allow you to deploy a Node.js-based service with minimal effort. It has a few different features:

1. GCP Project Bootstrap Script - Executes the GCP commands required to set up a new project and outputs the deployment config file used by the GitHub Workflows to actually build and deploy the service.
2. Minimal "functioning" expressjs service with some default project configuration (eslint/prettier/vitest/lint-staged/husky).
3. GitHub workflows that will do some standard CI/CD stuff like running the lint check and tests, building and pushing the Docker image, and deploying the Docker image to Cloud Run.

What This Repository Is Not

This repository is not meant to:

1. Be highly configurable
2. Produce a secure, production-ready service or deployment
3. Be an example of best practices

Who Should Not Use This Repository

If you are new to Node.js and aren't ready to deal with the hurdles that come with linting and testing requirements, using this template will probably be more frustrating than helpful. It has a non-minimal, opinionated configuration, so unless it just happens to behave exactly as you expect, you're probably going to end up spending a non-negligible amount of time trying to figure out how it's set up and why it's behaving the way it behaves.

How To Use This Repository

There are two primary use cases for this repository:

1. As an initial template that you use to create your own template that's customized to your liking; and
2. As a template that you use directly.

In the first scenario, you're most likely using the repository for the bootstrapping and deployment workflows and will have to spend some time reconfiguring the project itself to work like you want it (e.g., switch it to TypeScript, switch tests to Jest, change the linting rules, etc.). If you fall in this category, I will assume you know what you're doing and look through the repository and figure out what's going on. In this case, you'll probably just want to read the section on Bootstrapping Cloud Run and skip the rest.

In the second scenario, you probably want to know how the repository is designed to be used, in which case you should read the Walkthrough as well, where I explain the general workflow that I use. This is not meant as instructions on how you should use it, but merely giving you an idea of how I use it so you can make whatever adjustments you want to suit your own workflow.

Step-by-Step Instructions

IMPORTANT: The GCP project requires billing to be enabled so you will incur normal GCP-related charges. For the most part, this is limited to Artifact Registry and Cloud Run costs, which should be fairly low for a sample project. I am not responsible for any charges you incur.

1. Prerequisites

• Everything needed to work on a Node.js project in the first place and enough familiarity that you can look through the repository and figure out what's going on by default.
• Docker installed and setup (for local e2e tests)
• gcloud CLI installed (see here for instructions)
• Google application default credentials configured (see here for instructions)
• A GCP billing account configured (see here for instructions)
• Some general working knowledge of GCP, serverless, Cloud Run, and GitHub workflows is useful.

2. Cloning the repo

1. Create a new repository using this one as the template.
2. Clone the new repository to your local machine or wherever you want to do development.

3. Bootstrapping Cloud Run

During this process, we will set the configuration we need to bootstrap our Cloud Run service on GCP and then actually perform the bootstrapping. Although the process seems somewhat long, it should only take about ten minutes, and half of that is waiting for the bootstrapping script to set everything up!

1. Checkout a new branch (e.g., chore/init).
2. Run npm install then add and commit the package-lock.json file.
3. Open gcp-config.sh and
   1. Replace the empty string assigned to the GITHUB_REPO_PATH variable with your GitHub repository path (e.g., username/new-repo-name).
   2. Replace the empty string assigned to the SERVICE_NAME variable with your desired service name.
   3. Replace the empty string assigned to the PROJECT_NAME variable with your desired project name.
   4. Replace the empty string assigned to the BILLING_ACCOUNT_ID variable with your billing account id (e.g., 3215754-215487-659845).
   5. Review the other options and update as desired.
   6. Note that these values must conform to GCP requirements; no validation is performed so if they do not then the bootstrapping process may fail in the middle.
4. Navigate to your project directory in the terminal.
5. Run ./bootstrap.sh to create and configure your new project. This should take a few minutes. At least one command (the creation of the workload identity federation pool) is expected to fail and re-run several times after a waiting period. It should succeed after a minute or so.
6. If the script completes successfully, add and commit the newly generated deployment/config.yml file to your remote repository. If it did not, see the troubleshooting steps below.
7. Remove your billing account id from your gcp-config.sh file and then add and commit this file. (I do not know if there's any harm in leaving the billing account id in there, so maybe it's unnecessary to remove it.)
8. Push the latest commits to your GitHub repository.

4. Deploying the Initial Service

The previous step deployed a hello-world container that Google provides to the service, not the actual service in this repository. Thus, our next step is to actually build and deploy our own service.

1. Create a PR from the branch you pushed in the previous steps.
2. Wait for the workflows to complete. If there are any errors, fix them and then continue.
3. Merge your PR using whatever merge strategy you prefer.
4. Click on the Actions tab and you should see your Trigger Build and Deployment action running (although the actual name will depend on merge strategy you chose).
5. Click into the running workflow to see the status of the individual jobs.

If all goes well, both the Build and Push Docker Image job and the Deploy Cloud Run job will run successfully and you should be able to send a GET request to your service and get a JSON response of {"hello": "world"}. To figure out the URL for your newly deployed service, go to Cloud Run in the GCP console, click into your service, and the URL should be displayed towards the top.

Troubleshooting Steps

There are probably a few things that could go wrong during this process, but here are some details of the two most likely:

1. The bootstrap.sh script failed to complete - This could be any number of problems from invalid config values in gcp-config.sh to permissions issues. Generally, however, whatever fails will give you an indication of what the problem, but it's probably going to require some research to figure out what the root cause is. The bigger problem is that bootstrap.sh does not supprt re-trying in the case of failure, so you'll have a project that's not fully set up and you can't really do anything except (1) just delete the project and try again or (2) comment out the commands in bootstrap.sh that have run and then re-run it again. The latter actually works really well but generally requires understanding what's going on in the bootstrap.sh script. It's not complex but does require some time!

2. The build and deployment jobs failed - This could be for any number of reasons. Again, the error messages should be helpful. The most common problem will likely be that you didn't correctly specify the GITHUB_REPO_PATH env var. This should be the same basic structure you see in the URL bar on github.com for your repo. E.g., if my username is mstrofbass and my repo name is cloud-run-test, then the path will be mstrofbass/cloud-run-test, just like in the GitHub URL.

To fix this problem, we have to fix the Workload Identity Federation configuration:

1. Go to the GCP Cloud Console > IAM & Admin > Workload Identity Federation
2. You should see a table with your github pool listed. Click the link under Display Name to go to the pool.
3. When your pool opens up, you should see a section on the right side that has PROVIDERS and CONNECTED SERVICE ACCOUNTS tabs. PROVIDERS should already be selected and should have a table with your github provider in it. Click the pencil icon in that row.
4. After the provider page loads, scroll all the way down to where it says Attribute Conditions. It should have a value that looks like assertion.repository_visibility == "private" && assertion.repository in ["GITHUB_REPO_PATH"] where GITHUB_REPO_PATH is what you had in your gcp-config.sh file.
5. Update the value in the GITHUB_REPO_PATH spot to be the correct path to your GitHub repository.
6. Go back to your GitHub actions and retrigger the Trigger Build and Deployment action.