Skip to content

Latest commit

 

History

History
74 lines (48 loc) · 3.25 KB

README.md

File metadata and controls

74 lines (48 loc) · 3.25 KB
Raw

Overview

Agora Data Manager is a tool that loads the JSON files into Agora's document database instances in our AWS environments.

Purpose

This project allows Agora maintainers to update the Agora database with new versions of gene data from Synapse. This is a manually triggered, self-service update.

Execution

alt text

Workflow

To deploy an updated data version to the Agora development database

  1. Increment data-version in data-manifest.json on the develop branch.
  2. Commit the change
  3. The Github action CI system automatically updates the dev DB

To deploy an updated data version to the Agora staging database:

  1. Merge the data-version update from the dev branch to the staging branch.
  2. The Github action CI system automatically updates the dev DB

To deploy an updated data version to the Agora production database:

  1. Merge the data-version update from the staging branch to the production branch.
  2. The Github action CI system automatically updates the dev DB

Setup

Secrets

The following secrets need to be setup in Github for the scripts to deploy database updates:

Global secrets:

Variable Description Example
SYNAPSE_USERNAME The Synapse service user syn-service-user
SYNAPSE_PASSWORD The Synapse service user password supersecret

Context specific secrets for each environment that corresponds to a git branch (develop/staging/prod):

Variable Description Example
DB_HOST The database host dbcluster-mr0a782pfjnk.cluster-ctcayu3de2lt.us-east-1.docdb.amazonaws.com
DB_USER The database user dbuser
DB_PASS The database password supersecret

alt text

Self hosted runners

agora2-infra repository deploys a bastian host in AWS for each environment which have access to the databases. We manually configure a [Github self-hosted runner] for each bastian host, a label is applied to each runner to match the corresponding deployment branch name (develop/staging/prod). Each runner corresponds to an environment which corresponds to a git branch. The update is executed from these runners. When a push happens on a branch (i.e. develop), the update is executed on the agora-bastian-develop runner which in turn updates the development database.

alt text