Terra helper is a collection of scripts which leverage the Terra API and gsutil to perform cumbersome tasks on Terra. A more appropriate name for this repository may be Terra storage helper, as the scripts pretty much just deal with cleaning up workspaces or moving underlying buckets. The python package fiss also has a function called mop which tries to clean up intermediate files.
Navigate to the terra-helper folder for individual scripts and corresponding documentation. For use cases, refer to the docs folder. Current use cases documented include,
To use terra storage helper, you must have the following set up on your system
- Google Cloud SDK
- Python 3
Google Cloud and Terra use Google Cloud SDK to manage authentication. The application default credentials will be used. Please walk through Google's wonderful install and set up documentation if you have not. To login and set an application default, make sure that you've run the following,
gcloud auth login
gcloud auth application-default loginYou may need to export your google cloud auth token, add this to your bash profile:
export GCS_OAUTH_TOKEN=`gcloud auth application-default print-access-token`This package can be downloaded through Github on the website or by using terminal. To download on the website, navigate to the top of this page, click the green Clone or download button, and select Download ZIP. This will download this repository in a compressed format. To install using Github on terminal, type
git clone https://github.com/vanallenlab/terra-helper.git
cd terra-helperTerra helper uses Python 3.10. We recommend using a virtual environment and running Python with either Anaconda or Miniconda.
To create a virtual environment and install dependencies with Anaconda or Miniconda, run the following from this repository's directory:
conda create -y -n terra-helper python=3.10
conda activate terra-helper
pip install -r requirements.txtIf you are using base Python, you can create a virtual environment and install dependencies by running:
virtualenv terra-helper
source activate terra-helper/bin/activate
pip install -r requirements.txt