- General Workflow Information
- Utilizing the Workflow
The current GeneLab Reference Annotation Table (GL_RefAnnotTable-A) pipeline is implemented as an R workflow that can be run from a command line interface (CLI) using bash. The workflow can be executed using either a Singularity container or a local R environment. The workflow can be used even if you are unfamiliar with R, but if you want to learn more about R, visit the R-project about page here. Additionally, an introduction to R along with installation help and information about using R for bioinformatics can be found here at Happy Belly Bioinformatics.
Download the latest version of the GL_RefAnnotTable-A workflow:
curl -LO https://github.com/nasa/GeneLab_Data_Processing/releases/download/GL_RefAnnotTable-A_1.1.0/GL_RefAnnotTable-A_1.1.0.zip
unzip GL_RefAnnotTable-A_1.1.0.zipThe GL_RefAnnotTable-A workflow can be run using one of two approaches:
Please follow the instructions for the approach that best matches your setup and preferences. Each method is explained in detail below.
Note: If you encounter timeout errors, you can increase the default timeout (3600 seconds) by modifying the
options(timeout=3600)line at the top of theGL-DPPD-7110-A_build-genome-annots-tab.Rscript.
This approach allows you to run the workflow within a containerized environment, ensuring consistency and reproducibility.
Singularity is a containerization platform for running applications portably and reproducibly. We use container images hosted on Quay.io to encapsulate all the necessary software and dependencies required by the GL_RefAnnotTable-A workflow. This setup allows you to run the workflow without installing any software directly on your system.
Note: Other containerization tools like Docker or Apptainer can also be used to pull and run these images.
We recommend installing Singularity system-wide as per the official Singularity installation documentation.
Note: While Singularity is also available through Anaconda, we recommend installing Singularity system-wide following the official installation documentation.
To pull the Singularity image needed for the workflow, you can use the provided script as directed below or pull the image directly.
Note: This command should be run in the location containing the
GL_RefAnnotTable-A_1.1.0directory that was downloaded in step 1. Depending on your network speed, fetching the images will take approximately 20 minutes.
bash GL_RefAnnotTable-A_1.1.0/bin/prepull_singularity.sh GL_RefAnnotTable-A_1.1.0/config/software/by_docker_image.configOnce complete, a singularity folder containing the Singularity images will be created. Next, set up the required environment variables:
# Set R library path to current working directory
export R_LIBS_USER=$(pwd)/R_libs
# Create the specified R library path if it doesn't already exist
mkdir -p $R_LIBS_USER
# Set Singularity cache directory
export SINGULARITY_CACHEDIR=$(pwd)/singularityNote: The annotation database creation process requires FTP access through port 21. If you encounter connection issues, please verify that port 21 is not blocked by your network/firewall settings or try running the workflow on a system with unrestricted FTP access.
While in the directory containing the GL_RefAnnotTable-A_1.1.0 folder that was downloaded in step 1, you can now run the workflow. Below is an example for generating the annotation table for Mus musculus (mouse):
singularity exec \
--bind $(pwd):$(pwd) \
$SINGULARITY_CACHEDIR/quay.io-nasa_genelab-gl-refannottable-a-1.1.0.img \
Rscript GL_RefAnnotTable-A_1.1.0/GL-DPPD-7110-A_build-genome-annots-tab.R 'Mus musculus'This approach allows you to run the workflow directly in your local R environment without using containers.
We recommend installing R via the Comprehensive R Archive Network (CRAN):
-
Select the CRAN Mirror closest to your location.
-
Navigate to the download page for your operating system.
-
Download and install R (e.g., R-4.4.0).
Once R is installed, install the required R packages as follows:
Open a terminal and start R:
RWithin the R environment, run the following commands to install the required packages:
install.packages("tidyverse")
install.packages("BiocManager")
BiocManager::install("STRINGdb")
BiocManager::install("PANTHER.db")
BiocManager::install("rtracklayer")
BiocManager::install("AnnotationForge")
BiocManager::install("biomaRt")
BiocManager::install("GO.db")While in the directory containing the GL_RefAnnotTable-A_1.1.0 folder that was downloaded in step 1, you can now run the workflow. Below is an example of how to run the workflow to build an annotation table for Mus musculus (mouse):
Rscript GL_RefAnnotTable-A_1.1.0/GL-DPPD-7110-A_build-genome-annots-tab.R 'Mus musculus'The input and output data are the same for both Approach 1: Using Singularity and Approach 2: Using a Local R Environment.
Input data:
-
No input files are required. Specify the species name of the target organism using a positional command line argument.
Mus musculusis used in both the Singularity and the local R environment examples above.Notes:
- To see a list of all available organisms, run
Rscript GL-DPPD-7110-A_build-genome-annots-tab.Rwithout positional arguments. - The correct argument for each organism can also be found in the 'species' column of the GL-DPPD-7110-A_annotations.csv
- To see a list of all available organisms, run
-
Optional: A local reference table CSV file can be supplied as a second positional argument. If not provided, the script will download the current version of the GL-DPPD-7110-A_annotations.csv table by default.
Output data:
-
*-GL-annotations.tsv (Tab delineated table of gene annotations)
-
*-GL-build-info.txt (Text file containing information used to create the annotation table, including tool and tool versions and date of creation)
If the reference table does not specify an annotations database for the target organism in the 'annotations' column of the GL-DPPD-7110-A_annotations.csv file, the install_annotations function (defined in install-org-db.R) will be executed by default. This function can also be run as a stand-alone script:
Note: If you encounter timeout errors, you can increase the default timeout (3600 seconds) by modifying the
options(timeout=3600)line at the top of theinstall-org-db.Rscript.
Note: The annotation database creation process requires FTP access through port 21. If you encounter connection issues, please verify that port 21 is not blocked by your network/firewall settings.
# Set R library path if not already set
export R_LIBS_USER=$(pwd)/R_libs
# Create the specified R library path if it doesn't already exist
mkdir -p $R_LIBS_USER
# Set Singularity cache directory if not already set
export SINGULARITY_CACHEDIR=$(pwd)/singularity
singularity exec \
--bind $(pwd):$(pwd) \
$SINGULARITY_CACHEDIR/quay.io-nasa_genelab-gl-refannottable-a-1.1.0.img \
Rscript GL_RefAnnotTable-A_1.1.0/install-org-db.R 'Bacillus subtilis'Rscript GL_RefAnnotTable-A_1.1.0/install-org-db.R 'Bacillus subtilis'Input data:
-
The species name of the target organism must be specified as the first positional command line argument.
Bacillus subtilisis used in both the Singularity and local R examples above.Note: The correct argument for each organism can also be found in the 'species' column of the GL-DPPD-7110-A_annotations.csv
-
Optional: A local reference table CSV file can be supplied as a second positional argument. If not provided, the script will download the current version of the GL-DPPD-7110-A_annotations.csv table by default.
Output data:
- org.*.eg.db/ (Species-specific annotation database, as a local R package)