Skip to content

Using GTG

Jump to bottom
Cameron Goodale edited this page Jan 20, 2016 · 21 revisions

Using GTG

Install all the Python Modules listed in the what is needed section of the Project Home page.

Download the latest release from the Release Page

Grab the release you want in either tar.gz or zip format and uncompress them on your local machine.

Note the following files

config.json: contains the variables for the GTG. The config variables are broken up into two types, simple variables you'd change to adjust the temporal and spatial domains and the DIRS to input files and the advanced variables that will change how the algorithm detects MCCs.

Simple Variables (you want to change these before running the code):

LATMAX
LATMIN
LONMIN
LONMAX
DIRS.CEoriDirName
DIRS.mainDirStr
DIRS.TRMMdirName
startDateTime
endDateTime

baselineDataDir.zip: contains sample data.

code/

  • UserInterface.ipynb: notebook to use the GTG

  • iomethods.py: contains the I/O functions needed for the GTG

  • metrics.py: contains the core (atmospheric) metrics performed by the GTG

  • mccSearch.py contains all the functions needed

  • mccSearchUI.py contains a command line wizard for running the program

  • mainProg.py contains a sample of the general workflow of the order the modules should be called.

  • mainProgTemplate.py the general workflow of the order the modules should be called in order to complete an evaluation.

  • plotting.py: contains the core plotting functions for the GTG

  • utils.py: contains misc. functions needed by the GTG

  • variables.py: contains the classes for to set up the variables needed by the GTG

  • Scripts for developers *

  • gtg_lint_file: the config file for the GTG linting standards.

  • code/baselineTiming.py: script for timing individual components of the GTG

  • code/checkMERGBaseline.py: script for checking the differences between outputs from various GTG runs

How to run the GTG

New Users (Juypter Notebook)

Advanced Users (Python Command Line)

Run mccSearchUI.py

python mccSearchUI.py

mccSearchUI.py is the use of GTG for the weather application of finding mesoscale convective complexes (MCCs) in When you run the command line, mccSearchUI.py, there are a number of inputs that will be required. There are the main inputs you will have to supply (as well as their variable names in the code - especially useful for editing mainProg.py or tracing the code:

  • mainDirStr : this is the working directory where you wish all the output files –images, textfiles, clipped netCDF files- to be store
  • There is an option to preprocess data. The program
  • CEoriDirName : this is the directory where the original MERG data in netCDF format is stored
  • TRMMdirName : this is the directory where the original TRMM data in netCDF format is stored
  • Start date and time in the format yyyymmddhr
  • End date and time in the format yyyymmddhr

The following assumptions are made:

  • input data are in one folder. For MERG data this is CEoriDirName and for the TRMM data this is TRMMdirName in mainProg.py. These directories cannot be the same.
  • THERE IS NO FILE CHECKING. So please ensure ALL your files are there in netCDF format.

GTG workflow in weather application mccSearchUI.py

The general workflow of the program. The dashed lines indicate optional paths.

The findCloudElements (find_cloud_elements) can be completed in a parallel fashion leveraging Python's multiprocessing libraries on the vectorization branch of the code. This design was contributed by Gabriel Mel de Fontenay, who was a final year CS student at USC during his capstone project. The design is illustrated below.The parallelized implementation of find_cloud_elements that leverages the Python multiprocessing library.

Successful run

Once everything went well, the directory you indicated where outputs should be stored when prompted in the CL will be generated, and four folders should appear in it.

  • image/ stores all images generated from plots.
  • textFiles/ stores all the text files generated during the run, e.g. cloudElementsUserFile.txt that contains information about each cloud element identified
  • MERGnetcdfCEs/ contains the infrared data in masked netCDF files that have been generated for each cloud element identified
  • TRMMnetcdfCEs/ contains the precipitation data in clipped netCDF files that have been generated for each cloud element identified.
Clone this wiki locally