This neural network will predict land cover classification from Landsat-8 and Sentinel-2 data. This supports the NASA SBIR "Open-Source Deep Learning Classification and Visualization of Multi-Temporal Multi-Source Satellite Data",

https://sbir.nasa.gov/SBIR/abstracts/18/sbir/phase1/SBIR-18-1-S5.03-4282.html

DATA REPRESENTATION: The raw format of data used here are .tif files from the HLS dataset with one file for each spectral band (19). Each file contains data for 26 timestamps, of which 19 are used. These derive from four separate regions Arkansas, Ohio, California, Vermont(?).

There are also ground truth files for each of these regions (tifs) containing category information.

Reformatted versions of the satellite and ground truth tif files have been saved, which have been converted to numpy arrays. In the first case, a file has been saved for each spectral band and time stamp, for a specific geographical subregion. In the second case, for a given geographical subregion all the data from that subregion has been combined in a single HT x WD x BANDS x TIMEPTS array.

HOW TO RUN CROPNET: We assume that you have a directory containing .tif files with a file for each spectral band of interest, e.g. hls_cls_ark_01.tif, hls_cls_ark_02, etc. We also assume that you have a corresponding .tif file which contains the CDL labels, e.g. cdl_2016_neAR.tif. Let these be (e.g.)

HLS_DIR=/Datasets/HLS/ark CDL_PATH=/Datasets/HLS/ground_truth/cdl_2016_neAR.tif

To make a 4D numpy array out of a particular subregion, run

DEEPRES= cd $DEEPRES/cropnet OUTPUT_SUPDIR=/media/data/Datasets/HLS/test_imgs python make_images.py --hls-dir $HLS_DIR --cdl $CDL_PATH -o $OUTPUT_SUPDIR

This will put .npy and .png files into $OUTPUT_SUPDIR/hls, including the 4D band x time x ht x wd file, which will begin with "hls_tb". The command given above will chip out the default subregion, which is the500 pixel NW corner of the region. To chip out a custom (square) region, say as defined by the bounding box [500, 1000, 1250, 1750], run

python make_images.py --hls-dir $HLS_DIR --cdl $CDL_PATH -o $OUTPUT_SUPDIR -x 500 -y 1000 --image-size 750

%%% TRAINING THE MODEL %%% The model trains in two steps. First the autoencoder is trained, to take the combined spectral/temporal data and transform it into an false-color RGB image of the measured region. Second, it runs image segmentation on the false-color image to acheive the correct classification.

Training the Autoencoder: This is a bit of a work in progress (with respect to properly training on all data available) so how the data sets get specified is subject to change. $OUTPUT_SUPDIR is the same as the previous section.

TRAIN_DIR=$OUTPUT_SUPDIR/hls TEST_DIR=$OUTPUT_SUPDIR/hls AE_OUTPUT_DIR=~/Training/cropnet/sessions

cd $DEEPRES/cropnet python cropnet/cropnet.py -d $TRAIN_DIR --test-data-dir $TEST_DIR --test-image-x 500 --test-image-y 0 --test-image-size 500 -o $AE_OUTPUT_DIR

This will put a 'feats.npy' file in a subdirectory of $AE_OUTPUT_DIR entitled 'session_' where n is chosen so as not to overwrite any existing directory. feats.npy will be the 500x500x3 false-color RGB image generated by the autoencoder corresponding to the test image.

Training for Segmentation:

MODEL_DIR=/Training/cropnet/models CDL_PATH=/media/data/Datasets/HLS/test_imgs/cdl/cdl_2016_neAR_0_0_500_500.npy FEATS_PATH=/Training/cropnet/sessions/session_00/feats.npy

cd $DEEPRES/cropnet python cropnet/seg_trainer.py -d $FEATS_PATH -l $CDL_PATH -o $MODEL_DIR

This will put a model file named 'seg_model.pkl' in $MODEL_DIR.

Running Inference with the Segmenter: Call this function when you already have a trained segmentation model but just want to run inference and compare to the ground truth.

MODEL_PATH=$MODEL_DIR/seg_model.pkl

cd $DEEPRES/cropnet python cropnet/segmenter.py --model-path $MODEL_PATH -d $FEATS_PATH -l $CDL_PATH

This will put figures in a 'segmentations' subdirectory of the subdirectory that $FEATS_PATH is in.