Accessing data: Hints and Tips

---

Accessing Data over SSHFS

---

This example applies to accessing NEMO data on JASMIN from the local filesystem.

First make sure that you can mount and unmount the remote filesystem locally as described in the SSHFS notes.

Create a local directory to store links to the remote data. Mount the remote filesystem and navigate to the data. The NEMO data are stored in annual directories YYYY and have the form *d05T.nc, *d05U.nc etc.

From this directory run something like the following:

#!/bin/bash

for y in {YEAR_START..YEAR_END}
  do 
  cd $PATH_TO_REMOTE_DATA/$y
  mkdir $PATH_TO_LOCAL_DATA/$y
  for fn in *d05T.nc
    do
    ln -s $PATH_TO_REMOTE_DATA/$y/$fn $PATH_TO_LOCAL_DATA/$y/$fn
  done
done

then cd into $PATH_TO_LOCAL_DATA and run the following (filenames for illustration)

#!/bin/bash

for y in {YEAR_START..YEAR_END}
  do 
  cd ../$y
  ym1=`expr $y - 1`
  yp1=`expr $y + 1`
  ln -s ../$ym1/ORCA0083-N006_$ym1\1231d05T.nc ORCA0083-N006_$ym1\1231d05T.nc
  ln -s ../$yp1/ORCA0083-N006_$yp1\0105d05T.nc ORCA0083-N006_$yp1\0105d05T.nc
done

This adds the 5 day mean files either side of the current year to allow PyNEMO to interpolate onto the start and end days of each year.

A simple NCML file can then be written for each year:

<?xml version="1.0" encoding="UTF-8"?>
<netcdf title="aggregation example" xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
  <variable name="vozocrtx"  orgName="uo" />
  <variable name="vomecrty" orgName="vo" />
  <variable name="votemper" orgName="potemp" />
  <variable name="vosaline" orgName="salin" />
  <variable name="sossheig" orgName="ssh" />
  <aggregation type="union" >
     <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
        <aggregation type="joinExisting" dimName="time_counter" >
           <scan location="/projectsa/Eurobasin/forcing/ocn/src/ORCA0083-N006/1980/"  regExp=".*d05U\.nc$" />
        </aggregation>
     </netcdf>
     <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
        <aggregation type="joinExisting" dimName="time_counter" >
           <scan location="/projectsa/Eurobasin/forcing/ocn/src/ORCA0083-N006/1980/"  regExp=".*d05V\.nc$" />
        </aggregation>
     </netcdf>
     <netcdf xmlns="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2">
        <aggregation type="joinExisting" dimName="time_counter" >
           <scan location="/projectsa/Eurobasin/forcing/ocn/src/ORCA0083-N006/1980/"  regExp=".*d05T\.nc$" />
        </aggregation>
     </netcdf>
  </aggregation>
</netcdf>