ERA5

[sunnydays99]

Hello everyone,

I am trying to work with ERA5 as my main metforcing and have a few questions regarding the procedure required before being able to utilize it in my LIS runs. In the LDT user guide following features should be enabled:
ERA5 forcing directory: ./ERA5/
ERA5 forcing tile to grid mapping file: ../ERA5/mapping.nc
ERA5 forcing terrain height file: ../ERA5/era5_elev.nc
While I have found the terrain height file in the ERA5 catalogue, I am struggle to locate the forcing tile to grid mapping file. How can I attain it? Is there any other preprocessing I need to be aware of when working with ERA5?

Thank you so much for any advice!!

[dmocko]

Hi @sunnydays99

Are you on NCCS's Discover? That file is on that system, and I can point you to it.

If you are now, can you please share the filename of a typical ERA5 forcing file that you have?

LIS is expecting the files in the convention of: FORCING_YYYYMM.nc

Each file contains 11 forcing variables.

[sudarsan3010]

lis_config.txt
Hi @dmocko,

I have downloaded the ERA5 Single hourly forcings, by selecting the variables needed and downloaded it for a month in a single nc file and later renamed it to FORCING_200801.nc.

The selected variables were,

10m u-component of wind,
10m v-component of wind,
2m dewpoint temperature,
2m temperature,
Mean surface downward long-wave radiation flux,
Mean surface downward short-wave radiation flux,
Surface pressure,
Total precipitation

When I tried to run LIS, it gives me the following error

[INFO] Reading ERA5 file (bookend, 1 -/home/user6/sudarsan/LIS/forcing/ERA5_single/FORCING_200801.nc)
[ERR] nf90_inq_varid failed for psurf in read_era5 Stopping.
[ERR] endrun is being called

Did I miss selecting some variables in the forcing while downloading (as you mentioned 11 forcing variables)? Or do I need to download it in a different way?
Kindly suggest. I am attaching my lis.config as well.

Another query is, do we need a different mapping.nc file to use ERA5-Land forcings, or does the same mapping file work for both Single and Land forcings?

[dmocko]

Can you please copy the result of "ncdump -h FORCING_200801.nc" as a reply?

Here is the result of the file on our end:

netcdf FORCING_200801 {
dimensions:
        time = 745 ;
        Number_of_points = 340819 ;
variables:
        double time(time) ;
                time:units = "hours since 2008-01-01 00:00:00" ;
        double FRC_TIME_STP ;
                FRC_TIME_STP:units = "s" ;
        double LON(Number_of_points) ;
                LON:units = "degrees" ;
        double LAT(Number_of_points) ;
                LAT:units = "degrees" ;
        double ZS(Number_of_points) ;
                ZS:units = "m" ;
        double UREF(Number_of_points) ;
                UREF:units = "m" ;
        double ZREF(Number_of_points) ;
                ZREF:units = "m" ;
        double Tair(time, Number_of_points) ;
                Tair:units = "K" ;
        double Qair(time, Number_of_points) ;
                Qair:units = "kg/kg" ;
        double Wind(time, Number_of_points) ;
                Wind:units = "m/s" ;
        double DIR_SWdown(time, Number_of_points) ;
                DIR_SWdown:units = "W/m2" ;
        double LWdown(time, Number_of_points) ;
                LWdown:units = "W/m2" ;
        double Rainf(time, Number_of_points) ;
                Rainf:units = "kg/m2/s" ;
        double Snowf(time, Number_of_points) ;
                Snowf:units = "kg/m2/s" ;
        double PSurf(time, Number_of_points) ;
                PSurf:units = "Pa" ;
        double CO2air(time, Number_of_points) ;
                CO2air:units = "kg/m3" ;
        double SCA_SWdown(time, Number_of_points) ;
                SCA_SWdown:units = "W/m2" ;
        double Wind_DIR(time, Number_of_points) ;
                Wind_DIR:units = "deg" ;
}

You may not necessarily need all those variables. But if your time or Number_of_points are off (or a variable name),
then that could be why you are seeing that error.

[sudarsan3010]

I have downloaded the forcings in the format - NETCDF (Experimental).
The reply is as follows,

            d2m:long_name = "2 metre dewpoint temperature" ;
    short t2m(time, latitude, longitude) ;
            t2m:scale_factor = 0.00165225655000093 ;
            t2m:add_offset = 267.276448957174 ;
            t2m:_FillValue = -32767s ;
            t2m:missing_value = -32767s ;
            t2m:units = "K" ;
            t2m:long_name = "2 metre temperature" ;
    short msdwlwrf(time, latitude, longitude) ;
            msdwlwrf:scale_factor = 0.00683134214405815 ;
            msdwlwrf:add_offset = 292.267809304514 ;
            msdwlwrf:_FillValue = -32767s ;
            msdwlwrf:missing_value = -32767s ;
            msdwlwrf:units = "W m**-2" ;
            msdwlwrf:long_name = "Mean surface downward long-wave radiation flux" ;
    short msdwswrf(time, latitude, longitude) ;
            msdwswrf:scale_factor = 0.0194372491721728 ;
            msdwswrf:add_offset = 636.880906375414 ;
            msdwswrf:_FillValue = -32767s ;
            msdwswrf:missing_value = -32767s ;
            msdwswrf:units = "W m**-2" ;
            msdwswrf:long_name = "Mean surface downward short-wave radiation flux" ;
    short sp(time, latitude, longitude) ;
            sp:scale_factor = 0.895583081710741 ;
            sp:add_offset = 76751.6361928342 ;
            sp:_FillValue = -32767s ;
            sp:missing_value = -32767s ;
            sp:units = "Pa" ;
            sp:long_name = "Surface pressure" ;
            sp:standard_name = "surface_air_pressure" ;
    short tp(time, latitude, longitude) ;
            tp:scale_factor = 1.15628990713648e-06 ;
            tp:add_offset = 0.0378869950972339 ;
            tp:_FillValue = -32767s ;
            tp:missing_value = -32767s ;
            tp:units = "m" ;
            tp:long_name = "Total precipitation" ;

// global attributes:
:Conventions = "CF-1.6" ;
:history = "2024-05-13 05:48:39 GMT by grib_to_netcdf-2.28.1: /opt/ecmwf/mars-client/bin/grib_to_netcdf -S param -o /cache/data0/adaptor.mars.internal-1715579112.0580351-26503-14-2b649698-553f-47dc-b389-0c2d5339c8fc.nc /cache/tmp/2b649698-553f-47dc-b389-0c2d5339c8fc-adaptor.mars.internal-1715578751.631858-26503-9-tmp.grib" ;
}

[dmocko]

Hi - it appears to be cut-off. What is at the beginning/top of the result of "ncdump"?

Either way, your files aren't going to work. Have you tried the procedure and program that I outlined to the other user in this thread?

[sudarsan3010]

Hi @dmocko,

I Found my mistake, but still facing the same error. Last time, I have downloaded the wrong ERA5 product, and now even after downloading the correct product 'Near surface meteorological variables from 1979 to 2019 derived from bias-corrected reanalysis', I am getting the same error. I have also stictched the forcing variables as you have mentioned in the below thread.

Now the reply of "ncdump -h FORCING_200801.nc" is also similar to that of yours. Given below,

[user6@satellite ERA5_met]$ ncdump -h FORCING_200801.nc
netcdf FORCING_200801 {
dimensions:
time = 744 ;
Number_of_points = 340819 ;
variables:
float Wind(time, Number_of_points) ;
float LAT(Number_of_points) ;
float LON(Number_of_points) ;
float ZS(Number_of_points) ;
float UREF(Number_of_points) ;
float ZREF(Number_of_points) ;
float Tair(time, Number_of_points) ;
float Qair(time, Number_of_points) ;
float PSurf(time, Number_of_points) ;
float LWdown(time, Number_of_points) ;
float Snowf(time, Number_of_points) ;
float Rainf(time, Number_of_points) ;
float DIR_SWdown(time, Number_of_points) ;
}

The LDT is running successfully. This error pops only in LIS. I am also attaching my lis.config and lislog file. I am using the LISF v7.5.0

Error:
[INFO] Reading ERA5 file (bookend, 1 -/home/user6/sudarsan/LIS/forcing/ERA5_met/FORCING_200801.nc)
[ERR] nf90_get_var failed for ps in read_era5 Stopping.
[ERR] endrun is being called

forcing_variables.txt
lis_config.txt
lislog.txt

[sunnydays99]

@dmocko thank you so much for your fast reply!

Unfortunately I do not have access to discover, would it be possible for you to share the file here or in another way? I have downloaded the ERA5 files from Copernicus, so they have different naming and each forcing variable is separately per month. Here is an example of the naming style: Qair_WFDE5_CRU_199310_v2.1.nc . Can I process my files with LDT to obtain the correct naming style or did i accidentally download the wrong data?

[dmocko]

Yes, we can share the file with you, and I will do it soon.

Sorry for the slow reply this week (and the last couple of weeks) from the entire LIS team. We are having an annual review
this week, and we have been very busy preparing for it and now participating in the review.

About the naming style, we have a small FORTRAN program that we use to combine the files into a single FORCING_YYYYMM.nc file. I will share this program with you as well. It should work on the data that you downloaded (you'll just need to change the program to use that filename convention of the data that you downloaded). After you run this program, it will produce the data into the correct format/style that is expected by LDT and LIS.

[sunnydays99]

@dmocko Okay perfect, thank you so much that would be very helpful!

[sunnydays99]

@dmocko do you require my email address or is it possible to share the mapping file on github?
Kind regards and best of luck with the review meeting!

[dmocko]

HI @sunnydays99

Thanks for your patience. I have placed the mapping.nc file as well as our copy of the era5_elev.nc file here:
https://portal.nccs.nasa.gov/lisdata_pub/data/MET_FORCING/ERA5/

I have also placed the FORTRAN program to stitch together the individual variables to create a FORCING_YYYYMM.nc file,
that LDT/LIS expects as input for ERA5 forcing. You should just need to change the filenames of the individual input files
in the program, and then compile. You will also need the sampleforc.nc file I have also shared in the same location.

To make the executable, compile with these options:

ifort -c -O2 -traceback -nomixed_str_len_arg -names lowercase -convert big_endian -assume byterecl -I[path_to_netcdf]/include/  stitchData.F90
ifort -o stitchData stitchData.o -L[path_to_netcdf]/lib/ -lnetcdf -lnetcdff
rm *.o

Change [path_to_netcdf] to the path to your netCDF code (similar to what you did to compile LIS).

After you have the executable, just run the command for each month, like this:

    ./stitchData 199001
    ./stitchData 199002

…and so on for each month.

[sunnydays99]

Dear @dmocko,

thank you so much for your help and for the detailed explanation! Everything works well now and LIS is running with the ERA5 data now

[sudarsan3010]

Dear @dmocko,

Does your mapping.nc file also works for the ERA5-Land forcings at 0.1-degree resolution also?! or will work only at 0.25-degree resolution only?

If so, suggest the steps to create the mapping.nc file for ERA5-Land forcings as well.

[tais958]

Hi @sunnydays99 and @dmocko,
This is a very informative thread! I tried to follow the suggested steps from data download ('Near surface meteorological variables from 1979 to 2019 derived from bias-corrected reanalysis') to data stitch. It turned out the dimensions of forcing data (0.5 by 0.5 deg) and mapping file (0.25 by 0.25 deg) are not identical, so that I haven't had luck in getting the right LIS forcing data. Since it worked for @sunnydays99, I wonder if any of you could point out what might have went wrong in my case? Thanks in advance!

[tais958]

Hi @dmocko
Sorry for keeping repost but I do want to seek for your help if you could point me to the proper data source of ERA5 surface reanalysis. Many thanks in advance!

[dmocko]

Hi @tais958

I really don't know exactly where @sunnydays99 downloaded the ERA5 surface reanalysis.

If you look earlier this thread, you can see the filenames are of the convention:
Qair_WFDE5_CRU_199310_v2.1.nc

Can you look for files such as these in Copernicus?

[tais958]

Hi @dmocko

Thanks for response. I did look up what might be possible data source. What I get also showed:
"Qair_WFDE5_CRU_201501_v2.1.nc" However, they are in 0.5 deg resolution, which is not compatible to the dimensions defined in the script you provided. Could you confirm if the processed LIS forcing should be in 0.25 or 0.5 deg? Thanks very much!

[tais958]

Hi @dmocko

Just checked into the stitching Fortran script. It requires reading into netcdf files with "ERA5" in the file name, which seems to suggest "WFDE5_CRU" might not be the correct data to process? Below is the sampled code:

iret = nf90_open(path='Wind_ERA5_'//trim(dateinp)//'.nc',mode=NF90_NOWRITE, &

Any comments?

[dmocko]

For the latter question, you would need to simply rename the files downloaded to that expected as input by the stitching FORTRAN program.

As to where to find the 0.25-degree forcing data, I am not sure. @sunnydays99 - can you please help us out?

[dmocko]

Closing this discussion. The ERA5 data that can be used in LIS is NOT available outside of the immediate LIS team. We are working on code that will use ERA5 data that can be downloaded directly from Copernicus. Please look for a pull request for this code in the coming months!

[tais958]

Thanks @dmocko. I'll follow up on this and look forward to the code update.