ESA CCI soil moisture CDF file generation

[Gayathridevi123]

Hi,

1. I'm trying to generate an ESA CCI soil moisture observation-based CDF file. But from the log file, it is clear that LDT is not reading the ESA CCI SM data and the values in the generated CDF file is zero. config and the log file are attached. 
   ldt.config_esacci.txt
   ldtlog_observation.txt
   Kindly let me know the reason behind the above issue. 

2. I would like to know about the "Observation count threshold: " option.

3. Also "Temporal resolution of CDFs", if this option is set as monthly then what is the Observation count threshold value?

Thank you,
Gayathri

[karsenau]

Hi @Gayathridevi123:

1. Could you try to set a longer run time in your ldt.config file, since it is only currently set for 1 day?

2. The "Observation count threshold:" is designed to eliminate grid points with too few observations (applied more for DA observations and not for the model CDFs).

3. If you apply "monthly" for the CDF temporal resolution, we have selected at times "50" or "100" for the count threshold.

Please let us know if this addressed your questions.

[Gayathridevi123]

Hi,

1. Set the run time as 30 days, still not reading ESA CCI files. Attached the log and config fie. Is there any specific format for ESA CCI files directory? like I kept one-month files in the directory ECV... so the ECV directory has files like ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED-20020101000000-fv06.1.nc, ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED-20020102000000-fv06.1.nc, etc...
   ldt.config.txt
   ldtlog_observation.txt

2. Can you explain this option in a little bit more detail..Sorry, still I'm confused with this threshold value. In the case of model CDF, how does this option have to be set?

3. 50 or 100 count threshold will be applicable for 3 or 4-year simulation?

[karsenau]

Hi @Gayathridevi123:

1. Reviewing the files that you are reading in and version, it looks like the file naming convention should work. However, we download the files from the ESA CCI website into "year-based" directories (e.g., "1978", "1979, ... "2020",etc.). You can see the structure of the file directory in the following LDT reader for this DA obs dataset:

https://github.com/NASA-LIS/LISF/blob/master/ldt/DAobs/ESACCI_sm/readESACCIsmObs.F90

see subroutine: create_ESACCIsm_filename

So if you were to group files by year in to local year-named directories, LDT and LIS should both be able to read the files just fine, for the convention of files and format that you downloaded. Also, this subroutine reader is not flagging which files are not being read in (it inquires if the correct directory/file naming convention is present but nothing else). It may be helpful if we include a warning statement to users if the file is not being read in and why.

2. When generating the CDF statistics and histogram information, you want sufficient amount of data to be employed to have a more representative sample for the bias-correction and any scaling. This can be important for satellite-based observations that can be impacted by sampling frequency, and for some, by cloud cover, rain, frozen soils, etc. This option of the observation count threshold can help in this selection.

3. For shorter time periods, you may want to reduce your count threshold. However, if you have a longer period of record, it is recommended to use as many years of data as possible.

Hope this information helps better address your questions and issue. Please let us know if you have any other questions. Thanks again!

[Gayathridevi123]

Hi,

Sorry for my late reply. Yeah, grouping the ESA CCI files into year-based directories works. Thank you @karsenau for your help.

Gayathri

[karsenau]

Glad that solved your issue. Hope the rest of your runs go smoothly!