Geostatistically reasonable parameter field with PESTPP-IES

[DStrom1987]

Hi,

I have been running different instances of pestpp for some time now. Looking at the results from some pestpp-ies runs I find it difficult to reach a parameter field that varies in a geostatistically reasonable way. The ies_reg_factor doesn't really make that much of a difference (which I suppose is expected, since it doesn't know anything about the geological materials in play).

I still find that there are sort of "bullet-point" behavior which I would like to avoid. What would be the work-flow to achieve smoother transitioning of for example k-values When it comes to pestpp-ies?

Obviously its a bit hard to tell without the scale for K. But i think the patterns tell enough about variability to get an idea of how they all preform.

Below are three examples of a k-parameter field from the Rocky Mounatains Arsenal example in Model Muse.

Fig. 1. PESTPP-GLM, No regularization

Fig. 2. PESTPP-GLM, first order pearson thikonov with geostatistical variogram

Fig. 3. PESTPP-IES, ies_reg_factor = 0.25

Fig. 4. Ordinary pest with thikonov regularization

[jtwhite79]

Did you bake the same geostat correlation structure into the prior parameter ensemble that is being used to enforce the regularization with pest/pestpp-glm?

[jtwhite79]

the one prior cov matrix can and should (and must!) include all adjustable parameters and can be block-diagonal with each block corresponding to the correlated parameters with in a given pilot point set. Or you can make an pest-style unc file that also describes the same block-diagonal cov matrix. Or if you have heaps of parameters, it can be more efficient and easier to generate the (correlated) prior parameter ensemble prior to runnning ies - this can be done with pyemu or the pest utils - in this case, you dont need to pass the prior cov matrix to ies...

[jtwhite79]

...anyhow, without baking the same preferred geostatistical relations into the prior parameter ensemble as used in tikhonov regularization with pest/pestpp-glm, I'm not surprised the ies result has those spatially isolated high and low values - its simply replicating/using the spatial patterns found in the prior parameter ensemble...

[DStrom1987]

I see.. I have generated prior covariance matrices with mkppstat and ppcov_sva. So the prior covariences are in place. Its just that I lack the tools to pass the individual covariance matrices to single block diagonal prior cov matrix. So yeah, I suppose I have the geostatistical patterns that are being used in the underlying pest run.

Does the pyemu suite or pypest utils offer some functionality to combine these prior covs into a single file (.unc-style perhaps?).

[cnicol-gwlogic]

You've done the hard yards already with mkppstat and ppcov - it'll take you about 5mins to build an unc file (just a text file that refers to those .mat files you made, and/or specified standard deviations for any unrelated params). See the original pest manual for the format, or here. Then just refer to that unc file in the ++parcov() entry in your control file.

[DStrom1987]

Cool. I’ll look into it tomorrow. Thanks for the advice! :)

[DStrom1987]

At first i thought the problem was solved by changing one of the parameters to logarithmic scale, but in the end it had nothing to do with it..

Reading through the link you provided among other help sections on pest homepage, it sure looks like the ppcov_sva could be used in par with the parcov() argument for pest++. Can they be passed through an .unc in their native format or do they need further preprocessing to match the requirements of pest++?

Since pest hangs with exception when the .unc-file is passed, something obviously goes wrong with the prior cov. Any ideas?

(Exception: run() returned non-zero: -1073741819)

[jtwhite79]

We need to see the actually stdout/stderr from the commandline/terminal and/or .rec file to see the error message

[DStrom1987]

The .rec file doesnt give many clues I'm afraid.
rmamf6start1.txt

It does, however, seem to recognize the covariance file. Since when I loaded in another file it told me the following:

Input from the python script below:

I'm running it from the gmdsi-tutorial environment if thats any help...

[DStrom1987]

You've done the hard yards already with mkppstat and ppcov - it'll take you about 5mins to build an unc file (just a text file that refers to those .mat files you made, and/or specified standard deviations for any unrelated params). See the original pest manual for the format, or here. Then just refer to that unc file in the ++parcov() entry in your control file.

I took up on your advice and made the file cov.unc together with my covariance matrices (attatched). For whatever reason, when i load the .unc file into my parcov() control variable (pst.pestpp_options["parcov"] = os.path.join("cov.unc")). the master do not connect with the workers... It starts initializing the prior as the last step. It doesnt inform anything in particular in the rmr, or rec files regarding whats failing.

Can you see anything strange with the files that could cause this behavior?

cov.txt

RmaMf6Start.R_pp.R.1.Cov.txt
RmaMf6Start.Kx.K.1.Cov.txt

[cnicol-gwlogic]

You.may need to leave it work for a while to generate the ensemble. It will take longer with your prior cov than it did without any cov.

[DStrom1987]

I see, problem is that the master client crashes before it gets to work at all and removes all workers. I'm not sure how I could leave it to work then?

[DStrom1987]

It seems as if it doesnt read in the .unc-file correctly. I dont really see anything wrong with it and it should follow the conventions described in the pest manual. Can the fact that it contains non-numerical values play a role? the names of parameters that is.

I've tried to pass the .unc-file alone containing the covariances without any prior parameter- or observation ensembles.