Replies: 2 comments 2 replies
-
Did you try solving w/o noise model?
ml.solve(noise=False)
…On Thu, 25 Feb 2021 at 20:03, HenrikSpa ***@***.***> wrote:
Hi.
First let me say thanks for the excellent regression tool Pastas! I've
produced several regressions that are unbelievably good using this toolbox!
However, I've stumbled upon a problem. I'm trying to run what I assumed
was a very simple regression using just one series with observed values and
one waterlevel stress. I read the data using two sql queries which returns
two series with 1 hour values.
I resamples and interpolates the series to be sure they are both 1 hour
and contain no NaN-values.
I'm running Pastas version 0.14.0
I'm running Pastas version 1.1.0
I've uploaded the two plots, one with the original observation and stress
and the second with the result. As can be seen, the result is very wrong,
but I just can't figure out why. The simulation is several meters below the
observed time series!
Does anyone have an idea about what might be wrong?
import matplotlib.pyplot as plt
import pastas as ps
# Prepare data
obs = pd.read_sql(sql1, con, index_col=['date_time'], coerce_float=True,
parse_dates=['date_time'])['level_masl']['2016-11-08':]
lvl = pd.read_sql(sql2, con, index_col=['date_time'], coerce_float=True,
parse_dates=['date_time'])['level_masl']['2016-11-08':]
obs = obs.resample('1h').mean().interpolate()
lvl = lvl.resample('1h').mean().interpolate()
# First plot just to show the data
obs.plot(label='obs')
lvl.plot(ax=plt.gca(), label='stress')
plt.gca().legend()
# Create model, stress, solve and plot
ml = ps.Model(obs, name="GWL")
sm = ps.StressModel(lvl, ps.Polder, name='lvl', settings="waterlevel")
ml.add_stressmodel(sm)
ml.solve()
ml.plot()
# Description of the data
print("Obs")
print(str(obs.describe()))
print("Lvl")
print(str(lvl.describe()))
plt.show()
Result from describe()
Obs
count 3360.000000
mean 0.943069
std 0.061162
min 0.801700
25% 0.899215
50% 0.963230
75% 0.993213
max 1.023505
Name: level_masl, dtype: float64
Lvl:
count 3360.000000
mean 0.577475
std 0.096061
min 0.419120
25% 0.483245
50% 0.577105
75% 0.665942
max 0.791580
Name: level_masl, dtype: float64
[image: obs_stress]
<https://user-images.githubusercontent.com/17615137/109203327-50ddff00-77a4-11eb-8694-927bef7f4877.png>
[image: results]
<https://user-images.githubusercontent.com/17615137/109203335-54718600-77a4-11eb-828f-54449e1a8cf1.png>
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#273>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKM3SFA7B7OU4UA63FLRWTTA2NINANCNFSM4YHB2RAQ>
.
|
Beta Was this translation helpful? Give feedback.
1 reply
-
The Noise model sometimes does not perform as intended when the time
interval is very small.
Mind you: the program does exactly what it is programmed to do: it
minimizes the sum of the squared 'noise'. But if that 'noise' is not
actually a good representation of the actual noise (or it isn't even white
noise), then it gives undesirable results. This is a known problem with a
first order noise model. Better solutions are part of current research.
You often get a better result when minimizing the residuals (by putting
noise=False), but then you can not make any statistical inferences of the
results, as the residuals are likely correlated in time.
…On Fri, Feb 26, 2021 at 9:14 AM HenrikSpa ***@***.***> wrote:
It worked with ml.solve(noise=False)!
Should the noise be False most of the time, or is there something to gain
by having noise on?
And is there some limitation in the noise model which can explain why I
get the odd behavior while using it in this case?
[image: results_noise_false]
<https://user-images.githubusercontent.com/17615137/109271304-a00d4980-780f-11eb-8a99-160df0b931bb.png>
Many thanks and regards,
Henrik
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#273 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKM3SCTLM3DWBUQ32K3IXTTA5J6BANCNFSM4YHB2RAQ>
.
|
Beta Was this translation helpful? Give feedback.
1 reply
Answer selected by
HenrikSpa
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi.
First let me say thanks for the excellent regression tool Pastas! I've produced several regressions that are unbelievably good using this toolbox!
However, I've stumbled upon a problem. I'm trying to run what I assumed was a very simple regression using just one series with observed values and one waterlevel stress. I read the data using two sql queries which returns two series with 1 hour values.
I resamples and interpolates the series to be sure they are both 1 hour and contain no NaN-values.
I'm running Pastas version 0.14.0
I'm running Pastas version 1.1.0
I've uploaded the two plots, one with the original observation and stress and the second with the result. As can be seen, the result is very wrong, but I just can't figure out why. The simulation is several meters below the observed time series!
Does anyone have an idea about what might be wrong?
Result from describe()
Beta Was this translation helpful? Give feedback.
All reactions