Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug when nrestart=1 #5

Open
Qiyu-Song opened this issue May 5, 2022 · 12 comments
Open

Bug when nrestart=1 #5

Qiyu-Song opened this issue May 5, 2022 · 12 comments

Comments

@Qiyu-Song
Copy link
Collaborator

Qiyu-Song commented May 5, 2022

If nrestart=1, all fields will become NAN quickly (or maybe at first step after restart). Need to fixed.

@natgeo-wong
Copy link
Collaborator

I remember that nrestart=1 for v6.10 cannot retrieve the old stat file properly as well.

@Qiyu-Song
Copy link
Collaborator Author

You mean when dompiensemble=.true.? I'll try to check the write and read subroutines for restart files tomorrow.

@natgeo-wong
Copy link
Collaborator

just in general. I've tried to restart the experimental runs after accidentally timeout before and it's never worked for me.

@Qiyu-Song
Copy link
Collaborator Author

Oh, if an experiment stops accidentally, there will be problems with restart (incomplete file/mismatched steps in restart file and stat file). But I was talking about experiments that end normally. Just tested the clean version of 6.11.7 and it seemed to be good, but our version has this bug even if I set dompiensemble=.false.

@natgeo-wong
Copy link
Collaborator

Shouldn't you use nrestart = 2?

@Qiyu-Song
Copy link
Collaborator Author

Tried nrestart=2 but still got NAN even when dompiensemble=.false. Also tried with the original code and it worked well with restart. What other changes did you make to the code besides mpiensemble?

@natgeo-wong
Copy link
Collaborator

I added in other changes separately in other commits. I think the ones that most concern you would be the commits:

where I adjust how SAM perturbs the initial conditions.

@Qiyu-Song
Copy link
Collaborator Author

Qiyu-Song commented May 18, 2022

I don't think those 2 commits are relevant to this bug. I tested a clean version with changes only in the following files: main.f90, stepout.f90, restart.f90, hbuffer.f90, pressure.f90, pressure_mpiensemble.f90, and RAD_RRTM/rad.f90, and the restart worked well (both =1 or =2)

@Qiyu-Song
Copy link
Collaborator Author

Oh it seems that those two commits you mentioned earlier ARE the problem. I tried to replace the related files in those two commits into my "clean" version with mpiensemble, which performed well with restart, and then the restart didn't work.

@natgeo-wong
Copy link
Collaborator

natgeo-wong commented May 20, 2022

SFLR (sorry for late reply) I am currently not in the right frame of mind to work. But can we have a look at this when I am back (I'm back on Tuesday, but I will probably come into office on Wednesday)?

@Qiyu-Song
Copy link
Collaborator Author

Created a new branch qiyusong. Good for restart.

natgeo-wong added a commit that referenced this issue Sep 7, 2023
Now allowing for gridded archipelago of islands
@natgeo-wong
Copy link
Collaborator

I'll probably get around to testing nrestart=1 after this week hopefully when I finished with the current paper draft

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants