Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Currently failing longtests and examples #316

Open
tmbgreaves opened this issue May 5, 2021 · 21 comments
Open

Currently failing longtests and examples #316

tmbgreaves opened this issue May 5, 2021 · 21 comments
Assignees
@tmbgreaves

Comments

@tmbgreaves
Copy link
Contributor

tmbgreaves commented May 5, 2021 •
edited
Loading

This issue documents the remaining failing tests following the Actions/GCC10 PR work. I'll update this initial post with the latest details - feel free to edit if you have updates, or note in the discussion below.

There are currently six failing tests/examples needing attention:

  • EXAMPLE: driven_cavity : One failure, see note below
  • EXAMPLE: lock_exchange : 'TypeError: only integer scalar arrays can be converted to a scalar index' [0]
  • LONGTEST: lock_exchange_3d_parallel : 'Error: Process completed with exit code 137.' [1]
  • LONGTEST: saltfinger2d_adaptive : GMSH erroring during mesh generation
  • VLONGTEST: backward_facing_step_2d_longtest : Two testing failures, see below.
  • VLONGTEST: flredecomp_long : timing out after three days, needs rerunning manually.

VLong tests are running on the longtests repo at https://github.com/FluidityProject/longtests/actions

[0] https://github.com/FluidityProject/fluidity/runs/2522255272?check_suite_focus=true
[1] https://github.com/FluidityProject/fluidity/runs/2522625135?check_suite_focus=true
@tmbgreaves tmbgreaves mentioned this issue May 5, 2021
Merged
@tmbgreaves
Copy link
Contributor Author

tmbgreaves commented May 5, 2021 •
edited
Loading

Question moved from PR #311

@angus-g - I think I have a question for you: it looks like some of the examples / longtests which parse detectors are failing as a result of detector files now outputting as h5part, but the examples/tests are still configured to read old-style detectors output via stat_parse from the fluidity_tools module. On a first pass, tests which I suspect are failing from this are:

  • wetting_and_drying_balzano1_dg_parallel
  • hokkaido-nansei-oki_tsunami
  • lock_exchange_2d_Lagrangian_paths
  • square-convection-1e6
  • Stokes_subduction_zone_vanKeken2008_OneA

Does this sound like it could be a reasonable diagnosis and if so how straightforward would a fix be?

@tmbgreaves
Copy link
Contributor Author

tmbgreaves commented May 5, 2021 •
edited
Loading

Question moved from PR #311

UPDATED 2021/05/06 @jhill1 - I think I've now fixed the medp1dgp2 utScan bug.

@tmbgreaves
Copy link
Contributor Author

Question moved from PR #311

@matt-piggott - I'm seeing what I hope is a marginal test failure in the driven_cavity example on Ubuntu 20.04, where a test of convergence > 1.85 fails on:

driven_cavity: Running Quantity4ConvergesToSecondOrder:
[[1.72761663]
[2.23112378]
[2.61936127]]
driven_cavity: failure.

Can this be relaxed slightly?

@tmbgreaves tmbgreaves self-assigned this May 6, 2021
@tmbgreaves
Copy link
Contributor Author

@drhodrid - a follow-up on the updated particle tests, I don't think I've messed up the merge of your updates but am seeing what looks like a significant RMS error change from particle_entrainment_of_dense_layer, see:

https://github.com/FluidityProject/fluidity/runs/2509562204?check_suite_focus=true

Test is '560 +- 10', actual value is 90.25800673280742

@tmbgreaves
Copy link
Contributor Author

@drhodrid - in particle_rayleigh_taylor_mu10fold on Focal I'm seeing one failure of 'Max entrainment less than 0.9' but a test value of 'Max Entrainment 0.903234453893775'. Am I OK to relax that slightly to get a pass?

@tmbgreaves
Copy link
Contributor Author

Documenting BFS2D longtest failures:

Test: Running reference reattachment length is about 14.0
Value: Assigning reference_reattachment_length = [[ 14.60728793 60.00759976]
[ 14.46771101 70.00617095]
[ 14.34311088 80.00749879]
[ 13.94609803 90.00016529]
[ 14.14875746 100.01463062]
[ 14.11128432 110.01588166]
[ 14.30831721 120.0105066 ]
[ 13.11879199 130.01656148]
[ 13.82287353 140.01110268]
[ 13.5161791 150.00253021]
[ 13.19420174 160.01205175]
[ 13.09551441 170.0267938 ]
[ 12.79628411 180.01357642]
[ 12.66541669 190.00075329]
[ 12.54315814 200.0169508 ]]

Test: Running kepsilon reattachment length is approximately equal to the experimental value of 7.0
Value: Assigning kepsilon_reattachment_length = [[ 5.94598516 130.03556546]
[ 5.94630335 140.03053697]
[ 5.94655816 150.02550848]
[ 5.94676675 160.02047999]
[ 5.94694063 170.0154515 ]
[ 5.94708778 180.010423 ]
[ 5.94721393 190.00539451]
[ 5.94732327 200.00036601]]

Completed run at : https://github.com/FluidityProject/longtests/runs/2527092698?check_suite_focus=true

@drhodrid
Copy link
Contributor

Hi Tim. I just pushed some changes that will hopefully fix particle_entrainment_of_dense_layer. Please let me know if that's the case when its run through.

@tmbgreaves
Copy link
Contributor Author

tmbgreaves commented May 14, 2021 •
edited
Loading

Thanks Rhod, that's running now. I'll post results as they come in.

Update: test has run for significantly longer this time, which looks promising.

@drhodrid
Copy link
Contributor

drhodrid commented May 14, 2021 via email

@tmbgreaves
Copy link
Contributor Author

@drhodrid - this time was my error I think - sorry! I was running the previous version by mistake. But it still needs testing as it overran the 6h time limit on the standard queue, so I'm rerunning on the vlong queue now and will update with results.

@drhodrid
Copy link
Contributor

drhodrid commented May 17, 2021 via email

@tmbgreaves
Copy link
Contributor Author

tmbgreaves commented May 17, 2021 •
edited
Loading

Run started at https://github.com/FluidityProject/longtests/actions/runs/849706302

@tmbgreaves
Copy link
Contributor Author

@drhodrid - confirming pass: https://github.com/FluidityProject/longtests/actions/runs/849706302

Could I also nudge you on the hopefully-tolerance failure in particle_rayleigh_taylor_mu10fold ? I'm seeing one failure of 'Max entrainment less than 0.9' but a test value of 'Max Entrainment 0.903234453893775'. Am I OK to relax that slightly to get a pass?

@drhodrid
Copy link
Contributor

drhodrid commented May 18, 2021 via email

@tmbgreaves
Copy link
Contributor Author

Thanks @drhodrid - done and re-running at https://github.com/FluidityProject/longtests/actions/runs/852603097

@tmbgreaves
Copy link
Contributor Author

@drhodrid - another of yours, I'm afraid - Stokes_mms_p1dg_p2 is failing to converge. Might you be able to take a look when convenient and see if it's still a test you want, and if so propose a fix?

@drhodrid
Copy link
Contributor

drhodrid commented Jun 7, 2021 via email

@tmbgreaves
Copy link
Contributor Author

Perfect, all passing now, thank you @drhodrid

@tmbgreaves
Copy link
Contributor Author

@javalosp - this is the issue documenting failing tests :-) See the first entry for the summary of where we are at the moment - I think driven_cavity is (hopefully) a straightforward one to tackle first, as it looks like a tolerance failure.

@tmbgreaves
Copy link
Contributor Author

Here's a method for reproducing the failures using a test environment generated by Actions:

docker run -it fluidity/actions:focal-68ec40081346d1dab836f23b92742315a9c72430 bash -il
./bin/testharness -f <test>.xml

@tmbgreaves
Copy link
Contributor Author

Noting that @Patol75 is working on a general testing rewrite in PR #326 and has fixed at least some of these failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tmbgreaves tmbgreaves

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tmbgreaves @drhodrid