Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add flag to CompareWorkspaces so users can specify NaN == NaN behavior #38075

Merged
merged 3 commits into from
Oct 1, 2024

Conversation

rboston628
Copy link
Contributor

@rboston628 rboston628 commented Sep 24, 2024

Description of work

Summary of work

Adds a flag to set whether NaN values should be considered equal to the CompareWorkspaces algorithm.

Purpose of work

When CompareWorkspaces was originally written, it was written so that if two workspaces had NaN for a value, the two NaN values would compare as equal. This is contrary to the IEEE 745 specification, which says that NaN compares false against everything, including other NaNs, so that NaN == NaN should evaluate as false.

In attempting to fix this, it was found there are at least a few tests relying on the old behavior of comparing NaNs. Therefore, this provides a flag to CompareWorkspaces so that users can elect for NaNs to be considered equal.

See also Issue #38088

Related to ORNL EWM 7196
(no further details at that link, only linking for bookkeeping)

Further detail of work

Within CompareWorkspaces, the methods to check if two values are within a tolerance have been replaced to rely on the FloatingPointComparison operators, with a check to return true if both values are NaN and the NaNsEqual flag was set to true.

The CheckAllData flag will now work on two type of peaks workspaces.

More tests were added for CompareWorkspaces and FloatingPointComparison, to handle more cases involving NaNs or extreme numbers.

A few cppcheck suppressions were removed.

Also, some comments were added in some parts of the code. These places were found using a search for abs( and fabs( within the code for possible replacements with withinAbsoluteDifference. While not suitable for replacements, they could lead to improvements in the code.

It was necessary to fix several system tests, which relied on previous behavior for NaN comparison. When the NaNs were only in the uncertainties, then checking uncertainty was turned off. Otherwise, with NaNs in the y-values. the NaNsEqual flag was set for the comparison.

The following tests were updated to work with new NaN behavior:

  • ISISIndirectInelastic.OSIRISMultiFileSummedReduction
  • ISIS_PowderPolarisTest.FocusTestAbsorptionPaalmanPings
  • DirectILLAutoProcessTest.DirectILLAuto_PANTHER_Powder_Test
  • DirectILLAutoProcessTest.DirectILLAuto_SHARP_Powder_Test
  • MDNormHYSPECTest.MDNormHYSPECTest
  • SANSBeamCentreFinderCoreTest.SANSBeamCentreCoreRunnerTest
  • SANSReductionCoreTest.SANSReductionCoreRunnerTest
  • SANSILLReductionTest.ILL_D33_LTOF_Test
  • D7AbsoluteCrossSectionsTest.ILL_D7_TimeOfFlight_Test
  • ReflectometryQuickMultiDetector.ReflectometryQuickMultiDetector
  • ILLDirectGeometryReductionTest.IN4
  • ILLDirectGeometryReductionTest.IN5_Mask_Non_Overlapping_Bins
  • FlipperEfficiencyTest.FlipperEfficiencyPolarisedTest
  • FlipperEfficiencyTest.FlipperEfficiencyUnpolarisedTest
  • AbinsTest.AbinsCRYSTAL2D
  • SANSILLAutoProcessTest.D11_AutoProcess_IQxQy_Test
  • SANSILLAutoProcessTest.D11_AutoProcess_Multiple_Transmissions_Test
  • SANSILLAutoProcessTest.D33_AutoProcess_IPhiQ_Test
  • ISISIndirectEnergyTransferTest.ISISIndirectEnergyTransferTest
  • MuonProcessTest.MuonProcessTest

The following three tests relied on comparing NaNs to finite floating points, which used to evaluate as true. There is no way to fix these without fixing the reference files. They are marked to skip, and in their previous forms were not meaningful. It is possible replacing the NaNs with 0 could fix some of them.

  • CNCSReductionTest.CNCSReductionTest
  • SANSZOOMTomlFileConversion.ZOOMV1UserFileTest_m8
  • ARCSReductionTest.ARCSReductionTest

See also issue #38088

To test:

New tests were added to

  • CompareWorkspacesTest.h
  • TableColumnTest.h
  • FloatPointComparisonTest.h

covering several situations with NaNs and also with extreme values.

These tests should be convincing that the behavior of NaNs with and without the flag is as expected.


Reviewer

Please comment on the points listed below (full description).
Your comments will be used as part of the gatekeeper process, so please comment clearly on what you have checked during your review. If changes are made to the PR during the review process then your final comment will be the most important for gatekeepers. In this comment you should make it clear why any earlier review is still valid, or confirm that all requested changes have been addressed.

Code Review

  • Is the code of an acceptable quality?
  • Does the code conform to the coding standards?
  • Are the unit tests small and test the class in isolation?
  • If there is GUI work does it follow the GUI standards?
  • If there are changes in the release notes then do they describe the changes appropriately?
  • Do the release notes conform to the release notes guide?

Functional Tests

  • Do changes function as described? Add comments below that describe the tests performed?
  • Do the changes handle unexpected situations, e.g. bad input?
  • Has the relevant (user and developer) documentation been added/updated?

Does everything look good? Mark the review as Approve. A member of @mantidproject/gatekeepers will take care of it.

Gatekeeper

If you need to request changes to a PR then please add a comment and set the review status to "Request changes". This will stop the PR from showing up in the list for other gatekeepers.

@rboston628 rboston628 added this to the Release 6.12 milestone Sep 26, 2024
@rboston628 rboston628 added Framework Issues and pull requests related to components in the Framework Patch Candidate Urgent issues that must be included in a patch following a release labels Sep 26, 2024
@rboston628 rboston628 marked this pull request as ready for review September 27, 2024 15:35
@rboston628 rboston628 removed the Patch Candidate Urgent issues that must be included in a patch following a release label Sep 30, 2024
Copy link
Member

@peterfpeterson peterfpeterson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fairly minor change requests. I like the centralized method and making whether nan should be equal to nan is an excellent thing to make explicit.

Later on, we should consider modernizing TableColumn.

@@ -64,10 +65,10 @@ template <class Type> class TableColumn : public API::Column {
std::string name = std::string(typeid(Type).name());
if ((name.find('i') != std::string::npos) || (name.find('l') != std::string::npos) ||
(name.find('x') != std::string::npos)) {
if (length == 4) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot of work we could do to make TableColumn not so ugly. Example: move implementation into the cpp file. That is a job for a different PR.

@KedoKudo KedoKudo merged commit db4f29d into main Oct 1, 2024
10 checks passed
@KedoKudo KedoKudo deleted the ewm7196-nans-flag branch October 1, 2024 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Framework Issues and pull requests related to components in the Framework
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants