Avoid undefined behavior in FMA.

[hvdijk]

Overview

Avoid undefined behavior in FMA.

Reason for change

If the result is zero, the result is not denormal, so we do not need to follow the denormal special case logic. The check
rounding_mask & ansAsUint would always be zero, so it would never do anything, but the calculation of rounding_mask might have had undefined behavior due to an out-of-range exponent.

Description of change

Describe the intended behaviour your changes are meant to introduce to the
project and explain how they resolve the problem stated above. Detail any
relevant changes that may affect other users of the project, such as
compilation options, runtime flags, expected inputs and outputs, API entry
points, etc.

If you have added new testing, provide details on what tests you have added
and what the purpose of them is.

Anything else we should know?

If there's any other relevant information we should know that may help us in
understanding and verifying your patch, please include it here.

Checklist

• Read and follow the project Code of Conduct.
• Make sure the project builds successfully with your changes.
• Run relevant testing locally to avoid regressions.
• Run clang-format-16 (the most
  recent version available through pip) on all modified code.