Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

starlarktest/assert.star: allow +/-1 ULP in float eq comparisons #490

Merged
merged 1 commit into from
Jul 25, 2023
Merged

starlarktest/assert.star: allow +/-1 ULP in float eq comparisons #490

merged 1 commit into from
Jul 25, 2023
+39 −6

Conversation

adonovan
Copy link
Collaborator

Fixes #488

@adonovan adonovan mentioned this pull request Jul 24, 2023
Closed
@adonovan adonovan requested a review from jayconrod July 24, 2023 16:00
jayconrod
jayconrod approved these changes Jul 25, 2023
View reviewed changes
Copy link
Collaborator

@jayconrod jayconrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I'm actually surprised trig operations get within 1 ULP of accuracy. I remember that not being the case for JavaScript.

@adonovan adonovan merged commit 0d72639 into master Jul 25, 2023
13 checks passed
@adonovan adonovan deleted the float-ulp-s390 branch July 25, 2023 16:15
@adonovan
Copy link
Collaborator Author

I'm actually surprised trig operations get within 1 ULP of accuracy. I remember that not being the case for JavaScript.

Yeah, I was being optimistic. I may need to relax the check later, but the issue reported here was off by exactly 1 ULP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jayconrod jayconrod jayconrod approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

s390x precision seems to be a little bit off in some operations
2 participants
@adonovan @jayconrod