Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Non-normalised Decimal numbers #42

Open
joshcarp opened this issue May 14, 2019 · 1 comment
Open

Fix Non-normalised Decimal numbers #42

joshcarp opened this issue May 14, 2019 · 1 comment

Comments

@joshcarp
Copy link
Contributor

Currently numbers are "normalised" so that there are trailing zeros,
for example 1 is stored as a significand 100000000000000 with an exponent of -15

Generally outputs are normalised from ParseDecimal and arithmetic functions.
Some problems are:

  1. Errors may arise when Decimal encodings are parsed in using methods other then strings
  2. Arithmetic functions currently have a remove trailing zeros function to avoid integer overflow
    If Decimal64s are stored without trailing zeros arithmetic functions may be faster
@anzdaddy
Copy link
Member

anzdaddy commented May 14, 2019

In early experiments, I noticed that gcc stores the literal decimal 1 as 1 with an exponent of 0. So either:

  1. it doesn't bother to normalise (just store whatever representation is easiest in a given context), or
  2. it normalises the other way (use the smallest possible integer that retains all digits).

This may require further experimentation.

The key requirement, however, should be to ensure that the library correctly handles all valid representations of each number, not just one normalised form.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants