Skip to content

Formatting with increased precision seems like an error case #49

@jridgewell

Description

@jridgewell
const d = new Amount(1.23456, { significantDigits: 2 });

d.toString() // '1.2'

d.with({ significantDigits: 4 }).toString() // '1.200' or '1.235'?

From today's presentation, we discussed what should happen when formatting with a higher precision that the input's precision. I think we came to an agreement 1.235 is incorrect. But I think even 1.200 is incorrect, it implies a precision which is not captured in our input.

Quoting a bit from wikipedia:

The following types of digits are not considered significant:

  • Spurious digits that arise from calculations resulting in a higher precision than the original data or a measurement reported with greater precision than the instrument's resolution. link

As an example, the amount we hvae is the calculation of some operation. Eg, if we had a * b === 1.23456, with a = 2.46912 and b = 2.0. Even though the calculation has a precision precision of 6, the actual amount represented here only contains 2. It is incorrect to ever print this value as 1.23456 because I am only certain of the first 2 digits.

I would accept d.with(4).toString() as 1.2, making it very clear that we cannot create more precision from an amount that doesn't have that to begin with. But 1.200 seems wrong, because we have no idea what those 00 should be.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions