Skip to content

round_sf_with_strategy returns values which exceed the max allowed scale #705

@extremeandy

Description

@extremeandy

It appears the round_sf_with_strategy does not do appropriate checking on scale and can return invalid values.

Example:

let value = Decimal::new(1, 20)
    .round_sf_with_strategy(12, RoundingStrategy::MidpointAwayFromZero)
    .unwrap();

assert!(value.scale() <= 28);
print!("{value}"); // Also panics with error: 'CapacityError: insufficient capacity'

Perhaps the fix is as simple as capping the precision when doing the rescale? i.e.:

ops::array::rescale_internal(&mut array, &mut value_scale, (scale + digits - mantissa_sf).min(MAX_PRECISION_U32));

I also noted that the return type of round_sf_with_strategy is Option<Decimal>, and the docs say: "If the resulting number is unable to be represented by the Decimal number then None will be returned.". However, all code paths currently return Some.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions