Skip to content

DECFLOAT values with 16-31 decimal places decode/encode with the decimal point off by 16 places (SignSpecialPlaces masked with $2f instead of $3f) #3

Description

@mariuz

Summary

Any DECFLOAT value with 16 to 31 decimal places is corrupted in both directions of the TBCD conversion: it is read back (via ISQLData.GetAsBCD) and sent (via ISQLParam.AsBCD) multiplied by 10^16. Values with 0–15 or 32–34 decimal places are unaffected, which makes the bug easy to miss.

Cause

TBCD.SignSpecialPlaces keeps the number of decimal places in bits 0..5 (0..63) and the sign in bit 7, so the mask for the places field must be $3f. Both conversion paths in client/3.0/FB30ClientAPI.pas mask with $2f, dropping bit 4 (value 16):

// SQLDecFloatEncode (line 713)
exp := -(aValue.SignSpecialPlaces and $2f);
// SQLDecFloatDecode (line 800)
Result.SignSpecialPlaces :=  (-exp and $2f);

A places count of 16..31 has bit 4 set, so and $2f subtracts exactly 16 from it — the decimal point lands 16 positions to the right. 32..34 places have bit 4 clear again, which is why DECFLOAT(34) at full precision happens to survive.

Reproduction

Round trips at 8, 15, 16, 17, 22, 31 and 33 decimal places against Firebird 6.0 (LI-T6.0.0.2076, Linux aarch64, fpc 3.2.2), decode via OpenCursorAtStart(...)[0].GetAsBCD + BCDToStr, encode via SQLParams[0].AsBCD := StrToBCD(...) + server-side CAST(? AS VARCHAR(60)):

-- decode: literal -> AsBCD -> BCDToStr --
                   1.234567890123456 ->                     1.234567890123456   OK
                  1.2345678901234567 ->                     12345678901234567   *** WRONG ***
                 1.23456789012345678 ->                   12345678901234567.8   *** WRONG ***
            1.2345678901234567890123 ->              12345678901234567.890123   *** WRONG ***
   1.2345678901234567890123456789012 ->     12345678901234567.890123456789012   *** WRONG ***
 1.234567890123456789012345678901234 ->   1.234567890123456789012345678901234   OK
-- encode: AsBCD param -> server-side CAST to text --
(same pattern)

We hit this while porting a set of Firebird architecture-paper samples to fbintf (https://github.com/mariuz/conceptual-architecture-for-firebird-paper, samples/fpc/); the migration.pas sample demonstrates it live.

Fix

Change both masks to $3f. PR follows: after the fix all fourteen round trips above are exact.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions