Rob wrote:
^^^
This may cause problems, as we already have at least one regular Rob here
Thank you both very much.
You are welcome.
999999999999999934463 is the lucky number here for me. Truncation of
decimal places doesn't matter as I'm dealing with integers only.
I meant the truncation of binary "decimal" places of the mantissa-exponent
representation of the stored floating-point value. Just follow the
algorithm:
0. Let n be 999999999999999934463.
1. Convert n to binary:
,----------------------------- 70 bits ------------------------------.
N := 1101100011010111001001101011011100010111011110100111101111111111111111
[bc(1) rulez
]
2. Let the mantissa M be 1 <= M < 10 (binary):
,--------------------------- 69 bits -------------------------------.
M := 1.101100011010111001001101011011100010111011110100111101111111111111111
^[1]
E := 1000101 (69d)
3.1 Ignore the "1." to allow for greater precision:
,--------------------- 52 bits --------------------.
M := 101100011010111001001101011011100010111011110100111101111111111111111
These are 69 of available 52 bits for the mantissa M. Therefore,
3.2. Truncating the "binary" decimal places[^1]
leads to
S := 0
E := 10001 (17d) + bias
M := (1)1011000110101110010011010110111000101110111101001111
Therefore, the actual binary value stored is
1101100011010111001001101011011100010111011110100111100000000000000000
and the actual decimal value stored is
999999999999999868928(d)
^^
which is displayed rounded by .toString() as
999999999999999900000
^^^^^
Now compare with the intended value:
999999999999999934463
^^^^^
The difference to the intended value is 65535 when stored, 34463 when
displayed. Most certainly that does matter here, even if you are only
dealing with integers. I thought that would be clear to you already
by VK mentioning it correctly several times in this thread.
PointedEars