A number like 0.644696875 is represented internally as 0.6446968749999...:
N[FromDigits[RealDigits[0.644696875, 2], 2], $MachinePrecision]
(* 0.6446968749999999 *)
So if I ask NumberForm
to print this number with 8 decimals, I would expect it to be 0.64469687 and not 0.64469688 because the digit after the 87 is a 4 which is less than 5. But it is not what we get with NumberForm
:
NumberForm[0.644696875, {8, 8}]
(* 0.64469688 *)
So it looks like we have here two round operations when only one was requested:
- First Rounding: From 0.6446968749999999 to 0.644696875
- Second Rounding: From 0.644696875 to 0.64469688
I found this while comparing to Python, which doesn't double round. This leads to a result which I believe is correct:
ExternalEvaluate["Python", "'{:.8f}'.format(0.644696875)"]
(* 0.64469687 *)
Also notice that this floating point number is stored in the same way in both systems:
Divide @@ ExternalEvaluate["Python", "0.644696875.as_integer_ratio()"] == FromDigits[RealDigits[0.644696875, 2], 2]
(* True *)
Is Mathematica double rounding? Can this be avoided while still using machine numbers?
Motivation: I'm working on making a hash on an array of floating point numbers. This calculation should be the same from Mathematica and from Python. For this I need to be able to produce the same string representation of numbers in both system. This has proven more challenging than expected.
Update: I think Mathematica is double rounding just like Java does. Please see:
Update2: I asked support about this [CASE:4304365] and they said "It does appear that NumberForm is not behaving properly".
Comments
Post a Comment