Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Why not 1*(0.5-0.4-0.1) = 0?
Message
 
 
To
07/01/2005 04:04:56
General information
Forum:
Visual FoxPro
Category:
COM/DCOM and OLE Automation
Environment versions
Database:
MS SQL Server
Miscellaneous
Thread ID:
00974881
Message ID:
00975246
Views:
51
Tore,

And which is why in numerical analysis code all over the world you see statements like:
if ( abs( x - y ) <= epsilon )
   the two values are equal
else
   the two values are different
endif
Where epsilon is some "small" constant value like 1 x 10-6

>What I am saying, is that any integer has a defined binary value, like 0=0, 1=1, 2=10, 3=11, 4=100 and so on. So for integers they don't need a 100 pages standard. For decimals it does not work that way, so they have different standard, like IEEE standards, to define how to express them binary. I am far from an expert on this issue, but I know that no matter how you convert a decimal into binary, there is room for errors. On some systems you can specify the accuracy, let's say to 20 decimals, but internally most (all?) of those systems then simply multilpy the values by 10^20, and divide the result by 10^20. You don't overcome the problem completely, but for most calculations you will get the mathematically correct value.
df (was a 10 time MVP)

df FoxPro website
FoxPro Wiki site online, editable knowledgebase
Previous
Reply
Map
View

Click here to load this message in the networking platform