Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Why not 1*(0.5-0.4-0.1) = 0?
Message
From
07/01/2005 03:52:11
 
 
To
06/01/2005 23:34:32
Dragan Nedeljkovich
Now officially retired
Zrenjanin, Serbia
General information
Forum:
Visual FoxPro
Category:
COM/DCOM and OLE Automation
Environment versions
Database:
MS SQL Server
Miscellaneous
Thread ID:
00974881
Message ID:
00974985
Views:
39
>>With ANY decimal numbers you will have some rounding errors. Sometimes the programs masks the error, and you get the result you want, and sometimes you get the wrong value, there's simply no way to avoid that.
>
>Not quite correct. The following set of decimals (plus any integer part) has an exact representation in binary: 0.5, 0.25, 0.75, 0.125, 0.375, 0.625, 0.875, ... i.e. those that can be expressed as m/2**n.
Previous
Reply
Map
View

Click here to load this message in the networking platform