Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Why not 1*(0.5-0.4-0.1) = 0?
Message
From
07/01/2005 15:59:51
 
 
To
06/01/2005 18:19:49
Cetin Basoz
Engineerica Inc.
Izmir, Turkey
General information
Forum:
Visual FoxPro
Category:
COM/DCOM and OLE Automation
Environment versions
Database:
MS SQL Server
Miscellaneous
Thread ID:
00974881
Message ID:
00975255
Views:
63
>A friend of mine asked this to me today. I said yes,0 with confidence and he said OK then go and put that in Excel as a formula.
>I thought it was an Excel bug but he turned and said it happens in SQL server with float datatype! I couldn't yet see that myself but trust his experience.
>Any idea and solution?
>TIA
>Cetin

WARNING : I suggest to read with attention this message to anyone !

Cetin, this is the "associative rule" !( remember Age computation ? )

Try this on Excel:
=(0.5-0.4-0.1) => -2,77556E-17
=(0.5-0.1-0.4) => 0  && expected
* next is more interesting
=0.5-0.4-0.1   => 0  && not expected
Solution ( partial ! ) is Round() or use a numeric set where associative rule is valid.

Is simple to replicate it in VFP:
CLEAR
SET DECIMALS TO 0
SET FIXED OFF
Y=0.5
? TRANSFORM(M.Y-0.4-0.1,"@^")       && <=== HERE YOU SEE THE SAME -2,7755575615629E-17
? M.Y-0.4-0.1                       && <=== HERE VFP APPLY A GOOD ROUND() and print 0


* NEXT SHOW BECAUSE I WANTED TO ABANDON VFP 

X=M.Y-0.4-0.1   && X is zero ?
? M.X				&& VFP print zero, but you cannot say that x is zero
* with TRANSFORM(,"@^") you print the true value
? TRANSFORM(M.X,"@^") && x = -2,7755575615629E-17

* another proof: EMPTY() have a bug, and not ROUND() the variable before to check for zero

? EMPTY(m.x)  && OF COURSE .F. !

* DEFINITIVE PROOF IN VFP9:

* ON IEEE FLOAT 0 IS
?  0h+BINTOC(0.0,"B") &&  0h0000000000000000

* NOW RUN THIS INTO MS SQL AMALYZER :  SELECT CAST((0.5E0-0.4-0.1) AS BINARY(8)) 
* result is 0xBC80000000000000

* !!!!!! IMPORTANT HERE YOU SEE THE TRUE VALUE FOR X, AND IT IS EQUAL TO -2,7755575615629E-17
* the output is specular regarding to CAST of MSSQL

? 0h+BINTOC(m.x,"B")                && 0h00000000000080BC

* NEXT two line show another way to print not zero directly,
* BUT THE IMPORTANT THING IS THAT X is not ZERO !

? trans(0.5-0.4-0.1,"@^")		    && HERE VFP APPLY A GOOD ROUND(,)
? trans(0.50000000000-0.4-0.1,"@^") && HERE VFP APPLY A BAD  ROUND(,) 
Cetin, no software bugs, no cpu bugs,
only computation into a set where + and * don't respect the normal arithmetic rules.
The computation on Excel/C#/SQL/VFP are the same, ONLY THE output is different.

Fabio
Previous
Reply
Map
View

Click here to load this message in the networking platform