Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Why not 1*(0.5-0.4-0.1) = 0?
Message
 
To
06/01/2005 18:19:49
Cetin Basoz
Engineerica Inc.
Izmir, Turkey
General information
Forum:
Visual FoxPro
Category:
COM/DCOM and OLE Automation
Environment versions
Database:
MS SQL Server
Miscellaneous
Thread ID:
00974881
Message ID:
00974900
Views:
41
>A friend of mine asked this to me today. I said yes,0 with confidence and he said OK then go and put that in Excel as a formula.
>I thought it was an Excel bug but he turned and said it happens in SQL server with float datatype! I couldn't yet see that myself but trust his experience.
>Any idea and solution?
>TIA

Cetin,

How're doing? We haven't communicated in a long time.

I just have a SWAG, but I think it might be right. I tested this will both Excel and VFP 9.0 and got completely different results. IOW, Excel returned false and VFP, correctly, returned true.

The only thing that I can think of that might explain it is the Pentium class process floating point bug.

In Excel, when I entered the equation, 1*(0.5-0.4-0.1), it returned -2.77556E-17. Naturally, this isn't equal to zero.

Sorry, Cetin, that's all I've got. BTW, Congrats on being a "UT Personality".:-)
George

Ubi caritas et amor, deus ibi est
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform