Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Why not 1*(0.5-0.4-0.1) = 0?
Message
 
À
06/01/2005 19:04:56
Cetin Basoz
Engineerica Inc.
Izmir, Turquie
Information générale
Forum:
Visual FoxPro
Catégorie:
COM/DCOM et OLE Automation
Versions des environnements
Database:
MS SQL Server
Divers
Thread ID:
00974881
Message ID:
00974913
Vues:
31
>>>A friend of mine asked this to me today. I said yes,0 with confidence and he said OK then go and put that in Excel as a formula.
>>>I thought it was an Excel bug but he turned and said it happens in SQL server with float datatype! I couldn't yet see that myself but trust his experience.
>>>Any idea and solution?
>>>TIA
>>
>>Cetin,
>>
>>How're doing? We haven't communicated in a long time.
>>
>>I just have a SWAG, but I think it might be right. I tested this will both Excel and VFP 9.0 and got completely different results. IOW, Excel returned false and VFP, correctly, returned true.
>>
>>The only thing that I can think of that might explain it is the Pentium class process floating point bug.
>>
>>In Excel, when I entered the equation, 1*(0.5-0.4-0.1), it returned -2.77556E-17. Naturally, this isn't equal to zero.
>>
>>Sorry, Cetin, that's all I've got. BTW, Congrats on being a "UT Personality".:-)
>
>Hi George,
>I'm fine and busy all day:)

I expected as much.:-)

>I initially thought it as common roundation error but like you tested with VFP anyway. VFP5-8 returns it as a 0, C#, excel as that -2.7...E-17 value. Interestingly there are smaller numbers in float range close to 0.
>
>Thanks:)

You're welcome, but from you've said, it appears to be that .NET still has the same problems with rounding. Might be something to investigate.
George

Ubi caritas et amor, deus ibi est
Précédent
Répondre
Fil
Voir

Click here to load this message in the networking platform