Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Why not 1*(0.5-0.4-0.1) = 0?
Message
From
06/01/2005 18:47:18
Cetin Basoz
Engineerica Inc.
Izmir, Turkey
 
 
To
06/01/2005 18:29:50
General information
Forum:
Visual FoxPro
Category:
COM/DCOM and OLE Automation
Environment versions
Database:
MS SQL Server
Miscellaneous
Thread ID:
00974881
Message ID:
00974894
Views:
36
Never be surprised when you see I don't know the most basic things. Foxpro hide it from me:)
Yes even in C#.Net result is not 0! Very nice for scientific calculations.
Cetin

>Hi Cetin,
>
>There is no sulution as such. The problem is that NO decimal numbers has an accurate binary value, so it's always a matter of rounding. To get accurate calculations with decimals, you must multiply all the numbers by a 10, 100, 1000 or another power of 10, so that you only calculate the integers (whole numbers) which have accurate binary values. Then you must divede the result with the same factor. I am suprised you did not know that?
>
>>A friend of mine asked this to me today. I said yes,0 with confidence and he said OK then go and put that in Excel as a formula.
>>I thought it was an Excel bug but he turned and said it happens in SQL server with float datatype! I couldn't yet see that myself but trust his experience.
>>Any idea and solution?
>>TIA
>>Cetin
Çetin Basöz

The way to Go
Flutter - For mobile, web and desktop.
World's most advanced open source relational database.
.Net for foxheads - Blog (main)
FoxSharp - Blog (mirror)
Welcome to FoxyClasses

LinqPad - C#,VB,F#,SQL,eSQL ... scratchpad
Previous
Reply
Map
View

Click here to load this message in the networking platform