Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
INT()egering a Numeric type, expected value isn't correc
Message
From
23/06/2000 08:40:24
 
 
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00383077
Message ID:
00383675
Views:
16
Jesse, Fred:

Try this in VBA:
Dim x As Double
x = 3.999999999999999
MsgBox x
MsgBox Int(x)
and you will get the same results as in VFP. In fact, if you do it in the Word VBA (I have not tried with the other Office programs), the line x = 3.999999999999999 will display as x=4#. As I tried to explain in an earlier post, this is due to the fact that most fractions (3999999999999999 / 1000000000000000 in this case) don't have an finite binary representation. The "problem" is that computer works with 0 and 1, not 0,1,2,..9 and thus you are tempted to believe that 3.999999999999999 has an finite representation (because it has an finite number of digits after the decimnals) when in fact it doesn't. Truth is, the binary representation of that number represents a range of number.

Hope I am more clear this time.
Daniel
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform