When do you get 3.99999999996? When just :
?INT(nX / nY),
or when trying to store in the table?
>I have come across a problem in VFP 6. Maybe its just my machine but I thought I would post it.
>
>Given two values nX and nY where nX = 40 and nY = 10, compute the following: INT(nX / nY)
>
>With SET DECIMALS at anything less than or equal to 10, the answer is correct (4). With SET DECIMALS at 11 or higher, I get the result 3.99999999996 with the 6 repeating based on number of decimals.
>
>My computer:
>Pentium 4 2.4Ghz with 512M memory
>Nvidia G-Force4 Ti 4600 video card
>VFP 6 - SP5
>
>Too weird for me!
>Anybody have any thoughts on this
>
>Todd Cottengim
>Alpine Land Information Services
>Anderson, CA
Previous
Next
Reply
View the map of this thread
View the map of this thread starting from this message only
View all messages of this thread
View all messages of this thread starting from this message only