When do you get 3.99999999996? When just :
?INT(nX / nY),
or when trying to store in the table?
>I have come across a problem in VFP 6. Maybe its just my machine but I thought I would post it.
>
>Given two values nX and nY where nX = 40 and nY = 10, compute the following: INT(nX / nY)
>
>With SET DECIMALS at anything less than or equal to 10, the answer is correct (4). With SET DECIMALS at 11 or higher, I get the result 3.99999999996 with the 6 repeating based on number of decimals.
>
>My computer:
>Pentium 4 2.4Ghz with 512M memory
>Nvidia G-Force4 Ti 4600 video card
>VFP 6 - SP5
>
>Too weird for me!
>Anybody have any thoughts on this
>
>Todd Cottengim
>Alpine Land Information Services
>Anderson, CA
Précédent
Suivant
Répondre
Voir le fil de ce thread
Voir le fil de ce thread à partir de ce message seulement
Voir tous les messages de ce thread
Voir tous les messages de ce thread à partir de ce message seulement