Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
INT()egering a Numeric type, expected value isn't correc
Message
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00383077
Message ID:
00383647
Views:
16
>
>x = 3.999999999999999
>z = x * 4
>? z        && Prints 16.000000000000000
>? int(z)   && Prints 15
>
>
>In the above it is the ? x that is wrong not the INT(x). x is not 16, but is close to but below 16. INT(x) does exactly what it should do which is to return the integer portion of the value regardless of the decimal portion.
>
>This is a rounding error that is always going to be a problem.

OK, but just how are we supposed to recognize this particular circumstance ahead of time? All of the tools available within VFP show the value of z as 16. DISPLAY MEMORY shows the same as above and TRANSFORM turns it into a "16". So just what value is INT() using? I know, I know, its 15.99999999999x but still, it would seem to be a problem somewhere, even if it's just a presentation dicrepancy. If INT() doesn't see it as 16, then why is the translation to a character string showing it that way?
Fred
Microsoft Visual FoxPro MVP

foxcentral.net
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform