Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
INT()egering a Numeric type, expected value isn't correc
Message
From
22/06/2000 21:59:53
 
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00383077
Message ID:
00383569
Views:
18
>>>>>We have a field which is set as Numeric type say Amount N(10,4). Value of
>>>>>which was derived from different variables and fields (n1+n2+n3, etc.).
>>>>>Somewhere in the module, it is clear that Amount has a value of 16.0000 being
>>>>>shown in the debugger, but if we try to make use of INT(Amount), it gives us
>>>>>15. Trying to do the same procedure in the command window but this time using
>>>>>variables, it produces the correct result. Is there a hidden anomaly of
>>>>>using a Numeric type?
>>>>
>>>>As mentioned by several others, the answer here is yes. As suggested in Q258998, you might be able to use ROUND(nAmount, 0) instead of INT(nAmount) to get the value you expect.
>>>
>>>Garrett,
>>>
>>>In this case yes:
>>>Round(16.0000,2) gives 16
>>>Int(16.0000) should be 16 right? but it gives 15 which is attributed to what others are saying.
>>>
>>>What if?
>>>Round(16.5555,0) gives 17
>>>Int(16.5555) gives 16
>>>
>>>We just want to get the Integer value.
>>>
>>>What is the best alternative data type over numeric type?
>>
>>
>>Wait a minute, you put in INT(16.0000) and it gives you 15? Even when you use the constant 16.0000? Or am I mis understanding you. I get 16 on my machine...
>
>Try to read the original post at your convenience. The problem doesn't occur in the command window.

Well, actually it does:
x = 3.999999999999999
z = x * 4
? z        && Prints 16.000000000000000
? int(z)   && Prints 15
Fred
Microsoft Visual FoxPro MVP

foxcentral.net
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform