Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
INT()egering a Numeric type, expected value isn't correc
Message
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00383077
Message ID:
00383347
Views:
16
Hi Jess,

>Somewhere in the module, it is clear that Amount has a value of 16.0000 being shown in the debugger, but if we try to make use of INT(Amount), it gives us 15.

Like in?
Set Decimals to 18
nVal = 2.05*100
? nVal      && 205.000000000000000000
? Int(nVal) && 204 
That's because when numeric values are converted to floating point variables, some values that are simple values in the decimal system, have a repeating fraction when converted into the form mantissa*2^exponent.

Christof
--
Christof
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform