Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
What's wrong with VFP 5.0a's math (or is it me)?
Message
 
To
15/05/1997 18:19:04
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00032141
Message ID:
00032535
Views:
36
>>>I have a program with the following line of code:
>>>
>>>lPayment_Amount = lpct * tpd_payments_lte.weekly_ttd_rt
>>>
>>>When I run my program and check the debugger, the following values are used:
>>>
>>>10.8539997 = 0.04998 * 217.18
>>>
>>>When I ROUND(lPayment_Amount,2) I get 10.85.
>>>
>>>But when I do the math using the Windows 95 calculator (0.04998 * 217.18) I get 10.8546564 which should round to 10.86.
>>>
>>>Any ideas on how I can fix this?
>>>
>>>Thanks!
>>>
>>>Jeff Rusch
>>>UW-Madison
>>
>>Actually 10.8546564 rounded to 2 decimals does round to 10.85. if you round it to 3 decimals, it rounds to 10.855
>
>
>This rounding thing is a problem of consisteny else where. Excel would respond with 10.85, quicken say for 10.8546 .. that 6 rounds the 4 to 5, 5 to 6 and responds with 10.86. Not sure who makes sense except that quick matches my bank statement interest and excel doesn't. go figure
>Gary

If you want to round to upper x/100, try this:
ceiling(10.8546564*100)/100

:-)
If we exchange an apple, we both get an apple.
But if we exchange an idea, we both get 2 ideas, cool...


Gérald Santerre
Independant programmer - internet or intranet stuff - always looking for contracts big or small :)
http://www.siteintranet.qc.ca
Previous
Reply
Map
View

Click here to load this message in the networking platform