Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
What's wrong with VFP 5.0a's math (or is it me)?
Message
De
15/05/1997 18:19:04
 
 
À
15/05/1997 16:27:45
Bill Gravell
Indoor Purification System
Menifee, Californie, États-Unis
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Divers
Thread ID:
00032141
Message ID:
00032506
Vues:
39
>>I have a program with the following line of code:
>>
>>lPayment_Amount = lpct * tpd_payments_lte.weekly_ttd_rt
>>
>>When I run my program and check the debugger, the following values are used:
>>
>>10.8539997 = 0.04998 * 217.18
>>
>>When I ROUND(lPayment_Amount,2) I get 10.85.
>>
>>But when I do the math using the Windows 95 calculator (0.04998 * 217.18) I get 10.8546564 which should round to 10.86.
>>
>>Any ideas on how I can fix this?
>>
>>Thanks!
>>
>>Jeff Rusch
>>UW-Madison
>
>Actually 10.8546564 rounded to 2 decimals does round to 10.85. if you round it to 3 decimals, it rounds to 10.855


This rounding thing is a problem of consisteny else where. Excel would respond with 10.85, quicken say for 10.8546 .. that 6 rounds the 4 to 5, 5 to 6 and responds with 10.86. Not sure who makes sense except that quick matches my bank statement interest and excel doesn't. go figure
Gary
Gary
Helping Make Ideas Reality
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform