Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
What's wrong with VFP 5.0a's math (or is it me)?
Message
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Divers
Thread ID:
00032141
Message ID:
00032152
Vues:
37
>>I have a program with the following line of code:
>>
>>lPayment_Amount = lpct * tpd_payments_lte.weekly_ttd_rt
>>
>>When I run my program and check the debugger, the following values are used:
>>
>>10.8539997 = 0.04998 * 217.18
>>
>>When I ROUND(lPayment_Amount,2) I get 10.85.
>>
>>But when I do the math using the Windows 95 calculator (0.04998 * 217.18) I get 10.8546564 which should round to 10.86.
>>
>>Any ideas on how I can fix this?
>>
>>Thanks!
>>
>>Jeff Rusch
>>UW-Madison
>
>Jeff,
>
>The problem is with currency data types. They actually operate alot like dates, in that any manipulation on them yields another currency data type. Thus when you multiply by a numeric it is rounding down to the currency precision level, which in many cases is what you want to maintain compliance with currency math.
>
>In your case, use the MTON() function to get the result you desire.
>
>lPayment_Amount = lpct * MTON(tpd_payments_lte.weekly_ttd_rt)
>
>Given your preference here, you may want to consider changing your data structure to numeric or else you will have to make certain your app uses MTON() in lots of places.
>
>Randy

Hi Randy,

Thanks for the response. However, it doesn't appear to apply in this case. Tpd_payments_lte is a remote view of a MS SQL Server table. The remote view properties for the weekly_ttd_rt field is Numeric datatype and the SQL Server field is Decimal datatype. lPayment_Amount and lPCT are both local variables.

I still don't understand how VFP can calculate a totally wrong answer.

Jeff
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform