Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Inaccurate results when using large numbers
Message
De
12/06/2002 07:37:36
Hilmar Zonneveld
Independent Consultant
Cochabamba, Bolivie
 
 
À
12/06/2002 01:28:34
Paul Dorrington
Dorrodata Computing Pty Ltd
Carlton, Australie
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Divers
Thread ID:
00667226
Message ID:
00667272
Vues:
20
>In our VFP application to support Unit Trusts (Mutual Funds) we have the occassional need to record reasonably large transactions and we note that sometimes VFP appears to get it wrong in the sixth decimal place. (We are required to record 6 decimal places for some clients.)
>
>Have other people experienced this problem and are there any known work arounds other than the obvious one of not using the VFP ROUND() function.
>
>For example:
>
>Buy amount = $800,000,000.00
>Unit Price = 0.938208
>Units issued = 800,000,000.00 / 0.938208 = 852,689,382.311811
>
>In VFP command window 800000000 / 0.9382080 = 852,689,382.311811
>but ROUND(800000000 / 0.9382080,6) = 852,689,382.311812
>and ROUND(800000000 / 0.9382080,7) = 852,689,382.3118114
>
>In Excel 800000000 / 0.9382080 = 852,689,382.311811
>and ROUND(800000000 / 0.9382080,6) = 852,689,382.311811
>
>Any suggestions would be much appreciated.
>
>Thanks
>Paul Dorrington
>(Melbourne, Australia)

My download "math unlimited" can handle calculations of almost any size - but it isn't really recommended for a large amount of calculations, since it is relatively slow.

Hilmar.
Difference in opinions hath cost many millions of lives: for instance, whether flesh be bread, or bread be flesh; whether whistling be a vice or a virtue; whether it be better to kiss a post, or throw it into the fire... (from Gulliver's Travels)
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform