Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
SET DECIMALS TO is used if SET FIXED is OFF ?
Message
Information générale
Forum:
Visual FoxPro
Catégorie:
Autre
Divers
Thread ID:
00850956
Message ID:
00852507
Vues:
10
Garrett,

Not smoking anything my friend... those rules were drilled into my psyche during the 6 years I spend at the University of Missouri-Rolla studying nuclear engineering and computer science. It's a pretty fundamental rule about significant digits in calculations. 3.1 is not the same as 3.1000. Every number written has an implicit error of ± 1/2 the last significant digit. So 3.1 * 2.5567 is 7.9 ± 0.05. 3.1000 * 2.5567 is 7.9258 ± 0.00005. You need to forget the way your grade school teacher taught you to do arithmetic. *g*

>>If you have 5 significant digits in your operands (ie 2.5567), it is incorrect to expect
>>that calculated results should have more signifcant digits than the operands. When you
>>multiply 3.1 * 2.5567 the actual result should only have 2 significant digits because it
>>is controlled by the operand with the fewest significant digtis and the result should be 7.9.
>
>Mi dispiace, signore, ma what the hell have you been smoking? :-)
>
>
 2.5567
>   x3.1
> ------
> 2 5567
>76 701
>-------
>7.92577
df (was a 10 time MVP)

df FoxPro website
FoxPro Wiki site online, editable knowledgebase
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform