Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
SET DECIMALS TO is used if SET FIXED is OFF ?
Message
 
General information
Forum:
Visual FoxPro
Category:
Other
Miscellaneous
Thread ID:
00850956
Message ID:
00852663
Views:
17
Sergey,

This should be correct now:

3.1000 ± 0.00005 * 2.5567 ± 0.00005 = 7.9258 ± 0.00020

The error is calculated with:

? abs( 3.1000 * 2.5567 ) * sqrt( ( 0.00005 / 3.1000 ) ^ 2 + ( 0.00005 / 2.5567 ) ^ 2 )

Reference this article: http://www.stanford.edu/class/engr1n/Precision_E1.pdf

Thanks for catching my mistake.

>That's not what I was refering to. Are you sure that imlicit error in the result of following calculation is 0.00005 not bigger?
>
>3.1000 ± 0.00005 * 2.5567 ± 0.00005 is 7.9258 ± 0.00005
>
>P.S. I maybe mistaken but I'm to leazy to go and read books on the subject. :-)
df (was a 10 time MVP)

df FoxPro website
FoxPro Wiki site online, editable knowledgebase
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform