Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
SET DECIMALS TO is used if SET FIXED is OFF ?
Message
 
General information
Forum:
Visual FoxPro
Category:
Other
Miscellaneous
Thread ID:
00850956
Message ID:
00852713
Views:
18
David,

Thank you for taking time to check it. :-)

>Sergey,
>
>This should be correct now:
>
>3.1000 ± 0.00005 * 2.5567 ± 0.00005 = 7.9258 ± 0.00020
>
>The error is calculated with:
>
>? abs( 3.1000 * 2.5567 ) * sqrt( ( 0.00005 / 3.1000 ) ^ 2 + ( 0.00005 / 2.5567 ) ^ 2 )
>
>Reference this article: http://www.stanford.edu/class/engr1n/Precision_E1.pdf
>
>Thanks for catching my mistake.
>
--sb--
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform