Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Rounding problem?
Message
 
To
21/03/2000 11:48:45
Paul Frost
Instem Computer Systems Ltd
Stone, United Kingdom
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00348122
Message ID:
00348442
Views:
33
>I agree that it's not the best way to do such calculations (though for some calculations, setting it out in such a manner, clarifies what is being done, without the need for explicit comments).
>
>But it is a matter of some concern that
>a = 0.3500000000
>b = 943
>c = .1
>round( a * b * c, 2 ) # round( 0.3500000000 * 943 * .1, 2 )
>
>e.g. where else are such little discrepancies hidden, should all use of literals in expressions be avoided ?

As a general rule, probably yes. That's a generalization. There may be specific need to do so.

>A lot of the code I produce is used in calculating duty payable to Customs & Excise, from calculations similar to these. All of the calculations use variables, so no (apparent) problem. If checks were performed on my results though, two seemingly identical calculations, as above, returning different results could seriously undermine the clients view of the reliability of the entire system.
>
True, but I think highly unlikely. There are a number of cases where rounding produces slight decrepancies.

Let me just throw one more thing to you and Janet. To the best of my knowledge, this is not a problem with the ROUND() function specifically. This is a problem with how the expression is evaluated. This is because the evaluation occurs before the function is called and not within the function itself. Now where the real source of the problem is, I don't know. I mentioned VFP and the FPU before, but let's add the C compiler as well. Afterall, that's what's "under the hood".< g >
George

Ubi caritas et amor, deus ibi est
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform