Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Rounding problem?
Message
 
To
21/03/2000 11:17:43
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00348122
Message ID:
00348419
Views:
19
>Well, asigning to a variable doesn't exactly work:
>
>a = .35000000 * 943.0 *.100
>ROUND(a,2) = 33.00
>
>BUT
>a = 33.0050000000000
>ROUND(a,2) = 33.01
>
>My values to multiply are pulled from a table, hence the number of decimals used, and rounding each number first can't be done since I don't want to lose any of the decimal values. I'm going with the rounding the result to 5 decimals and then rounding to 2 which seems to work for my situation.

You're right, but I still think it's a needless concern. The following math, which is far more likely to occur works fine

a = 35000000 / 100000000
b = 9430 / 10
c = 100 / 1000
lnval = a * b * c
? round(lnval, 2) && 33.01

As I said in the other thread, the literals may be forcing the value beyond the scope of a double. However, whatever you feel comfortable with is fine with me.
George

Ubi caritas et amor, deus ibi est
Previous
Reply
Map
View

Click here to load this message in the networking platform