Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Rounding problem?
Message
From
22/03/2000 00:21:57
 
 
To
20/03/2000 17:03:26
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00348122
Message ID:
00348729
Views:
23
>It is interesting that just dropping off one of the zeroes from one of the three numbers will give you the correct answer, for example:
ROUND(.35000000 * 943.0 *.100 ,2) = 33.00
>ROUND(.3500000 * 943.0 * .100, 2) = 33.01
>ROUND(.35000000 * 943. * .100, 2) = 33.01
>ROUND(.35000000 * 943.0 * .10, 2) = 33.01
Just to make it even more interesting, these results are affected by the value of SET DECIMALS. For SET DECIMALS from 0 to 11 the results are as you show them. For SET DECIMALS of 12 and up, all four expressions yield 33.00.

There is a documented bug affecting the ROUND() function. See KB # Q157954 and my reply to George Tasker elsewhere in this thread.
Rick Borup, MCSD

recursion (rE-kur'-shun) n.
  see recursion.
Previous
Reply
Map
View

Click here to load this message in the networking platform