Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
SET DECIMAL AND TRANSFORM
Message
From
17/12/1999 17:41:47
 
 
To
17/12/1999 12:35:31
General information
Forum:
Visual FoxPro
Category:
Other
Miscellaneous
Thread ID:
00305350
Message ID:
00305508
Views:
27
This looks like a case of the bug described in article Q164359 at http://support.microsoft.com/support/kb/articles/q164/3/59.asp, "BUG: Incorrect Evaluation of Equation with Variables".

The problem can be avoided by splitting the equation into pieces and applying rounding where appropriate to intermediate results. The code below gives the correct results for set("decimals") from 1 to 8.
clear
for lni = 1 to 8
    do myTest
endfor
RETURN

PROCEDURE myTest
set decimal to lni
? "set('decimals') = " + ALLTRIM( STR( SET("DECIMALS")))
w_PRICE = 3.047
w_SC1 = 0
w_SC2 = 0
*  Replace this line...
*w_VAL = (w_PRICE*(100-w_SC1)/100)*(100-w_SC2)/100
*  ...with these three lines
lnx = round( w_PRICE*(100-w_SC1)/100, 3)
lny = (100-w_SC2)/100
w_VAL = lnx * lny
*  Now your code again...
w_result = round(105.000 * w_VAL,2)
? w_result
? transform(w_result,"999999.999999")
?
>I use SET DECIMAL TO 6 to display with TRANSFORM() number with 6 decimals. I
>have note that "SET DECIMAL TO " modify the result. Why ? The FoxPro
>documentation play:
>
> SET DECIMALS
> Specifies the number of decimal places displayed in numeric
>expressions.
>
>
>Example 1 with SET DECIMAL TO 6
>============================
>set decimal to 6
>w_PRICE = 3.047
>w_SC1 = 0
>w_SC2 = 0
>w_VAL = (w_PRICE*(100-w_SC1)/100)*(100-w_SC2)/100
>w_result = round(105.000 * w_VAL,2)
>? w_result
>? transform(w_result,"999999.999999")
>
>ERROR !!! w_RESULT is 319.93. The correct result is 319.94
>
>
>Example 2 with SET DECIMAL TO 5
>============================
>set decimal to 5
>w_PRICE = 3.047
>w_SC1 = 0
>w_SC2 = 0
>w_VAL = (w_PRICE*(100-w_SC1)/100)*(100-w_SC2)/100
>w_result = round(105.000 * w_VAL,2)
>? w_result
>? transform(w_result,"999999.999999")
>
>OK !!! w_RESULT is 319.94.
>
>
>Thanks,
>Maurizio
Rick Borup, MCSD

recursion (rE-kur'-shun) n.
  see recursion.
Previous
Reply
Map
View

Click here to load this message in the networking platform