Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
A big problem: how I can compute sum and round result
Message
From
15/11/2003 12:12:31
 
 
To
All
General information
Forum:
Visual FoxPro
Category:
Other
Title:
A big problem: how I can compute sum and round result
Miscellaneous
Thread ID:
00850264
Message ID:
00850264
Views:
61
HI,

On Thread #842819 a ROUND() problem is discussed.

No, i start by a simple two fields values, and want compute the sum of they product;
the standard precision is set to 11.
SET DECIMALS TO 11
SET FIXED ON
CREATE CURSOR orderRows (quantity N(10,1),weight N(9,2))
INSERT INTO orderRows VALUES (3.9,55.15)

CALCULATE SUM(quantity*weight) TO calcTotalWeight
? ROUND(calcTotalWeight,2)
SELECT SUM(quantity*weight) totalweight FROM orderRows INTO ARRAY selTotalWeight
? ROUND(selTotalWeight,2)

? 'NO ROUND  : ', calcTotalWeight,selTotalWeight,IIF(calcTotalWeight=selTotalWeight,' EQUAL',' NOT EQUAL')

FOR k=14 TO 1 STEP -1
 ? TRANS(m.k,'@R ROUND  99  : '), ROUND(calcTotalWeight,m.k),ROUND(selTotalWeight,m.k),IIF(ROUND(calcTotalWeight,m.k)=ROUND(selTotalWeight,m.k),' EQUAL',' NOT EQUAL')
next

? (calcTotalWeight-selTotalWeight)*1024*1024*1024*1024*1024 && binary shift the difference 
I put two questions ( i known the answer ):
First problem:
Why CALCULATE and SELECT return two different values ?

Seconds problem:
Why ROUND(,2) of the results are different ?

I put another question ( i donnot known the answer ):
I can use VFP for do computations and to trust me of the results?

If for any use of VFP language i need do complete analysis of all the process of calculation, I have lost all the advantage to use a top level language.

Thanks

Fabio
Next
Reply
Map
View

Click here to load this message in the networking platform