Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
What is a more accurate way to round numbers than Round(
Message
De
17/09/2003 17:09:51
 
 
À
17/09/2003 15:12:58
Hilmar Zonneveld
Independent Consultant
Cochabamba, Bolivie
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Divers
Thread ID:
00830026
Message ID:
00830114
Vues:
26
Hilmar,

All fields in the source table and resulting summary tables are N(18,6), and all variables check out to be type "N". NO currency data types to be found. Using VFP6 SP5.

I just find it puzzling that manually setting the variables to (supposedly) the same value as that returned by sum yields different results when using round() function. It must have something to do with the way that the numbers are internally calculated.
clear
Set decimals to 4
select EMISSIONUNITREPORT2
sum EMISBYAVG to m.lnTotalAVG
      *!* result = 58271.270050
select EMISSIONUNITREPORT3
sum EMISBYAVG to m.lnTotalAVG1
      *!* result = 58271.270050

?m.lnTotalAVG - m.lnTotalavg1
     *!* result = 0.000000

?Type("m.lnTotalAVG")
     *!* result = "N"

?Type("m.lnTotalAVG1")
     *!* result = "N"

?str(m.lnTotalAVG,30,18) 
     *!* result = "58271.270049999400000000"

?str(m.lnTotalAVG,30,18) 
      *!* result = 58271.270049999900000000

*!* this is how my test program was checking each total
? round(m.lnTotalAVG,4) = round(m.lnTotalAVG1,4)
      *!* result = .f.
? round(m.lnTotalAVG,4)
      *!* result = 58271.2700
? round(m.lnTotalAVG1,4)
      *!* result = 58271.2701  && Wrong!!


*!* now manually set to same values as returned by sum

m.lnTotalAVG = 58271.270049999400000000
?m.lnTotalAVG
     *!* result = 58271.270049999400000000

m.lnTotalAVG1 = 58271.270049999900000000
?m.lnTotalAVG1
      *!* result = 58271.270049999900000000

?Type("m.lnTotalAVG")
     *!* result = "N"

?Type("m.lnTotalAVG1")
     *!* result = "N"

? round(m.lnTotalAVG,4) = round(m.lnTotalAVG1,4)
      *!* result = .T. && ROUND WORKS AS EXPECTED HERE
? round(m.lnTotalAVG,4)
      *!* result = 58271.2700
? round(m.lnTotalAVG1,4)
      *!* result = 58271.2700
>Perhaps you can check whether any of the intermediate numbers are of type currency?
>
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform