Information générale
Catégorie:
Codage, syntaxe et commandes
>>Yeah, I found that out too. My solution is to ROUND my result to 5, then ROUND to whatever I really want, like 2. That seems to work, but sounds buggy to me!
>
>Janet,
>
>It has something to do with the literals you're using. As was mentioned elsewheres, modifying the literals by removing the unnecessary zeroes, produces the correct (and expected) result. If, for example, you were to assign the result of the math to a variable and round the variable, it works correct. Further, I wouldn't think that, in actual usage, you'd use this anyway. If you needed 33.01, then that's what you'd use.
I know what you're saying here makes sense, & as a mathematician & engineer, I would agree there is a difference in meaning between 0.35 & 0.3500000. But as a database programmer, I would expect no difference in value between the two.
It's just another symptom of the inaccuracies inherent in floating point calculations on digital computers, & highlights the need to know how the language of choice represents such numbers.
There are still inconsistencies in operation though, try
r = 0.35 * 943 * .1
r1 = 0.350000000000 * 943 * .1
? r = r1 (displayed result = .T.)
? round( r, 2 ) = round( r1, 2 ) (displayed result = .F.)
so the numeric results are equal, but rounding them leads to a difference ! Surely a bug ?
Paul
Précédent
Suivant
Répondre
Voir le fil de ce thread
Voir le fil de ce thread à partir de ce message seulement
Voir tous les messages de ce thread
Voir tous les messages de ce thread à partir de ce message seulement