Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Rounding Errors
Message
De
02/10/2003 19:09:24
 
 
À
02/10/2003 18:07:45
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Titre:
Divers
Thread ID:
00834520
Message ID:
00834590
Vues:
27
Hi Fred,

No big deal, I just thought I would point out, for those who don't know, that round() function will return incorrect results when the value being rounded is the result of a calculation. I understand and accept that it is due to the nature of computers and floating point calculation, but checking the value of x after that sample calculation returns 533.805000000000, yet round(x,2) returns 533.80. That just seems a little flaky to me but for my purposes, it has not been a real problem. To me ?x should return 533.8049999999999 if that is what the internal calculated value came out to be. Then I would at once see why round(x,2) did not return the result that I expected.

Another little oddity with the example calculation and to see why rounding errors occur:
set decimals to 4
x = (.0001 * 533.8050) /.0001
?x                          && returns 533.805000000000
?round(x,2)                 && returns 533.80
?val(str(x,25,18))          && returns 533.8050
?round(val(str(x,25,18)),2) && returns 533.81 which is the "correct" result.
*!* to see the actual calculated result of (.0001 * 533.8050) /.0001
set decimals to 18
x = (.0001 * 533.8050) /.0001
?x                          && returns 533.804999999999900000
?round(x,2)                 && returns 533.80
?val(str(x,25,18))          && returns 533.804999999999900000
?round(val(str(x,25,18)),2) && returns 533.80 which is the "incorrect" result.
For someone who is not aware of this behavior, it can be more than a little confusing, especially when he hears "Nothing I try (constant, variable, field) with that value rounds incorrectly". Rounding on a calculated value will frequently return errors in rounding.






>>Fred,
>>I discovered this strange behavior a while ago and started thread #830026 "What is a more accurate way to round numbers than round() function". I never got any real solution but the rounding errors were not so significant to be a show stopper. If 533.8050 is the result of a calculation rather than a number entered into a field, variable or constant, you might see this. It doesn't always happen but it does happen and is repeatable.
>>
>>Try this :
>>
y = 533.8050
>>?y
>>?round(y,2)
>>x = (.0001 * 533.8050) /.0001
>>?x
>>?round(x,2)
>>
>>
>>
>
>
>Well, that's just the nature of floating point representation in digital computers. There's no way to get it 100% accurate without lots of tricks. My guess would be you just endded up internally with something like 533.8049999999999999999 so it did exactly what it was supposed to do. I guess you could always just add .005 before you do the ROUND().
>
>
>>
>>>>I have a program that uses the round(exp,2) function. It works 99% of the time but there is a situation that has come up that is confusing me. I have a number, 533.8050, that I am trying to round up to 533.81. It seems that it is rounding down. Has anyone else experienced this problem? I could be way off, but I think this is what is happening.
>>>>If so, how do you get around it?
>>>>Thanks!
>>>
>>>How exactly are you seing this? Nothing I try (constant, variable, field) with that value rounds incorrectly.
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform