Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Rounding Errors
Message
De
02/10/2003 19:53:10
 
 
À
02/10/2003 19:26:42
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Titre:
Divers
Thread ID:
00834520
Message ID:
00834603
Vues:
27
Thanks Fred,

I used set decimals to 18 since ?str(x,25,18) is valid but ?str(x,25,19) generates an error "Width or decimal place argument is invalid" and set decimals to 19 generates a syntax error. I assumed that 18 was the maximum setting that I could use. It seems that round only gets it wrong when rounding the result of a calculation and then it is dependent on the number and the calculation used to get there and the number of decimal places rounded. I can't say that I totally understand it, and I know it doesn't occur with rounding every calculated value, but it does happen so work around it when I have to.

Elmer

>Great example, Elmer. Far better an explanation then I could give on this topic, that's for sure.
>
>Just a note about your SET DECIMALS TO 18. Since VFP internally only support about 15 or 16 digits (total) of precision, there's no way it could ever give you a correct answer to that accuracy. Even with your setting of 4 in your first example, it's merely a "rounded" view of the internal representation. The error must occur somewhere out there past the 12 decimals shown. I get this when I simply add a trailing 0 to one of your calculations causing a 13 digit decimal:
>
>x = (.0001 * 533.8050) /.00010   && 533.8049999999999
>
>That maxes out to 16 digits. If you'll notice with your 18 digits, all you see is 0's past the 13th decimal place.
>
>It is interesting that if you apply your formula to 33.8050, it doesn't seem to matter how many trailing 0's you add, the number is still just 33.805000000000000000. I can't seem to get it generate any more digits in the result and it does round it correctly.
>
>
>>Hi Fred,
>>
>>No big deal, I just thought I would point out, for those who don't know, that round() function will return incorrect results when the value being rounded is the result of a calculation. I understand and accept that it is due to the nature of computers and floating point calculation, but checking the value of x after that sample calculation returns 533.805000000000, yet round(x,2) returns 533.80. That just seems a little flaky to me but for my purposes, it has not been a real problem. To me ?x should return 533.8049999999999 if that is what the internal calculated value came out to be. Then I would at once see why round(x,2) did not return the result that I expected.
>>
>>Another little oddity with the example calculation and to see why rounding errors occur:
>>
set decimals to 4
>>x = (.0001 * 533.8050) /.0001
>>?x                          && returns 533.805000000000
>>?round(x,2)                 && returns 533.80
>>?val(str(x,25,18))          && returns 533.8050
>>?round(val(str(x,25,18)),2) && returns 533.81 which is the "correct" result.
>>*!* to see the actual calculated result of (.0001 * 533.8050) /.0001
>>set decimals to 18
>>x = (.0001 * 533.8050) /.0001
>>?x                          && returns 533.804999999999900000
>>?round(x,2)                 && returns 533.80
>>?val(str(x,25,18))          && returns 533.804999999999900000
>>?round(val(str(x,25,18)),2) && returns 533.80 which is the "incorrect" result.
>>
>>
>>For someone who is not aware of this behavior, it can be more than a little confusing, especially when he hears "Nothing I try (constant, variable, field) with that value rounds incorrectly". Rounding on a calculated value will frequently return errors in rounding.
>>
>>
>>
>>
>>
>>
>>>>Fred,
>>>>I discovered this strange behavior a while ago and started thread #830026 "What is a more accurate way to round numbers than round() function". I never got any real solution but the rounding errors were not so significant to be a show stopper. If 533.8050 is the result of a calculation rather than a number entered into a field, variable or constant, you might see this. It doesn't always happen but it does happen and is repeatable.
>>>>
>>>>Try this :
>>>>
y = 533.8050
>>>>?y
>>>>?round(y,2)
>>>>x = (.0001 * 533.8050) /.0001
>>>>?x
>>>>?round(x,2)
>>>>
>>>>
>>>>
>>>
>>>
>>>Well, that's just the nature of floating point representation in digital computers. There's no way to get it 100% accurate without lots of tricks. My guess would be you just endded up internally with something like 533.8049999999999999999 so it did exactly what it was supposed to do. I guess you could always just add .005 before you do the ROUND().
>>>
>>>
>>>>
>>>>>>I have a program that uses the round(exp,2) function. It works 99% of the time but there is a situation that has come up that is confusing me. I have a number, 533.8050, that I am trying to round up to 533.81. It seems that it is rounding down. Has anyone else experienced this problem? I could be way off, but I think this is what is happening.
>>>>>>If so, how do you get around it?
>>>>>>Thanks!
>>>>>
>>>>>How exactly are you seing this? Nothing I try (constant, variable, field) with that value rounds incorrectly.
Précédent
Répondre
Fil
Voir

Click here to load this message in the networking platform