Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Rounding Errors
Message
De
02/10/2003 18:07:45
 
 
À
02/10/2003 18:03:12
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Titre:
Divers
Thread ID:
00834520
Message ID:
00834568
Vues:
28
>Fred,
>I discovered this strange behavior a while ago and started thread #830026 "What is a more accurate way to round numbers than round() function". I never got any real solution but the rounding errors were not so significant to be a show stopper. If 533.8050 is the result of a calculation rather than a number entered into a field, variable or constant, you might see this. It doesn't always happen but it does happen and is repeatable.
>
>Try this :
>
y = 533.8050
>?y
>?round(y,2)
>x = (.0001 * 533.8050) /.0001
>?x
>?round(x,2)
>
>
>


Well, that's just the nature of floating point representation in digital computers. There's no way to get it 100% accurate without lots of tricks. My guess would be you just endded up internally with something like 533.8049999999999999999 so it did exactly what it was supposed to do. I guess you could always just add .005 before you do the ROUND().


>
>>>I have a program that uses the round(exp,2) function. It works 99% of the time but there is a situation that has come up that is confusing me. I have a number, 533.8050, that I am trying to round up to 533.81. It seems that it is rounding down. Has anyone else experienced this problem? I could be way off, but I think this is what is happening.
>>>If so, how do you get around it?
>>>Thanks!
>>
>>How exactly are you seing this? Nothing I try (constant, variable, field) with that value rounds incorrectly.
Fred
Microsoft Visual FoxPro MVP

foxcentral.net
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform