Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Rounding Errors
Message
From
02/10/2003 18:07:45
 
 
To
02/10/2003 18:03:12
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Title:
Miscellaneous
Thread ID:
00834520
Message ID:
00834568
Views:
27
>Fred,
>I discovered this strange behavior a while ago and started thread #830026 "What is a more accurate way to round numbers than round() function". I never got any real solution but the rounding errors were not so significant to be a show stopper. If 533.8050 is the result of a calculation rather than a number entered into a field, variable or constant, you might see this. It doesn't always happen but it does happen and is repeatable.
>
>Try this :
>
y = 533.8050
>?y
>?round(y,2)
>x = (.0001 * 533.8050) /.0001
>?x
>?round(x,2)
>
>
>


Well, that's just the nature of floating point representation in digital computers. There's no way to get it 100% accurate without lots of tricks. My guess would be you just endded up internally with something like 533.8049999999999999999 so it did exactly what it was supposed to do. I guess you could always just add .005 before you do the ROUND().


>
>>>I have a program that uses the round(exp,2) function. It works 99% of the time but there is a situation that has come up that is confusing me. I have a number, 533.8050, that I am trying to round up to 533.81. It seems that it is rounding down. Has anyone else experienced this problem? I could be way off, but I think this is what is happening.
>>>If so, how do you get around it?
>>>Thanks!
>>
>>How exactly are you seing this? Nothing I try (constant, variable, field) with that value rounds incorrectly.
Fred
Microsoft Visual FoxPro MVP

foxcentral.net
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform