Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Round() giving different results
Message
From
12/12/2000 15:35:32
 
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00452446
Message ID:
00452490
Views:
20
>If I do the following two commands I get different results. Am I missing something?
>
>?round(2390.00*.072500,2)
>173.28
>
>?round(2390.000000*.072500,2)
>173.27


I just tried it and I think you have probably discovered a bug. I'm guessing it has to do with the fact that your second example has 4 more significant digits in the first operand and this causes the underlying software to add greater precision to its arithmetic algorithm...and somewhere there's a little bug in the conversion.

Notice that if you don't round, VFP produces a different number of significant digits in the console result. Also, I noticed that VFP puts the results of the rounded versions in different places on the screen. To me, this is all an indication that some threshold is being crossed in terms of the size of the internal representations of the operands and results, and there's a problem with the rounding function (my guess....the step in the rounding function that adds 5 to the least significant digit is adding it to the wrong digit!).

Note that the following gives the correct result:

?round(round(2390.000000*.072500,3),2)

I think there's a VFP channel for reporting bugs, though I cannot remember how to access it.

Good catch. Who knows! Correcting this error may prevent the next Jupiter probe from shooting off into bozoland!
"The Iron Fish: The water is cold...but the fish don't mind"
...Jay Jenks, boyhood chum
Previous
Reply
Map
View

Click here to load this message in the networking platform