Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Not so logical
Message
From
08/09/2009 10:05:15
 
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Title:
Miscellaneous
Thread ID:
01422244
Message ID:
01423058
Views:
103
>Hi Moises,
>
>>Maybe INT() will work because this application does not require precision
>
>This has absolutely nothing to with the precision needed by the application. As soon as your program performs operations other than addition or subtraction, you have to deal with fractions. As soon as you have fractions, you can have situations like the following:
>
>? int(2.050000000000000044*100) && 205
>? int(2.050000000000000043*100) && 204
>
>Both values are clearly greater than 205. Yet, the second line will report 204. And that's entirely correct, because the integer part of the second line is 204. INT() is usually misused as a way to remove fractions, but that is NOT what INT() does. If you want to remove fractions, use ROUND(,0), always, ever! If you want to case a floating point to an integer use INT().

Christof,

or VFP is not correct
or MSSQL is not correct:
SELECT FLOOR(2.050000000000000044*100) -- 205
SELECT FLOOR(2.050000000000000043*100) -- 205
Here MSSQL is correct on this case.

VFP have a literal conversion error:
? int(2.0500000000*100) && 205
? int(2.05000000000*100) && 204 !!!!!
A conversion round error is acceptable,
but it cannot violate the monotonic of the operation:
if x>=y and f() is monotonically increasing then f(x)>=f(y)
Because
? 2.050000000000000043 >= 2.05 && true
? int(2.050000000000000043*100) >= int(2.05*100) && false
then conversion is bad!

MSSQL,with others literals numbers,
show this error too;
unfortunately the SQL team's response shows that are still missing at sea!

Is obscure for me to understand why, in 2009,
who writes conversion functions still make these mistakes trivial.
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform