Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Strange and dangerous behaviour
Message
 
 
To
19/10/2004 12:29:25
General information
Forum:
Visual FoxPro
Category:
Other
Miscellaneous
Thread ID:
00952515
Message ID:
00952885
Views:
7
Fabio,

Once again, we must go down this road, and I post more for lurkers reading the thread than for you since you continue to post this false statement.

>Can MSSQL and .NET calculate 0.1 exactly ? yes.

No they can not. 0.1 does not have an exact representation in a binary floating point number because it takes an infinite number of bits to exactly calculate/store 0.1.

Quoting three passages from the SQL Server Help:

Approximate Numeric Data
Approximate numeric (floating-point) data consists of data preserved as accurately as the binary numbering system can offer. Approximate numeric data is stored using the float and real data types in SQL Server. For example, because the fraction one-third in decimal notation is .333333 (repeating), this value cannot be represented precisely using approximate decimal data. Therefore, the value retrieved from SQL Server may not be exactly what was stored originally in the column. Additional examples of numeric approximations are floating-point values ending in .3, .6, and .7.


Approximate number data types for use with floating point numeric data. Floating point data is approximate; not all values in the data type range can be precisely represented.

The IEEE 754 specification provides four rounding modes: round to nearest, round up, round down, and round to zero. Microsoft® SQL Server™ uses round up. All are accurate to the guaranteed precision but can result in slightly different floating-point values. Because the binary representation of a floating-point number may use one of many legal rounding schemes, it is impossible to reliably quantify a floating-point value.

And finally here are some concrete examples which disprove your statement:
select pi() / ( pi() * 10 ) as onetenth, 
   case when pi() / ( pi() * 10 ) = 0.1 then 'yes' else 'no ' end as isitequal -- it says yes, and onetenth displays as 0.10000000000000001
select sqrt(2.0) / sqrt(200.0) as onetenth, 
   case when sqrt(2.0) / sqrt(200.0) = 0.1 then 'yes' else 'no ' end as isitequal -- it says yes, and onetenth displays as 0.10000000000000001

select 1.234567890123E+307 / 1.234567890123E+308 as onetenth, 
   case when 1.234567890123E+307 / 1.234567890123E+308 = 0.1 then 'yes' else 'no ' end as isitequal -- it says no, and it onetenth displays as 9.9999999999999992E-2
select 1.23456789017E+307 / 1.23456789017E+308 as onetenth, 
   case when 1.23456789017E+307 / 1.23456789017E+308 = 0.1 then 'yes' else 'no ' end as isitequal -- it says yes, and it onetenth displays as 0.10000000000000001
select 1.2345678901E+307 / 1.2345678901E+308 as onetenth, 
   case when 1.2345678901E+307 / 1.2345678901E+308 = 0.1 then 'yes' else 'no ' end as isitequal -- it says no, and it displays onetenth displays as 9.9999999999999992E-2

select 1.234567890123E-306 / 1.234567890123E-305 as onetenth, 
   case when 1.234567890123E-306 / 1.234567890123E-305 = 0.1 then 'yes' else 'no ' end as isitequal -- it says yes, and it onetenth displays as 0.10000000000000001
select 1.23456789017E-306 / 1.23456789017E-305 as onetenth, 
   case when 1.23456789017E-306 / 1.23456789017E-305 = 0.1 then 'yes' else 'no ' end as isitequal -- it says yes, and it onetenth displays as 0.10000000000000001
select 1.2345678901E-200 / 1.2345678901E-199 as onetenth, 
   case when 1.2345678901E-200 / 1.2345678901E-199 = 0.1 then 'yes' else 'no ' end as isitequal -- it says yes, and it displays onetenth displays as 0.10000000000000001

select 1.2345678901E+307 / 1.2345678901E+308 as onetentha, 
   sqrt(2.0) / sqrt(200.0) as onetenthb, 
   case when 1.2345678901E+307 / 1.2345678901E+308 = sqrt(2.0) / sqrt(200.0) then 'yes' else 'no ' end as isitequal -- it says no, and it displays onetentha displays as 9.9999999999999992E-2 and onetenthb as 0.10000000000000001
A mathematical 0.1 is not and will never be exactly equal to a binary compuer calculated 0.1
df (was a 10 time MVP)

df FoxPro website
FoxPro Wiki site online, editable knowledgebase
Previous
Reply
Map
View

Click here to load this message in the networking platform