Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Bugs with literal double-precision floating-point number
Message
From
24/11/2003 10:47:39
 
 
To
24/11/2003 09:42:38
General information
Forum:
Visual FoxPro
Category:
Other
Miscellaneous
Thread ID:
00852922
Message ID:
00852951
Views:
17
>1. Question.
>
>How many literal digits are meant in VFP ?
>
>SET DECIMALS TO 18  && YOU CAN PUT ANY VALUE HERE, RESULT NOT CHANGE
>SET FIXED OFF       && YOU CAN PUT ON HERE, RESULT NOT CHANGE
>? 3.119999999999971000000000000000000000E140  && 3.11999999999999000000000000000000E+140
>? 3.1199999999999710000000000000000000000E140 && 3.12000000000000000000000000000000E+140
>
>
>1 Problem: why VFP print dpfpn literals with 32 decimals ?
>
>2 Bug : 37° literal zero it has a weight not zero
>
>then
>
>? 'all zero are zero ?' , 3.119999999999971000000000000000000000E140 >= 3.1199999999999710000000000000000000000E140
>
>
>rule is literal1>=literal2 -> value1>=value2 is not respected.
>
>How many literal digits are meant in VFP ? Sure at least 37 decimals.
>
>This is a little problem because it appears with much decimals.
>
>Fabio

Fabio,

Help (under Visual Foxpro system capacities) says :
Digits of precision in numeric computations 16 which is a half too many

I think the mantissa of a double is 52 bits. Hence, log10(2^52) yields 15.65 digits of precision.

I would go for 15 to be on the safe side.
Gregory
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform