Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Decimal bug
Message
From
13/03/2003 13:29:51
Hilmar Zonneveld
Independent Consultant
Cochabamba, Bolivia
 
 
To
13/03/2003 13:13:54
Todd Cottengim
Alpine Land Information Services
Redding, California, United States
General information
Forum:
Visual FoxPro
Category:
Troubleshooting
Title:
Miscellaneous
Thread ID:
00765485
Message ID:
00765494
Views:
16
>I have come across a problem in VFP 6. Maybe its just my machine but I thought I would post it.
>
>Given two values nX and nY where nX = 40 and nY = 10, compute the following: INT(nX / nY)
>
>With SET DECIMALS at anything less than or equal to 10, the answer is correct (4). With SET DECIMALS at 11 or higher, I get the result 3.99999999996 with the 6 repeating based on number of decimals.

I don't get this. I have VFP 6, currently I have no service pack applied.
Difference in opinions hath cost many millions of lives: for instance, whether flesh be bread, or bread be flesh; whether whistling be a vice or a virtue; whether it be better to kiss a post, or throw it into the fire... (from Gulliver's Travels)
Previous
Reply
Map
View

Click here to load this message in the networking platform