Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Error in calculations?
Message
From
12/07/2004 08:32:08
Hilmar Zonneveld
Independent Consultant
Cochabamba, Bolivia
 
 
To
11/07/2004 08:15:53
General information
Forum:
Visual FoxPro
Category:
Other
Miscellaneous
Thread ID:
00923104
Message ID:
00923260
Views:
19
I think if you use Currency type, you will get complete accuracy for addition and subtraction (if you don't need more than 4 decimals, and don't get an overflow). Also, for multiplication and division, you will get as much accuracy as 4 decimals allows.

>Hi,
>
>Is there a bug in Foxpro/VFP?
>
>I have tried this program both in VFP6.0 and FPD2.6 and on two different machines (both running Win98SE), and the results are the same.
>
>Each subtraction adds 1 (one to the 14th decimal position.
>
>I first discovered the problem using addition instead of subtraction; then the increment took place at each 7th iteration.
>
>set decimals to 18
>y = 100.0000000000000000 && 16 zeroes
>set step on
>for n = 1 to 100
> y = y - .1
>next
>
>How can I trust VFP arithmetics after this? I have not made an in-depth study of the problem, so I don’t know if the error only is active when using many decimals.
>
>The reason I discovered the problem was that I was attempting to create as close an approximation as possible of the square root of 1000.
>
>I was not able to get past 31.62277660168379, which squared gave
>9.999 999 999 999 999 8 00000
>
>I kept adding .1 to the temporary root figure until the square, would exceed 1000, then I reversed the last addition to the temp. root, shifted the ‘1’ one position to the right and repeated.
>
>Rolf
Difference in opinions hath cost many millions of lives: for instance, whether flesh be bread, or bread be flesh; whether whistling be a vice or a virtue; whether it be better to kiss a post, or throw it into the fire... (from Gulliver's Travels)
Previous
Reply
Map
View

Click here to load this message in the networking platform