Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Trying to deal with some limitations
Message
From
05/12/2003 10:53:54
Hilmar Zonneveld
Independent Consultant
Cochabamba, Bolivia
 
 
To
05/12/2003 09:28:33
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00856215
Message ID:
00856275
Views:
20
>>Currency values have a relatively large range; 15 digits for the integer part, and 4 decimals.
>>
>>VFPs built-in data types don't support a higher precision. Double becomes inaccurate for large numbers. Something like 15 to 16 digits precision, but the last digits are always doubtful.
>>
>>My download #10044 allows for calculations with an almost unlimited number of digits (the limit is the VFP array limit), but it is relatively slow, compared with the VFP built-in data types and calculations.
>
>( 040909750000200257 + 6 ) * 2430
>
>
>Hi Hilmar;
>
>Thanks for your repli..
>
>Number1 = transform(040909750000200257+6,"@L 999999999999999999")
>Number2 = "2430"
>? Number1, "*", Number2, "=", oMath.Multiply(Number1, Number2)
>
>
>Your routine give me as result:
>99410692500486631800
>
>Different from windows calculator:
>99410692500486639090
>
>Is there something that I can do to deal with this?

Yes. Do the addition with the Math Object, too. Your transform(...) code already gives wrong results to the Math Object.

Also, the Windows Calculator seems to give the correct number in this case, but for larger numbers, it might give you inexact results, too.
Difference in opinions hath cost many millions of lives: for instance, whether flesh be bread, or bread be flesh; whether whistling be a vice or a virtue; whether it be better to kiss a post, or throw it into the fire... (from Gulliver's Travels)
Previous
Reply
Map
View

Click here to load this message in the networking platform