Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Large numbers with high decimal precision
Message
From
31/08/2011 03:01:42
 
 
To
30/08/2011 16:44:21
Dragan Nedeljkovich (Online)
Now officially retired
Zrenjanin, Serbia
General information
Forum:
Visual FoxPro
Category:
Other
Environment versions
Visual FoxPro:
VFP 9 SP2
OS:
Windows XP SP2
Miscellaneous
Thread ID:
01522171
Message ID:
01522388
Views:
42
>>>Have you tried shifting the decimal point? You could multiply by 100000 to shift the decimal point left for storing and then when retrieving divide back by 100000 for calculations. The number that you multiply/divide is determined by the number of significant digits that you want to maintain.
>>
>>
>>That won't change a thing. Double structure has an exponent and a mantissa which keeps 15.95 significant digits. Multiplying or dividing by a power of 10 has no impact on the mantissa, only on the exponent
>
>Exponent is binary, i.e. in powers of two, so your 1/10 may actually be 0.999999999... something.

True - what I wanted to say is the multiplying/dividing does not change the number of significant digits
Gregory
Previous
Reply
Map
View

Click here to load this message in the networking platform