Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Large numbers with high decimal precision
Message
From
30/08/2011 16:44:21
Dragan Nedeljkovich (Online)
Now officially retired
Zrenjanin, Serbia
 
 
To
30/08/2011 15:17:25
General information
Forum:
Visual FoxPro
Category:
Other
Environment versions
Visual FoxPro:
VFP 9 SP2
OS:
Windows XP SP2
Miscellaneous
Thread ID:
01522171
Message ID:
01522328
Views:
37
>>Have you tried shifting the decimal point? You could multiply by 100000 to shift the decimal point left for storing and then when retrieving divide back by 100000 for calculations. The number that you multiply/divide is determined by the number of significant digits that you want to maintain.
>
>
>That won't change a thing. Double structure has an exponent and a mantissa which keeps 15.95 significant digits. Multiplying or dividing by a power of 10 has no impact on the mantissa, only on the exponent

Exponent is binary, i.e. in powers of two, so your 1/10 may actually be 0.999999999... something.

back to same old

the first online autobiography, unfinished by design
What, me reckless? I'm full of recks!
Balkans, eh? Count them.
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform