Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Large numbers with high decimal precision
Message
De
30/08/2011 13:58:02
 
 
À
30/08/2011 13:03:14
Information générale
Forum:
Visual FoxPro
Catégorie:
Autre
Versions des environnements
Visual FoxPro:
VFP 9 SP2
OS:
Windows XP SP2
Divers
Thread ID:
01522171
Message ID:
01522300
Vues:
68
>>>>>>>>Any help ?
>>>>>>>
>>>>>>>Have you tried N(33,10) in VFP ?
>>>>>>
>>>>>>Create table testit (mynumber n(33,10))
>>>>>>
>>>>>>gives an error that "field width or number of decimal places is invalid"
>>>>>>
>>>>>>In the table designer the best I can get is 20,9
>>>>>>
>>>>>>Am I missing some trick?
>>>>>
>>>>>Sorry, have you tried B(10) ?
>>>>
>>>>Yes, and B(16)
>>>>
>>>>Both store 123456789012.12350000
>>>
>>>Did you check VFP Help Double Type?
>>
>>Again, yes. Double stores the number as 123456789013
>>
>>As Al and HIlmar have confirmed what I am seeing is a limitation of VFP.
>
>Have you tried shifting the decimal point? You could multiply by 100000 to shift the decimal point left for storing and then when retrieving divide back by 100000 for calculations. The number that you multiply/divide is determined by the number of significant digits that you want to maintain.

No, that can't be right. The "double" data type only have something like 56 bits, which is a maximum of about 16 or 17 significant digits (try 2**56)

That means that only first 16 digits or so are significant, anything after that is garbage.
Jim Nelson
Newbury Park, CA
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform