Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Precision problem in Foxpro - * BUG ? *
Message
De
01/10/2003 12:39:35
 
 
À
01/10/2003 12:13:55
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Divers
Thread ID:
00833874
Message ID:
00833886
Vues:
29
>Experiencing very specific problems when multiplying several variables (each having about 3 to 4 decimals), we could try out the following example:
>
>SET DECIMALS TO 2
>? 1/100 && result: 0.01
>? 1/1000 && result: 0.00
>? (1/1000)*1000 && result: 1.00
>? (1/1000000)*1000000 && result: 1.00
>
>but:
>
>SET DECIMALS TO 15
>? 1/100000 && gives: 0.000010000000000 = ok
>? 1/1000000 && gives: 0.000001000000000 = ok
>? 1/10000000 && gives: 0.000000000000000 = WHAT THE H'LL ??
>? (1/10000000)*10000000 && gives: 0.000000000000000 = ????
>
>So there must be definitely a calculation problem in Foxpro.
>
>If you read the manuals, you see: ... "
>The correct calculation is performed and stored internally, but all subsequent displays of the result use the DECIMALS setting at the time the number was calculated. ... "
>
>As calculators of HP or Texas Instruments or any other brand usually work with at least 10 or 14 figures behind the coma, I can't understand what Foxpro is doing here...
>
>Btw, using the ROUND function does allways return a correct result.

Does it? In my case even Round() did not return a valid value. It's puzzling.
Groet,
Peter de Valença

Constructive frustration is the breeding ground of genius.
If there’s no willingness to moderate for the sake of good debate, then I have no willingness to debate at all.
Let's develop superb standards that will end the holy wars.
"There are three types of people: Alphas and Betas", said the beta decisively.
If you find this message rude or offensive or stupid, please take a step away from the keyboard and try to think calmly about an eventual a possible alternative explanation of my message.
Précédent
Répondre
Fil
Voir

Click here to load this message in the networking platform