Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
What's wrong with VFP 5.0a's math (or is it me)?
Message
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Divers
Thread ID:
00032141
Message ID:
00032232
Vues:
44
>>>variable1 = variable2 * table.field
>>>
>>>If variable2 = 0.04998, the value of table.field is 217.18, then I should get variable1 = 10.8546564. Instead I get variable1 = 10.8539997.
>>>
>>>While my program is running, if I go to the command window and type variable2 = variable1 * 217.18 I also get the incorrect answer. But without the program running, if I type variable1 = 0.04998, variable2 = variable1 * 217.18, I get the correct answer.
>>>
>>I used to have this trouble in PL/I, they used binary processing to compute and odd results were common. Anyway, did you try setting a variable 3 to the table.field before evaluating to see if result is good? And also maybe try the KB doc's idea of using ROUND?
>
>
>check your set decimals....

After more study, I think the only bug the KB doc is referring to is that evaluating an equation as T/F can give a false "T/F" answer:

? x = y + z

The actual numeric calculations appear correct whatever set deci is set to. You're right, set deci has an effect on a T/F equation evaluation as above. But I tried set deci earlier with Jeff's odd answer just as a calculation, and can't come up with it for any set deci #...

Sigh of relief, anyway...for a moment there I thought maybe we were re-entering that nasty early-Pentium twilight zone. (s)
The Anonymous Bureaucrat,
and frankly, quite content not to be
a member of either major US political party.
Précédent
Répondre
Fil
Voir

Click here to load this message in the networking platform