Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
What's wrong with VFP 5.0a's math (or is it me)?
Message
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Divers
Thread ID:
00032141
Message ID:
00032212
Vues:
42
>>I took a look since seeing this got me concerned (I do lots of statistical calculating). The bug that's shown in the KB article is too many (unneeded) parentheses in an equation, so we just have to watch it when we use them...that's not your problem, is it?
>
>Are you sure you're looking at the same KB article? In the steps to reproduce they list a formula that containsa number of parenthesis, but it doesn't say that this is the only situation that produces the problem.
>

You could be right, I'm inferring that since removing the extra parens gives correct solution, that's the only problem. I would hope that my inference is right, but I think I'll test it later to check it.

>My problem is this: when my program runs, I have a line of code that is something like:
>
>variable1 = variable2 * table.field
>
>If variable2 = 0.04998, the value of table.field is 217.18, then I should get variable1 = 10.8546564. Instead I get variable1 = 10.8539997.
>
>While my program is running, if I go to the command window and type variable2 = variable1 * 217.18 I also get the incorrect answer. But without the program running, if I type variable1 = 0.04998, variable2 = variable1 * 217.18, I get the correct answer.
>
>I am unable to reproduce this by creating a new program.
>

I used to have this trouble in PL/I, they used binary processing to compute and odd results were common. Anyway, did you try setting a variable 3 to the table.field before evaluating to see if result is good? And also maybe try the KB doc's idea of using ROUND?
The Anonymous Bureaucrat,
and frankly, quite content not to be
a member of either major US political party.
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform