Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
What's wrong with VFP 5.0a's math (or is it me)?
Message
De
14/05/1997 10:48:34
 
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Divers
Thread ID:
00032141
Message ID:
00032168
Vues:
44
>>
>>If I multiple in VFP I get 10.8546564. BTW, rounding 10.8546564 should give you 10.85 not 10.86. So it looks like you are getting the right results (10.85) but where does the 10.8539997 come from ???
>>
>>-Doug-
>
>If I multiply using the numbers directly, I also get the correct answer. But in my program I get an incorrect answer and debugger indicates the values of the variables to be as I've indicated. So I'm guessing that VFP isn't using the exact numbers the debugger has indicated.
>
>Also, how come 10.8546564 rounds to 10.85. Shouldn't the 6 in 10.8546 make it 10.855 and then the 5 make it 10.86?
>
>Jeff

When your program is stopped and the debugger is displaying those numbers, have your tried calculating the numbers (using the variables or whatever) from the Command window? Something does sound out of whack.

I think you answered your own question, you are rounding twice. The whole idea of rounding is to look at the value to the immediate right of the desired resolution and ignore the rest. If you perform a "double" round, you are not getting a valid division of results (ie. you are bias towards rounding up).

-Doug-
Précédent
Répondre
Fil
Voir

Click here to load this message in the networking platform