Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Decimal bug
Message
De
13/03/2003 13:13:54
Todd Cottengim
Alpine Land Information Services
Redding, Californie, États-Unis
 
 
À
Tous
Information générale
Forum:
Visual FoxPro
Catégorie:
Problèmes
Titre:
Decimal bug
Divers
Thread ID:
00765485
Message ID:
00765485
Vues:
59
I have come across a problem in VFP 6. Maybe its just my machine but I thought I would post it.

Given two values nX and nY where nX = 40 and nY = 10, compute the following: INT(nX / nY)

With SET DECIMALS at anything less than or equal to 10, the answer is correct (4). With SET DECIMALS at 11 or higher, I get the result 3.99999999996 with the 6 repeating based on number of decimals.

My computer:
Pentium 4 2.4Ghz with 512M memory
Nvidia G-Force4 Ti 4600 video card
VFP 6 - SP5

Too weird for me!
Anybody have any thoughts on this

Todd Cottengim
Alpine Land Information Services
Anderson, CA
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform