Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Will SET PRECISION fix odd math?
Message
 
 
To
24/02/2006 11:30:09
General information
Forum:
Visual FoxPro
Category:
Troubleshooting
Environment versions
Visual FoxPro:
VFP 6 SP5
OS:
Windows XP SP2
Network:
Novell 4.x
Database:
Visual FoxPro
Miscellaneous
Thread ID:
01098837
Message ID:
01098950
Views:
16
Paul,

The binary computer isn't sitting there with a pencil and paper multiplying the values by longhand. Floating point numbers are always "approximations" of the actual number and you have to understand what that means when you are using them in iterative calculations.

> Thanks for taking the time to respond. Is it too much to ask for mathematical function that I believe the schools are teaching around the 3rd grade to just 'work right'? 823.4721345000 * 10 = 8234.721345000 except when I situationally decide to tack a 1 on the end of it? I think that's not how they're teaching it <g>.
df (was a 10 time MVP)

df FoxPro website
FoxPro Wiki site online, editable knowledgebase
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform