Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Precision problem???
Message
From
14/03/2014 16:58:53
 
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Environment versions
Visual FoxPro:
VFP 9 SP2
OS:
Windows 7
Network:
Windows 2003 Server
Database:
MS SQL Server
Application:
Web
Miscellaneous
Thread ID:
01596565
Message ID:
01596575
Views:
37
>My colleague (tester) told me she reproduced the same problem locally with 30 minute lessons. The incentive of less than .5 is still firing for them.
>This doesn't make much sense since the SET DECIMALS is not changed in the application and it's default VFP value.
>I am going to try to play with it later myself in debugging mode.


When you get into C# and are doing this kind of code, if you use a double instead of a float, you'll find much greater precision that, even though it rounds, will likely not cause you these issues.

You can simulate the double usage in VFP as well by using a non-invasive syntax like this:
 k = k + (9999999999 - 9999999999)

* Right now, k has been internally upsized to a 64-bit double, which is demonstrable if you
* SAVE TO to a memory file and use this viewer to parse it out to text:
https://github.com/RickCHodgin/SaveToParser/blob/master/test/test.prg

Full project:
https://github.com/RickCHodgin/SaveToParser
Previous
Reply
Map
View

Click here to load this message in the networking platform