Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Precision problem???
Message
 
 
To
15/03/2014 01:05:48
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Environment versions
Visual FoxPro:
VFP 9 SP2
OS:
Windows 7
Network:
Windows 2003 Server
Database:
MS SQL Server
Application:
Web
Miscellaneous
Thread ID:
01596565
Message ID:
01596613
Views:
55
>>>>My colleague (tester) told me she reproduced the same problem locally with 30 minute lessons. The incentive of less than .5 is still firing for them.
>>>>
>>>>This doesn't make much sense since the SET DECIMALS is not changed in the application and it's default VFP value.
>>>>
>>>>I am going to try to play with it later myself in debugging mode.
>>>
>>>SET DECIMALS only controls the display of numbers, not what their actual values are. If it is set to 2 or 4, you will see it rounded to that many decimal places. Since it gets rounded to only 4 decimal places, you don't see where the loss of precision causes it to go below .5.
>>
>>In other words, SET DECIMALS has nothing to do with the nature of the problem, it only shows the problem if set to bigger value, right?
>>
>>Do you have other ideas of fixing besides rounding? I don't want to round to 1 digit as I don't know what numbers they will want to compare with.
>
>I do't get why you don't just compare ltEnd to ltStart+1800 ?

Did you see the script I posted in that thread? The user enters lower bound and (optionally) upper bound (in hours) and I need to compare value with it. It will be a bit more complex to change the expression to multiple the bounds instead of dividing the expression to turn into hours.
If it's not broken, fix it until it is.


My Blog
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform