Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Precision problem???
Message
 
 
To
14/03/2014 15:39:13
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Environment versions
Visual FoxPro:
VFP 9 SP2
OS:
Windows 7
Network:
Windows 2003 Server
Database:
MS SQL Server
Application:
Web
Miscellaneous
Thread ID:
01596565
Message ID:
01596568
Views:
51
>>Hi everybody,
>>
>>I have strange problem. In SQL Server we have these two datetime values:
>>
>>2014-03-04 09:30:00.000	2014-03-04 10:00:00.000
>>
>>In VFP I have the following script
>>
>>(end_time - start_time)/3600
>>
>>and compare it with .5
>>
>>If it's less than .5 it should be true and otherwise it should be false.
>>
>>So, as we see here, the duration is exactly half an hour and I should get false. But I am getting true and I am not sure why and what may be wrong.
>
>If you set the decimals to high value such as 8, the expression ends up resolving to .4999997132. Casting (end_time - start_time) as int also shows a result of 1799, indicating that the result evaluates to slightly less than the expected 1800. I would guess that there is a floating point issue with either one of the time's representations or the resulting subtraction.
>
>The easiest way around it is to round the result of the subtraction or cast it to a currency.

Thank you.
If it's not broken, fix it until it is.


My Blog
Previous
Reply
Map
View

Click here to load this message in the networking platform