Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Precision problem???
Message
From
14/03/2014 15:39:13
 
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Environment versions
Visual FoxPro:
VFP 9 SP2
OS:
Windows 7
Network:
Windows 2003 Server
Database:
MS SQL Server
Application:
Web
Miscellaneous
Thread ID:
01596565
Message ID:
01596567
Views:
72
This message has been marked as a message which has helped to the initial question of the thread.
>Hi everybody,
>
>I have strange problem. In SQL Server we have these two datetime values:
>
>2014-03-04 09:30:00.000	2014-03-04 10:00:00.000
>
>In VFP I have the following script
>
>(end_time - start_time)/3600
>
>and compare it with .5
>
>If it's less than .5 it should be true and otherwise it should be false.
>
>So, as we see here, the duration is exactly half an hour and I should get false. But I am getting true and I am not sure why and what may be wrong.

If you set the decimals to high value such as 8, the expression ends up resolving to .4999997132. Casting (end_time - start_time) as int also shows a result of 1799, indicating that the result evaluates to slightly less than the expected 1800. I would guess that there is a floating point issue with either one of the time's representations or the resulting subtraction.

The easiest way around it is to round the result of the subtraction or cast it to a currency.
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform