Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Precision problem???
Message
 
 
À
14/03/2014 15:39:13
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Versions des environnements
Visual FoxPro:
VFP 9 SP2
OS:
Windows 7
Network:
Windows 2003 Server
Database:
MS SQL Server
Application:
Web
Divers
Thread ID:
01596565
Message ID:
01596577
Vues:
40
>>Hi everybody,
>>
>>I have strange problem. In SQL Server we have these two datetime values:
>>
>>2014-03-04 09:30:00.000	2014-03-04 10:00:00.000
>>
>>In VFP I have the following script
>>
>>(end_time - start_time)/3600
>>
>>and compare it with .5
>>
>>If it's less than .5 it should be true and otherwise it should be false.
>>
>>So, as we see here, the duration is exactly half an hour and I should get false. But I am getting true and I am not sure why and what may be wrong.
>
>If you set the decimals to high value such as 8, the expression ends up resolving to .4999997132. Casting (end_time - start_time) as int also shows a result of 1799, indicating that the result evaluates to slightly less than the expected 1800. I would guess that there is a floating point issue with either one of the time's representations or the resulting subtraction.
>
>The easiest way around it is to round the result of the subtraction or cast it to a currency.

Do you know if EXECSCRIPT uses the same default SET DECIMALS? Because the script is executed using EXECSCRIPT function.
If it's not broken, fix it until it is.


My Blog
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform