Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Fixed numeric passed as float, why and how to avoid
Message
From
30/12/2015 02:38:12
Dragan Nedeljkovich (Online)
Now officially retired
Zrenjanin, Serbia
 
 
To
29/12/2015 17:35:55
General information
Forum:
Visual FoxPro
Category:
Databases,Tables, Views, Indexing and SQL syntax
Environment versions
Visual FoxPro:
VFP 9 SP2
OS:
Windows Server 2003
Network:
Windows 2008 Server
Database:
MS SQL Server
Application:
Web
Miscellaneous
Thread ID:
01629363
Message ID:
01629432
Views:
29
>>This is the same question as in the other thread (and it's solved there), but this time I only want to know this:
>>
>>When I send a SQL statement containing a field value, like ", ?alias.field, ...", and the field is numeric (i.e. not integer, not float, haven't checked for currency), VFP's parser sends it like this
>>
>>
exec sp_executesql '{sql statement here}', '...parameter list..., @p21 as float, ...', ...., {my field value as float}
>>
>>The case at hand is that the field with a value of 0.58 gets passed as 0.57999999999996 (didn't count the nines, but it's the usual 15.5 significant digits). Does anyone know how to tell fox to pass it as numeric, or generally how to avoid this kind of rounding error (and numeric types are supposed to be immune to this, when passed in text format)?
>
>the VPF ?cast(m.var AS I) work,
>but CAST(m.var AS N(5,2)) not work.
>This because N(5,2) is a formatting declaration over the float.
>The issue is a design lack because when you read SQL decimal,
>VFP map it to numeric field.
>
>Naomi says, only in the SQL Server world a value can exists as a decimal number.

Yes, that's my conclusion too. The VFP parser gets the expression as either integer or general numeric (and perhaps currency), and sends in that format. Nadia's solution would work when passing parameters; for passing values from cursoradapter it's a bit irrelevant because the rounding which occurs SQL side gets the values stored correctly.

Tried this yesterday
select case when 0.58=0.57999999999999996 then 'equal' else 'unequal' end
select case when 0.58=CAST(0.57999999999999996 as float) then 'equal' else 'unequal' end
and got that the first one returns unequal and the other equal on SQL 2008R2 and SQL2014, so even in a where clause, if it was passed as a ?var parameter, it would still work correctly because VFP parser does declare it as float - the 2nd case applies.

back to same old

the first online autobiography, unfinished by design
What, me reckless? I'm full of recks!
Balkans, eh? Count them.
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform