Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Number of decimal places
Message
De
05/10/2010 02:08:06
 
 
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Versions des environnements
Visual FoxPro:
VFP 9 SP2
Divers
Thread ID:
01483618
Message ID:
01483935
Vues:
70
>>>>>>You can see my solution in the thread I mentioned to Tore - I also first need to convert to decimal and make a pick on the precision. I chose 11 digits as using higher precision changes the original number (as it was float 0.99 it will become 0.98000089, something like that).
>>>>>
>>>>>Find the number of decimal of a float is a waste of time,
>>>>>because a solution does not exist.
>>>>>Some floats have an infinite number of decimal places,
>>>>>so we must first convert them into decimals.
>>>>>At this point the problem becomes to find the number of decimal places of a decimal ...
>>>>
>>>>I agree. So, how do you find a number of decimal places of a decimal?
>>>
>>>1. you design the db then you known
>>>2. COLUMNPROPERTY(..,'SCALE')
>>>3.
>>>
>>>-- this work with 1 or more decimals
>>>select	CHARINDEX('.',REVERSE(NULLIF(CAST(Value % 1 as varchar(40)))))-1
>>>-- with 0 decimals, CASE WHEN Value % 1=0 THEN 0 ...
>>>
>>
>>Something is wrong at the last step - can you clarify?
>>
>>
>>declare @t table (Value float null)
>>insert into @t
>>select
>>-41139.99
>>union all select
>>10062.99
>>union all select
>>1600000
>>union all select
>>134979.87
>>union all select
>>-154062.04
>>union all select
>>-153832.17
>>union all select
>>-1235677.922308
>>union all select
>>null
>>
>>select	T.value, F.*, F1.*, F2.* from @t T
>>cross apply (select CAST(Value as decimal(38,11)) as nValue) F
>>cross apply (select	CAST(nValue % 1 as varchar(40)) as cValue) F1
>>cross apply (select CHARINDEX('.',REVERSE(cValue)) as DecimalPointPos) F2
>>
>>I think we need to search for 0, not for .
>
>The best I can do
>
>declare @t table (Value float null)
>insert into @t
>select
>-41139.99
>union all select
>10062.99
>union all select
>1600000
>union all select
>134979.87
>union all select
>-154062.04
>union all select
>-153832.17
>union all select
>-1235677.922308
>union all select
>null
>
>select	T.value, F.*, F1.*, F2.* from @t T
>cross apply (select CAST(Value as decimal(38,11)) as nValue) F
>cross apply (select	CAST(nValue % 1 as varchar(40)) as cValue) F1
>cross apply (select SUBSTRING(cValue, charindex('.', cValue)+ 1,LEN(cValue)) as Decimals) F3
>cross apply (select patindex('%[1-9]%',REVERSE(Decimals)) as Non0) F4
>cross apply (select LEN(Decimals) - (case when Non0 = 0 then LEN(Decimals) else Non0 - 1 end) as DecimalPointPos) F2
>
>but again, 11 seems to convert it to non-precise.

You have to let go of the float.
Any cast (as decimal ()) you do,
there is always a float number where the conversion is wrong.
Throw away FLOAT and uses DECIMAL!
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform