Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Convert string 1/4 or 1/2 to numeric
Message
De
20/01/2016 09:15:19
Dragan Nedeljkovich
Now officially retired
Zrenjanin, Serbia
 
 
À
20/01/2016 05:50:51
Lutz Scheffler
Lutz Scheffler Software Ingenieurbüro
Dresden, Allemagne
Information générale
Forum:
Visual FoxPro
Catégorie:
Autre
Divers
Thread ID:
01629854
Message ID:
01629984
Vues:
54
>>Wouldn't
>>
lnNominator = 2^(int(LEN(RTRIM(STR(lnFrac,15,15),"0")*log(10)/log(2)))+1)
>>be more correct? Because for three decimal digits you need two to the power of ten to have your error below 0.001... and 3*log(10)/log(2) is 9.9658, and the +1 there gives you 10.
>
>My code is only valid for fractions that have nominators in powers of 2. 1/2; 1/4;/1/8;...
>So 0.1 is not a valid paramater.
>Bill only likes to transform those, and for that you only need the number of digits.
>IOW decimal fraction of 1/(2^n); n in {0;1;2;3;..} will fit into n decimal places
>
>Edit:
>There should be no precision problem. numbers base 10 and base 2 both can store such fractions wihout any loss in precision (IOW they will not become periodic), since both work with 2 in there base.

Ah, but life isn't fair... You assume you have such values, but what if the values come from measurements? With my addition you get to round any set of decimals to a fraction with a 2**n denominator to the same precision you had with the decimals, with perhaps +/- 1 error on the last digit.

back to same old

the first online autobiography, unfinished by design
What, me reckless? I'm full of recks!
Balkans, eh? Count them.
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform