Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Convert string 1/4 or 1/2 to numeric
Message
From
20/01/2016 09:15:19
Dragan Nedeljkovich
Now officially retired
Zrenjanin, Serbia
 
 
To
20/01/2016 05:50:51
Lutz Scheffler
Lutz Scheffler Software Ingenieurbüro
Dresden, Germany
General information
Forum:
Visual FoxPro
Category:
Other
Miscellaneous
Thread ID:
01629854
Message ID:
01629984
Views:
53
>>Wouldn't
>>
lnNominator = 2^(int(LEN(RTRIM(STR(lnFrac,15,15),"0")*log(10)/log(2)))+1)
>>be more correct? Because for three decimal digits you need two to the power of ten to have your error below 0.001... and 3*log(10)/log(2) is 9.9658, and the +1 there gives you 10.
>
>My code is only valid for fractions that have nominators in powers of 2. 1/2; 1/4;/1/8;...
>So 0.1 is not a valid paramater.
>Bill only likes to transform those, and for that you only need the number of digits.
>IOW decimal fraction of 1/(2^n); n in {0;1;2;3;..} will fit into n decimal places
>
>Edit:
>There should be no precision problem. numbers base 10 and base 2 both can store such fractions wihout any loss in precision (IOW they will not become periodic), since both work with 2 in there base.

Ah, but life isn't fair... You assume you have such values, but what if the values come from measurements? With my addition you get to round any set of decimals to a fraction with a 2**n denominator to the same precision you had with the decimals, with perhaps +/- 1 error on the last digit.

back to same old

the first online autobiography, unfinished by design
What, me reckless? I'm full of recks!
Balkans, eh? Count them.
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform