Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Convert string 1/4 or 1/2 to numeric
Message
From
20/01/2016 09:30:01
Lutz Scheffler
Lutz Scheffler Software Ingenieurbüro
Dresden, Germany
 
 
To
20/01/2016 09:15:19
Dragan Nedeljkovich (Online)
Now officially retired
Zrenjanin, Serbia
General information
Forum:
Visual FoxPro
Category:
Other
Miscellaneous
Thread ID:
01629854
Message ID:
01629985
Views:
44
>>>Wouldn't
>>>
lnNominator = 2^(int(LEN(RTRIM(STR(lnFrac,15,15),"0")*log(10)/log(2)))+1)
>>>be more correct? Because for three decimal digits you need two to the power of ten to have your error below 0.001... and 3*log(10)/log(2) is 9.9658, and the +1 there gives you 10.
>>
>>My code is only valid for fractions that have nominators in powers of 2. 1/2; 1/4;/1/8;...
>>So 0.1 is not a valid paramater.
>>Bill only likes to transform those, and for that you only need the number of digits.
>>IOW decimal fraction of 1/(2^n); n in {0;1;2;3;..} will fit into n decimal places
>>
>>Edit:
>>There should be no precision problem. numbers base 10 and base 2 both can store such fractions wihout any loss in precision (IOW they will not become periodic), since both work with 2 in there base.
>
>Ah, but life isn't fair... You assume you have such values, but what if the values come from measurements? With my addition you get to round any set of decimals to a fraction with a 2**n denominator to the same precision you had with the decimals, with perhaps +/- 1 error on the last digit.

I think he does prefab paper.
If the values are not in this, it is cheaper to have a translation table, since those values do not need to be exactly translated, they are more nominal. Customer likes to see its expected named paper, even if it is a bit wrong.
Words are given to man to enable him to conceal his true feelings.
Charles Maurice de Talleyrand-Périgord

Weeks of programming can save you hours of planning.

Off

There is no place like [::1]
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform