Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
9.995 is less than 9.99500?? Since when?
Message
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00930178
Message ID:
00930180
Views:
17
Russell,

You got bitten by floating point numbers. What you see isn't what is stored in those properties/variables. Try
SET DECIMAL TO 16 
? .FunctionResults
? FormatArray(i, 2)
>I should have put PRE in there to format this better:
>
>
>.FunctionResults                        9.995       N
>FormatArray(i, 2)                       9.99500     N
>.FunctionResults < FormatArray(i, 2)    .T.         L
>.FunctionResults = FormatArray(i, 2)    .F.         L
>
>
>
>>I guess I can't attach a file or insert a graphic, but I have a watch window that looks like this:
>>
>>.FunctionResults 9.995 N
>>FormatArray(i, 2) 9.99500 N
>>.FunctionResults < FormatArray(i, 2) .T. L
>>.FunctionResults = FormatArray(i, 2) .F. L
>>
>>It's probably just a silly thing with me, but somehow I was under the impression that 9.995 was NOT less than 9.99500. But I only graduated college and MS has all those smart people that went on to graduate school and even earned their PhD, so I guess they're right and not me.
--sb--
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform