Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
STR() rounds numbers longer than 17 digits...?
Message
 
 
To
03/01/2001 09:25:42
Andrzej Majlich
Vulcan sp. z o. o.
Wroclaw, Poland
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00458883
Message ID:
00458928
Views:
21
VFP doesn't handle numbers greater than 16 digits of precision (left and right of decimal point). It rounds from there.

Check out the Help
Visual FoxPro Documentation - Using Visual FoxPro - Programmer's Guide - Appendix - Visual FoxPro System Capacities - Digits of Precision in Numeric Computations

>I've been trying my index expression after changing field type from numeric to currency: STR(price,16,2).
>As -922337203685477.5808 is the low limit for CURRENCY type numbers, I tried it in a command window:
>
>x = -$922337203685477.5807
>wait window STR(x, 21, 4)
>

>I expected '-922337203685477.5807' string but got '922337203685477.6000'.
>After some investigation I found that STR() prints only 16 most meaning digits rounding the rest:
>
>
>str(-922337203685477.5807,21,4)=='-922337203685477.6000'
>
>str(-022337203685477.5807,21,4)== '-22337203685477.5800'
>
>str(-002337203685477.5807,21,4)== '-2337203685477.5810'
>
>str(-000337203685477.5807,21,4)== '-337203685477.5807'
>

>
>Or maybe it's an effect of some misterious SET?
Larry Miller
MCSD
LWMiller3@verizon.net

Accumulate learning by study, understand what you learn by questioning. -- Mingjiao
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform