Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
How to fix rounding bug
Message
From
24/10/2007 03:18:51
 
General information
Forum:
Visual FoxPro
Category:
Troubleshooting
Environment versions
Visual FoxPro:
VFP 9 SP1
Miscellaneous
Thread ID:
01258001
Message ID:
01262964
Views:
20
hmmm...
Not perfect.
Try with m.nNumber = -9 and m.nPrecision = 2
The result is -8.99

This is better:

Lparameters m.nNumber , m.nPrecision
Return Iif( m.nNumber < 0 , ;
-1 * Round(Int(Val(Transform(-1* m.nNumber*10^m.nPrecision+.5)))/10^m.nPrecision, m.nPrecision), ;
Round(Int(Val(Transform(m.nNumber*10^m.nPrecision+.5)))/10^m.nPrecision, m.nPrecision) )
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform