Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
How to fix rounding bug
Message
From
24/10/2007 14:17:17
 
 
To
24/10/2007 03:18:51
General information
Forum:
Visual FoxPro
Category:
Troubleshooting
Environment versions
Visual FoxPro:
VFP 9 SP1
Miscellaneous
Thread ID:
01258001
Message ID:
01263246
Views:
20
>hmmm...
>Not perfect.
>Try with m.nNumber = -9 and m.nPrecision = 2
>The result is -8.99
>
>This is better:
>
>Lparameters m.nNumber , m.nPrecision
>Return Iif( m.nNumber < 0 , ;
> -1 * Round(Int(Val(Transform(-1* m.nNumber*10^m.nPrecision+.5)))/10^m.nPrecision, m.nPrecision), ;
> Round(Int(Val(Transform(m.nNumber*10^m.nPrecision+.5)))/10^m.nPrecision, m.nPrecision) )

Will this solution work in all cases ?
Or is it better to write a fll file which works ?
Andrus
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform