Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Incorrect rounging of 6 digits pecision
Message
From
22/04/2002 17:16:10
 
General information
Forum:
Visual FoxPro
Category:
Troubleshooting
Miscellaneous
Thread ID:
00647785
Message ID:
00647891
Views:
17
This works for me on VFP7 SP1 and VFP6 SP5 running under NT4 SP6. Your mileage may vary.

VAL(STR(t1.price * t2.quantity, 20, 2))


Jim Fisher


>Well, obviously, vfp works incorrectly here. So you need to deceive it somehow. This works for me:
>round(10*t1.price*t2.quantity/10,2)
>
>>Yuri,
>>
>>thank you for reply.
>>
>>In real life, price and quantity fields contain any values up to 6 digits after comma.
>>I need to round the RESULT of multiplication to two digits.
>>I cannot round the operands because in this case the result may be incorrect if operands have 6 digits after comma !
>>
>>How to implement the rounding in this case ?
>>
>>>OK, I see the problem. It occurs when the sum of the decimals to round exceed 12. Perhaps you my use:
>>>
>>>?ROUND( round(t1.price,5)*t2.quantity,2)
>>>
>>>
>>>
>>>>>The product is 25.425. Why do you expect 25.43? It should be 25.42.
>>>>
>>>>Because, when you round 24.425 to two decimals, you get 24.43 (since the first digit dropped is a 5, VFP rounds up). Try it with the ROUND() function.
>>>>
>>>>Hilmar.
Previous
Reply
Map
View

Click here to load this message in the networking platform