Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
How You deal with VFP Rounding BUG
Message
From
23/11/2007 02:27:14
 
 
To
22/11/2007 19:29:13
General information
Forum:
Visual FoxPro
Category:
Other
Environment versions
Visual FoxPro:
VFP 9 SP1
OS:
Windows XP SP2
Network:
Windows 2003 Server
Database:
Visual FoxPro
Miscellaneous
Thread ID:
01270811
Message ID:
01270852
Views:
21
>Hi,
>
>I am interesting in what is the best practice to deal with VFP rounding in calculation proble.
>
>error 1. ? 550000.815*1.00*1.00*1.00*1.00 wrong result
>error 2. Round() bug.
>
>Do you round every number and then perform calculations but that we come back to error 1.
>Or you do calculus and then round (with some good round routine).

No computer in the world is able to calculate decimal numbers accurately because digital is all about integers. When a computer converts a decimal number to digital, it must decide which of many different standards it must follow. Every standard has it trade offs, so it's a tough decision, and somebody will always complain.

The ONLY way to calculate decimal numbers correctly via a computer is to convert the decimal numbers to integers during the calculations, and then converting it back to decimals again after the calculations. But using this method you must make risk reaching the upper limit of integer calculations of the computer.

So it's always a choice, no single choice is perfect.
Previous
Reply
Map
View

Click here to load this message in the networking platform