Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
On VFP 4000 sums are not reliable!
Message
De
16/12/2005 06:44:09
 
 
À
16/12/2005 06:10:53
Information générale
Forum:
Visual FoxPro
Catégorie:
Base de données, Tables, Vues, Index et syntaxe SQL
Versions des environnements
Visual FoxPro:
VFP 9
Database:
Visual FoxPro
Divers
Thread ID:
01078586
Message ID:
01078593
Vues:
9
>this is not a bug, but a VFP feature by design
>
>CLEAR
>CLOSE TABLES ALL
>
>CREATE CURSOR table1 (id I, coeff1 I)
>INSERT INTO table1 VALUES (0,32767)
>INSERT INTO table1 VALUES (1,32767)
>
>CREATE CURSOR TEST (fk I,n N(9,6))
>
>FOR K=1 TO 4000
>	INSERT INTO TEST VALUES (m.k%2,0.100000)
>NEXT
>
>SELECT	SUM(n*coeff1) VFP,'13106800,000000' Exact FROM test JOIN TABLE1 ON FK=ID
>
>* of course ROUND() is useless
>SELECT	ROUND(SUM(ROUND(ROUND(n,6)*ROUND(coeff1,6),6)),6) VFP,'13106800,000000' Exact FROM test JOIN TABLE1 ON FK=ID
>
>
>this is a simple example, but it is possible to build a combination of numbers
>what it produces smaller values but however wrong.
>
>A workaround exists, but it is orrible

Fabio,

I think that this is due to the fact that all N(..) arithmetic is done in/by Double

You know that
- max precision of a double = 15.95 digits (log10(2^53))
- max precision of a Currency = log10(2^63) = 18.96 = 15 integer + 4 fractional
- the floating point exponent is a power of two, not 10

If you can do with max 4 digits after the decimal point, use Currency data type
CLEAR 
CLOSE TABLES ALL

CREATE CURSOR table1 (id I, coeff1 I)
INSERT INTO table1 VALUES (0,32767)
INSERT INTO table1 VALUES (1,32767)

CREATE CURSOR TEST (fk I,n Y)

FOR K=1 TO 4000
	INSERT INTO TEST VALUES (m.k%2, $0.1)
NEXT

SELECT	SUM(n*coeff1) VFP,'13106800,000000' Exact FROM test JOIN TABLE1 ON FK=ID
Gregory
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform