Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Any Else Experienced This
Message
From
05/02/2011 03:57:46
Dragan Nedeljkovich (Online)
Now officially retired
Zrenjanin, Serbia
 
 
To
04/02/2011 20:39:33
General information
Forum:
Visual FoxPro
Category:
Other
Miscellaneous
Thread ID:
01498759
Message ID:
01498865
Views:
52
>>I have fixed the issue by using rounding at a specific point, but am just curious to see if this is the reason!
>
>It's been a while since I looked at this, but IIRC datetimes actually resolve to fractions of a second. So, you can have two that look identical - down to the second - but if you do a = comparison, you get .F.

Datetime is stored as two 32-bit integers - one for Julian date number, the other for milliseconds since midnight. As soon as you start any arithmetic with them, all bets are off. They probably get converted into IEEE standard floats, as defined by the processor.

>My understanding is VFP supports at least 13 sig figs with any one floating-point calculation.

Fifteen and a half, i.e. the sixteenth has about five values, can't be trusted. I've encountered this during the big inflation of 1993, when order of summing was significant - if the result, at any time during the calculation, exceeded the 15 digits limit, you'd get zeros to the right. Put simple, having 12345678901234567-12345678901234567+555 would give 555, but 12345678901234567+555-12345678901234567 would give incorrect result. Just tried, you get 556. Try with more digits, and you get 560, 512, and then zero. However, 12345678901234567890+555-12345678901234567890+555 still gives 555, because the last result, going from left to right, didn't exceed the width.

back to same old

the first online autobiography, unfinished by design
What, me reckless? I'm full of recks!
Balkans, eh? Count them.
Previous
Reply
Map
View

Click here to load this message in the networking platform