Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Decimal precision
Message
From
23/05/2012 04:21:05
 
 
To
22/05/2012 22:25:46
General information
Forum:
ASP.NET
Category:
Coding, syntax and commands
Environment versions
Environment:
VB 9.0
OS:
Windows 7
Network:
Windows 2003 Server
Database:
MS SQL Server
Application:
Web
Miscellaneous
Thread ID:
01544101
Message ID:
01544118
Views:
27
>In VFP, the default is rounded at two decimal places such as:
>
>? 25/49.88
>
>...will result at 0.50.
>
>However, changing the decimal precision to 10 will give 0.5012028869.
>
>In .NET, if I have defined a variable to be a Double. If I do the same calculation, will I obtain right away a better precision?

A double in VFP and .NET are both 64-bit values so I guess precision will be the same (although the results may not be identical - and in practice even in .Net can vary depending on the platform)

If you need *more* precision in .NET there's always the System.Decimal type - but it is 128-bit and has a smaller range....

> If yes, what would be the default number of decimals place that would be considered?

As Gregory points out - you may be confusing displayed values with stored values.
If you need to actually round a double value then use Math.Round().
If it's just for display then use one of the ToString() overloads e.g:
.ToString("C") , ToString("#000.000") etc. See :
http://msdn.microsoft.com/en-us/library/dwhawy9k.aspx and http://msdn.microsoft.com/en-us/library/0c899ak8.aspx
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform