Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
HOW TO REJECT BAD DATA????
Message
General information
Forum:
Visual FoxPro
Category:
Troubleshooting
Miscellaneous
Thread ID:
00019169
Message ID:
00019224
Views:
37
>>If each of these numbers were in a field in a dbf(one per record) How could I programmatically throw out numbers that were way off? I have tried diffrent forms of average and standard deviation, but nothing seems quite right. I need some high school algebra or statistics lessons I think.
>
>What your describing in statistical lingo is called throwing out the outliers. Most people use a figure of 2 standard deviations off the mean to give you a range for outliers (this removes anything about 90% or more off the mean) Some also choose 3 std dev which is about 95%.
>
>To do this in vfp use the calculate command with the std dev option to calculate the std dev for the field and the mean function to calculate the mean of the field. Then do a scan which will delete (or however you want to remove data) records that are above (2*std dev)+mean or below mean-(2*std dev).
>
That's what we suggested previously, but now that we know these are housing costs, using this method may not be good, the distribution probably isn't very much like 'normal' (bell-shaped). The above is only really valid for normal-like data. Of course you can plot the curve and see what it looks like if you have sufficient # of values...then you might be okay...(But see other discussion on tossing outliers)
The Anonymous Bureaucrat,
and frankly, quite content not to be
a member of either major US political party.
Previous
Reply
Map
View

Click here to load this message in the networking platform