>>Try this for a 1,000,000 record (same argument you used). Since it's is the norm that the tables are NOT filtered, and they *can* filter them. Also these main tables seldom reach such amount of records. (most mid-sized companies don't have 1,000,000 clients, articles, employees)

>1M records is not at all unreasonable for a mid-sized business, at least from my POV.

I said, most companies. A client I work for has a gross business in selling medical tests for hopitals doing $ 20M. their largest main table (Articles) is about 7,000 records. A programm I developed for the health world organisation deals with about 100,000+ patients on a stand-alone machine, no SET FILTER performance problems up here.

>In the shipping industry, it's not unusual to deal with mapping tables based on 5 digit postal code to 5 digit postal code rate mapping; there are over 78K valid 5 digit postal codes. Typical rate charts from the national-level trucking firms and LTL freight carriers run in the 800K-1M record range. I provide rating systems for some of them - I may have 10-15 national-level carriers on-hand now, and a similar number of regionals. Even end-users with a single warehouse will have 70-80K records per carrier in the base rating tables they use when arranging shipments, and typically they'll have business with 4-8 different carriers.

I would not regard this mid-sized business anymore.

>With the advent of mass mailing and emailing, it's not unusual for people with even one man shows to have lists of hundreds of thousands of potential contact names and addresses. Charities, churches, political organizations, the guy selling Ronco(tm)'s latest handy dandy widget...

Note that I was talking about 1+ million records.

>You need to open your eyes and look at the world around you. We live in an era where storing and examining huge data stockpiles has become a routine way of doing business for a lot of people.

My eyes are open for the business I'm making a living off. In my business I don't often see 1M+ records in a table. I wonder if the majority of the VFP community does. Also note that I was talking about MAIN tables, by which I mean tables such as articles, employees, clients, etc.. not about tables for OLAP purposes.

But of course you're right. If you know you've got to deal with massive amounts of data, you'll need to adapt another strategy, like you don't want to adress more than a few hundred items in a combobox. This decisions should be made at design time.

>>It still depends on the selectivity of the filter. If only a few rows are going to be displayed, yep, you've got a performance problem. But if you want to display all 1,000,000 records, p-views are definitely not an attractive alternative. Since the UI design says that all rows must be accesable at default, SET FILTER is the better choice.

>If I had an end-user who really needed to deal with all 1M records, or even a significant fraction of those 1M records, through a browse a dozen or two records at a time, I'd be looking into psychiatric care for the individual or some general help for him in terms of clarifying the data needed to do the task at hand.

In my cases the grid displays all possible items (lets say persons) on the first page. The other pages show detail data of the selected item in the grid. The user is able to display the grid, sort the table, and use an incremental search to find the record he/she needs. This way, It doesn't matter if there are only a few hundred or a few hundred thousand items in the grid. No lets say i've got a persons table of 50,000 records and the users knows that the person he/she is looking for lives in amsterdam, is male and works for company X. The users applies the filter and the grid lists all posible candidates, picks one out and makes the changes. The time needed for list these candidate would probably not differ too much from a SQL-query to be executed, but has the advantage that you're working live data.

The performance drop will not be a significant problem because this feature is only ment for the more exceptional search cases. If a specific search is getting structual, I might be looking for another solution depending on the performance problems arising.

>p-views and SQL SELECTs work well for my applications, letting the user deal with convenient sets of records, they scale well, and are easily moved to a backend environment when data set size or network performance becomes a major issue.

I fully understand the advantages and disadvantages of p-views; they both have some different properties whereof performance is one that is in benefit of both, depending on the circumstances. Personally I'm very comfortable in using p-views on the many side of a one-to-many relation. I've found some circumstances where filters are much more convinient than p-views. From my POV they're different animals which only have one thing in common: filtering data. Working with live data, index usage, persistency, buffering and performance caracteristic are others which also are important for decision which to use.

>I'm sure some people will continue to use xBASE efffectively; for me, and the people I work with, it doesn't make sense, and won't put us in a position to exploit the inforamtion technologies now reaching the marketplace.

Interesting.. Recently I've read an article translated from a document released by the Gartnerr group (november 1998 ?), saying that they predict a great future for aggregrated query results. From what I did understand of it, this means that rather than copying data to a queryresult cursor, it uses live data to construct the query results resulting in a huge performance increase. I'll bet MS is at least researching this for SQL-server, maybe they've already implemented it in some form in SQL-server. I can see big parallels with the xBase equivalents achieving the same. From my POV this means that among the other xBase features, the SET FILTER command is an important feature that is also showing up in the internals of server RDBMSs.

>If I had to page down through a million records in a grid or browse, my fingers would cramp up if nothing else.

Using an incremental search would not have this problem.

Walter,