>I understand that. We have over 1.5 GB in one table. Still dont understand why a user needs to see so many records to work with. If you use COMs then it's so easy to have havy proccessing on remote machine without pumping tons of data over the network.
>Mark

Mark,

The users don't need to see the records at all, I never said they did. Since we are on a segmented LAN moving 3-4 Meg to a user would not be an issue even if they did. In many situations we do move sizable amounts of data to clients for them to process as they need using such tools as Access and Excel. That has absolutely nothing to do with my original post.

As far as moving heavy processing to remote machine, that’s exactly what were doing.

As an example of what I was referring to, a user requests a statistical analysis of sales for the past year. They select their parameters and submit a request via Intranet. A COM server takes the request and issues a series of commands to the appropriate data warehouse(s). We do as much data processing as possible on the data servers to reduce the amount of data returned to the middle tier. In this example the data sets being accessed contained over 3 gig of joined data which was reduce to around 3.4 Meg. I personally feel that ratio was exceptional and it did not put an excessive processing load on the data servers. A good deal of work has been done to ensure proper load balancing between what data processing can be done on the data servers verses moving large volumes of data across the LAN. The data that is transferred is localized to a dedicated seqment so bottlenecks to the client segments are not an issue. A major problem with doing extensive processing on the data servers is scalability. After you have the taxed the biggest, badest server it gets quite expensive to implement mirrored servers - I know cause we already have one. Data servers should be used for handling data requests, not doing huge amounts of data processing.

Now that the VFP COM has a subset of data, the complicated, resource intensive data processing can begin. Here's my original grip, I've used ADO to get the 3.4 Meg subset of data and must use 51 resource pegging seconds to move the 3.4 Meg recordset into a VFP cursor. This is something that should take only a couple of seconds at worst case. What if the resulting data set was 50 Meg? - do the math. Please don't tell me I should have used an ODBC connection, cause it's not an option. After the statistical analysis processing is completed the resulting report is sent to the client either for viewing or printing.

Today and even more so in the future we will need to access data via OLE DB. This is a key part of MS's universal data access strategy and for VFP to be a solid contender in the COM server game it needs to be able to talk to any data source.

Sorry for the lengthy reply :).

Regards,
Mike