Thank you Darrel... these are some very good points
>From what I can tell your modular and data breakdowns are logical and I don't think you have gone over the edge in any way :-)
>The one area that raises my suspicions a little are your "Processed Tables" and "Report Tables". Every time I have attempted to create psuedo tables for the benefit of processing I have regretted it at a later point. The maintainence over head is very painful, but I guess that's why you are working with xCase. I suspect that many of these psuedo tables could be replaced by views or stored procedures that generate cursors. If you really must make the data persistant, could you not create tables using SELECT INTO TABLE and then clean them up afterwards.
>
I can understand your questioning my "Processed Tables", I didnt go this route the first time around with my 2.6 app. And yes, I will substitute a view or cursor wherever I can. But some of my tables are millions of records, and since I have no good primary keys this makes anything I join together time consuming. My thought is to use views/cursors in place of the Report Tables at the end of the modules, keep my psuedo-tables from Preprocess, and perhaps dump out the original source data to help with disk space. With the transfer, Restock, and Preprocess modules run at night, the replication makes up for itself with user response because when a report is requested from the app, half the work is already done.

>The other advantage of views and cursors from stored procedures is it is usually pretty easy to determine where the data came from when people come questioning.
>
When I'm questioned is over the phone... and I need to answer in terms of where the data came from in relation to the unix boxes. :( And with Stored Procedures, dont I want to avoid them since I'll be upsizing to SQL Server once the Data Warehouse System is in place? I thought I learned somewhere Stored Procedures and Triggers would be lost in Upsize process? The way I've been designing the actual processing is with all methods, straight *.prg procedures, and these cool Event Objects from the Mere Mortal framework.

>One technique that I have always wanted to really test out on a serious application is the OBDC text file driver. Using VFP's remote views and the OBDC textfile driver it is possible to do SQL's directly on the external text files. This means that your pre-process step could generate the joined and replicated tables directly from the text files without the APPEND FROM step. I have no-idea what the performance of this driver is like in comparison to the append from. It also means that you would now have a s an OBDC Datasources to manage for each text file. But it would remove a bunch more tables. I just thought I would mention it.
>
Now this is really interesting!!! If I can eliminate the overhead of Zap/Append/Maintain Tags just from a programatic standpoint, shoot minor performance hits might be worth it. I never heard ODBC did textfiles... almost sounds like god's gift to Rox in regards to this project if it works! (I currently feed 7 text files into each of my source tables). Thank you Darrel... this is alot of good insights.