Hi Gar,

>Thanks so much for your help.
>
>My is not a straight recreation job -- more a data cleaning job that involves reading the same stuff over and over again, and changing a substantial portion of it. The reason for the Zeon chip is that it is required to get maximum speed in passing data between disk and Ram if we really are going to take tadvantage of the all this Ram.
>
While I surely would like to have a XEON to play with, I really can't see it making any significant difference. One of the more recent ATC chips would save considerable bucks and be, I would guess, 99+% as effective.

>From what you are saying the Raid will buy us quite a lot, so we should definitely stick with it. This make sense: database == faster the disk the better.The reason we are doing 5 rather than 0, is that 5 buys us some extra reliablity, whereas 0 is LESS reliable than not having raid at all.
>
>For the most part network suport is not an issue. The really big jobs will be done on a stand alone basis. We will be doing some manual error correction that involves accessing the data from a remote station -- but these will also involve getting very few records, making some changes at the client end and than passing it back. I suspect the speed on these is more affected by decent applicaton design than hardware. In the batch jobs, network stuff is (as you say) not an issue.
>
>The question still remains -- is the RAM worth it? We can end up almost within hardware budget if we get the 512 Meg of RAM, and go about 50% over hardware budget if go to 1 Gig of Ram. But hardware cost is trivial compared to developer cost, and the cost of having to wait a long time for jobs to finish. So if the Gig will buy us something real, I have no problem fighting for it. That is the question. Will the Gig get us improved speed. The only thing I know on this is that Microsoft says the more RAM the better, and some people on this thread have added that RAM only does you good if your chip can access it directly. (Most modern chips and motherboards can.)
>

I see from later replies that you have decided to get the RAM. A good move I think! Remember though that you will likely have to play with some settings (VFP almost certainly and OS too if using NT) to make best use of the stuff.

Cheers,

Jim N


>Any able to confirm the above? Or deny it? I checked at the Foxpro Advisor web site, but unfortunately most of the key articles are available only on the CD or the print adition and not at the site. I will have to check out some back issues in the library but meantime ...
>>Hi Gar,
>>
>>I have read Brett's replies. His sounds like a straightforward recreation job. Is that also the nature of the work you have to do?
>>
>>You say standalone workstations, implying that networking is *not* at all a consideration. Is that correct, and would you also plan on running only 1 such job at a time on a workstation?
>>
>>Some general comments on your "superstation" configuration:
>>
>>I doubt that XEON is required or will buy you much at all (based strictly on reading, *not* on actually using one).
>>A job I had involving HUGE files and lots of processing (about 5 years ago) used a StorageWorks disk array and It seemed to be fantastic. No doubt they have gotten much better since then too. While it was configured for RAID5, I'm not so sure that was the best option. My *guess* would be that RAID0 would be best for strictly speed. But again, depending on the nature of the work this might also be overkill. Maybe some of the top level SCSI drives available are enough, especially if attention is paid to spreading key tables across different drives.
>>I believe that the more RAM the better, but this no doubt depends on how you can control its allocation for the OS, buffering and VFP.
>>
>>Good luck with this
>>
>>Jim N
>>
>>
>>>>>We are looking at some batch processes involing large databases, including some individual files near the gig range. Mostly these jobs will be run in exclusive mode, using standalone stations.
>>>>>
>>>>>I have been asked to spec these stations.
>>>>>
>>>>>I was thinking of zeon chips, raid five, and 1 Gig of Ram with Perc 3 raid controller and 128 Meg of cache. The question is will this really gain a great deal? Will 1 Gig really give us a significant gain over 512 Meg or 256 Meg. Is raid going to buy us that much gain in I.O. In general can anyone suggest the best configuration for this kind of thing?
>>>>
>>>>When we have to load data here from the Army. I use a Pentium III 450 128MB Memory PC to process the files.
>>>>
>>>>Statistics
>>>>
>>>>3 Tables
>>>>frn 15,000 records 12 Indexes
>>>>lot 600,000 records a memo field and 20 Indexes including a PHdbase Index
>>>>comp 5,500,000 records 3 Indexes
>>>>
>>>>The original tables are zapped, indexes dropped, and everything is recreated.
>>>>The File processing will take about 3.5 hours, which is plenty of time to accomplish this as a nightly process.
>>>>
>>>>This biggest processing is the creation of the Phdbase Index on the memo field of the lot table (about 2.5 hours of the 3.5)
>>>>
>>>>Total DBC and table sizes when finished is approx 1.7 GB
>>>>
>>>>Just my 2 cents and my experience with large data files.
>>>
>>>That is helpful. But the question is would my "super" configuration run the same job in one hour? Or a half hour? Or 15 minutes? Or would it take the same 3.5 hours? The reason I am asking is that we have a LOT of these large jobs to run. Also even if we would gain from an improved machine, would we gain from one this improved? Is any of this sheer waste? For example will we seen NO detecable gain between 512 Meg and 1 gig? Or a few seconds. Some of the VFP 6.0 documentation suggests that VFP can take advantage of as much memory as we can give it. But is this really the case? Ditto raid 5 scsci disks are the kind of thing you use for SQL server. Will VFP benefit signifcantly with large files?