Has anyone else done anything simultaneously using multiple computers to crunch data? I'm thinking about writing a system that will use from 30 to 100 computers at night when they are just idling. I plan to put a stub program in the scheduler that will launch at say, 8:00 pm, then check in to see what tasks are there to be run.
I'm thinking the computers will idle for about 5 minutes with a timer, then once all have logged in to a table, I'll know how many pcs are there, and can then distribute the workload. The first application will take about 50,000 records which will be used for geocoding. Currently, I get about 3600 hits per hour on one machine. Using 30 - 100 computers, it wouldn't take very long to process that entire file.
This is all theory right now, but there's no doubt it will work. Any comments? Another thought is to even bump this up by launching two programs per pc.
Comments anyone?
John Harvey
Shelbynet.com
"I'm addicted to placebos. I could quit, but it wouldn't matter." Stephen Wright