>The reason I say this is that I'm beginning to believe that unless such tests show a significant (10%+?) and consistent difference, they're meaningless. Why? Win32 is a pre-emptive multi-tasking environment and there's nothing that can (or better, should) be done to change this. This means, to me, that because of this any such test may be impacted by the current program losing its time slice of the processor cycles. I believe (and this is a guess) that this may be the reason that I saw such inconsistent results.
This would be correct. When doing performance testing, I make sure that I have nothing else running on the box. Even on my multi-proc box, if I'm doing multi-threaded testing, this is an issue. Check the docs for the MS Web Stress tool, and it says to not run the test on the same box as the server, because being a client
and a server can affect results. If the standard deviation of a half dozen runs is much out of whack, I figure I've missed something in how I set up the test and try to track down what I may have done wrong.
Mike Stewart