Factors such as latency to update filesystem comes into play when you've got that many files. Speed of the datalink comes into play too -- the slower the datalink, the more pronounced the latency for updating the directory files.
It reminds me of the discussion that came about at the office many years ago (DOS6/Win3.1 and Win95 machines with parallel port ZIP drive). My supervisor had tried to prove me wrong (I was PKZIPping the files first, then copying the ZIP file while he was suggesting just simply copying the files over to the ZIP drive "and not waste time by ZIPping") by proposing a "race" -- and ended up proving that my method was indeed faster. My ZIP-then-copy operation finished before the direct copy operation, despite the fact that I'd started around 5 minutes later (we ended up aborting the direct copy after letting it run half an hour longer after I'd finished on my computer -- the progress bar was still indicating less than 50% completion at the time).