This may not be relevant, I don't know the details of the operating system or the network you're using, but some years ago using DOS & IBM Token -Ring network, I discvered that there is a rapid increase in directory search time when the number of files in a directory exceeds a multiple of 256, & this happens on each increase in multiple of 256.

I was doing a similar search of a large directory, if the file was in the 1st 0-253 files in the directory the access time was almost the same, in the 254-509 entries took a lot longer but similar in time, from 510 to 766 took an even longer time but all around similar time.

Graphing the access time, showed sudden drops at 254, 510 etc. (The discrepancy of 2 from multiples of 256 is accounted for by the . and .. subdirectories). This problem continued even when we changed to a different network OS. The way we solved the problem was to split the files up into multiple subdirectories containing less then 254 files each.

The problem may not be the same (& the solution not acceptable), but it may be worth graphing access times for each file in sequence to see if there is something like this happening.

Paul