>That's a novel, creative idea - I'll save that one for smaller files. Unfortunately, I'm searching a 25-character field (avg = 18 chars) in a 16.25M record file. The new search file is one record per letter in the field and each record has 15 bytes (10 chars + 4 for a key + 1 for delete mark). So 18 * 16.25M = 292.5M letters (records). 292.5M * 15 bytes puts me at 4.4G - so, indeed it will be too large for this particular application.

Unfortunately, I don't know of a better solution for this particular problem.

Well, the idea could be adapted... but you will still use lots of disk space.

For one, you could split the lookup tables horizontally, but this would require several searches instead of one.

Or, you could split by letter, or letter ranges. You would still use lots of disk space, but solve the 2 GB table limit.

Of course, the 10 bytes was only a suggestion - you could make the keys smaller (remember, if you find a match for a record within the first "n" bytes, for longer expressions you will have to check the actual record to see the remaining characters).

The fact that the tables are this big would also make the original creation of these tables take quite a while, and maintenance slow.