>>>I have to import a discogs 21GB xml data file.
>>>
>>>Someone wrote a library that uses the API to read files larger than 2Gb.
>>
>>IAC use a streaming parser, not a DOM based one for such huge amounts. In your shoes I'd look at Java / SAX based solution or split the large file into bettter digestable slurps of a few hundred MB with script file API if the XML can be easily parted.
>
>I am writing in VFP,
>using vfp2c32.fll to read the file.

The base OS functions called from there are probably identical to the WinAPI and scripting object used ones. For me the ease of using in vfp editor after declaring COM "typed" variable was the main reason ;-)

The MS parsers AFAIR had ISTREAM implemented, no idea if those are able to access >2GB in the latest implemetations, eliminating need for preprocessing.

With Java parsing chances of "just reading" are probably even better as 2GB should not matter, although with Java usually the glueing together is more work than with MS tools.

Still, one of the cases where vfp might not be the best choice to do all in, as working on laaaarge strings even if able to access in files >2GB can be much slower than in languages like C, C# or Java. But you are in a better position to judge on the complexity and speed requirements of the task ;-)