Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Big XML
Message
From
10/12/2015 04:28:49
 
 
To
09/12/2015 11:29:40
General information
Forum:
Visual FoxPro
Category:
XML, XSD
Title:
Environment versions
Visual FoxPro:
VFP 9 SP2
OS:
Windows 10
Network:
Windows 2008 Server
Database:
MS SQL Server
Application:
Web
Miscellaneous
Thread ID:
01628714
Message ID:
01628766
Views:
63
>>>I have to import a discogs 21GB xml data file.
>>>
>>>Someone wrote a library that uses the API to read files larger than 2Gb.
>>
>>IAC use a streaming parser, not a DOM based one for such huge amounts. In your shoes I'd look at Java / SAX based solution or split the large file into bettter digestable slurps of a few hundred MB with script file API if the XML can be easily parted.
>
>I am writing in VFP,
>using vfp2c32.fll to read the file.

Did you try if oSAXXMLReader.parseURL(strURL)
https://msdn.microsoft.com/en-us/library/ms760372%28v=vs.85%29.aspx

can handle a 21 GB file? If so, "parsing" in vfp would need only to create the fitting implementations of the needed interfaces of the SAX parser according to the functionality needed in your case. Probably much faster, unless there is a specific begin/end pattern repeated in the file you can use to slice the 21GB into practical data chunks with strextract()
Previous
Reply
Map
View

Click here to load this message in the networking platform