Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
XMLTOCursor performance
Message
General information
Forum:
Visual FoxPro
Category:
XML, XSD
Miscellaneous
Thread ID:
00803746
Message ID:
00804036
Views:
38
Hi Paul,

XMLTOCURSOR() internally uses XMLDOM( MS XML 3) which isn't design for processing large XML sets. You may try SAX (Simple API for XML) which generally better suited for extremely large XML documents. The code by Doug Hennig from Re: Can I use SAX in VFP? Thread #613336 Message #613392 may help you started.

>I'm writing an app. that interfaces with a website to download accounting information into QuickBooks. They update the parts list on the site on a fairly regular basis by importing a number of Excel spreadsheets. The vendor that developed the site has provided me an interface where I can download the new parts (all 35,000 of them) whenever I'd like, via an XML file (formatted so that I can use XMLTOCURSOR() on it). Zipped, it's around 600K. Unzipped, it's around 11 Megs. They don't have any mechanism in place to determine which parts are new or changed, which is the reason they're just giving me the entire list. So, I don't have any control over the filesize or amount of information I get. (yes, I realize most apps. don't pass that much data via XML)
>
>I've been finding the XMLTOCURSOR() command, with that size XML file, to be really slow. It took over 1.5 hours to convert into a cursor (it only takes maybe a minute to convert their data into XML via the website). I wrote some code that would do it manually via STREXTRACT which took a little under 2 hours (but had the added benefit of giving me some feedback as it ran, but the downsize of not handling encoded characters). Anyone have any great ideas on how to speed this up? I was thinking about breaking up the file into chunks and feeding that to XMLTOCURSOR(), but I don't know if I'll just end up with more code that takes the same amount of time to run - is the conversion time linear or exponential, or something else? I also thought about using XSL to convert it to some sort of delimited file, then using APPEND FROM to import it. I'd love to hear from anyone that might have tried any of this for real feedback before I waste a bunch of time on these ideas.
>
>Thanks,
--sb--
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform