Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Bulk loading of text data (CSV files)
Message
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00169303
Message ID:
00169330
Views:
24
Have you tried APPEND FROM myfile.csv DELIMITED? You can just create a cursor with the appropriate fields, then do the append. If you need any indexes, create them after the append. I'm not sure how that will compare to low-level file functions, but there's only one command so you'll definitely have less interpreter load.

You could also just copy out the files to DBFs and zip them. When you needed to get at an archived table you would just unzip the DBF.

>Each day we have a VFP application write "old" transactions to a monthly text file for archival purposes (fields are alltrimmed to remove blanks - in order to save space). Each of these monthly text files are written in a CSV format (fields separated by commas, fields surrounded by double quotes if the field contains embedded quotes and/or commas). Another VFP utility must be able to load these archives in order to query and print a report.
>
>The problem is that these "text" files can contain an average of 100,000 records. Each record can be 300 to 600 bytes long. I am trying to find a means of "bulk" loading this data and then parsing it (turning CSV format into fields, vars, array elements, or whatever to manipulate the data). Using the low level file commands work great, but takes why too long. A user might need to load 3 of these archive files - this could take hours!
>
>As I'm sure others have discovered, VFP does not understand the CSV format so I need to deal with it myself. Has anyone had similar issues? Have you found a fast way to buld load text data? Is VB or C any faster? Should I write a C routine and link to it?
>
>Any Idea? Please help.
>
>Thanks
>Dean
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform