>>
>>With CSVs, one thing I've done in the past:
>>
>>- Before importing, read in the file to a string variable using FILETOSTR()
>>- Copy the top row (column headers/names) to a separate variable etc. This is everything up to the first carriage return/line feed combination i.e. CHR( 13 ) + CHR( 10 ) . You can then parse that variable to get the column names
>>
>>Then import normally using the CSV option.
>>
>>Actually, thinking about it you can get the headers with FILETOSTR() either before or after the CSV import, the timing doesn't matter.
>
>Sounds very inefficient on large files ;-)
>Using FGets should always give you nearly linear performance, as only first line is read independant of file size. as long as first line smaller than the 8K limit. For those cases breaking the limit and also very large, alines() on a FRead of 65K should be faster and similarly timed
The cases where I've done this are 4 calls in an import process that overall takes about 40 minutes - they haven't been the bottleneck so far ;)
The largest CSV files I've had to deal with have been only ~1M. If you think about it that might even fit in current CPU caches :)
Still, your points are well taken - if I have to deal with large files or run this process a lot I'll experiment with the LLFFs.
Regards. Al
"Violence is the last refuge of the incompetent." -- Isaac Asimov
"Never let your sense of morals prevent you from doing what is right." -- Isaac Asimov
Neither a despot, nor a doormat, be
Every app wants to be a database app when it grows up