>I'm thinking of what you're telling us. Several months ago I was trying to optimize quite long and complex procedure. See this long thread
What is faster: insert from memvar or insert fields? Thread #
1139366 and in particular Message #
1139515>
>I think, in my tests I never tried append blank/scatter syntax and that's unfortunate. May be this would give that procedure a performance boost?
Hi folks,
Well from my tests, I definately feel that if you are going to do an Insert, rather than append blank/gather memvar, that it's best if you don't open the destination table yourself, but rather let foxpro do it. If you don't want to read all of this, please skip to the last paragraph.
I first implemented Walter's advice, and included all of the Set environment settings he suggested, & flock()'ed the table before the insert. I tried the test again and same results, very slow if I opened the table first, and even if I did Flock() before the insert. (The table was opened exclusively as well).
Cetin, Regarding the m., all of the memvars were already prefaced with m. as I was only modifying what was in memory from the previous Scatter Memvar command.
What I did find was that the speed of the operation was different depending on which network share I placed it on, as I expected (the processor speeds of each our servers varies, as does the speed of the hardrives. We have old, newer and VERY old servers here).
To remove the server speed factors out entirely, I placed the test data and and the short program that I posted here on the UT onto my C: drive and executed it from there. Of course everything ran MUCH faster than on any of the servers,
but I still noticed that on my C: Drive, if I didn't open the table before the insert, 10,000 records would be inserted into the destination table in only 1/2 of a second. If I instead opened the table at the beginning of the procedure, the 10,000 records took 5 seconds to insert. I still don't know why opening the table first slows VFP9 down, whereas VFP6 doesn't have a noticeable change in speed. However I hope that I've written here helps anyone else that may be experiencing this type of problem in the future when importing large amounts of data. Though the data we are importing eventually is placed into a SQL backend, we are initally importing 50-80 Gigabytes of it every day into VFP, hence the speed of this operation is extremely important.
