Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
How to improve the speed
Message
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00185626
Message ID:
00185685
Views:
19
>I need to import 42 000 records from a SDF file into a table. I also need to eliminate all duplicated phone number for the current list. So, before importing each record, I issue a
>
>LOCATE FOR (iList = lnCurrentList) AND (cPhone = lnCurrentPhone)
>IF FOUND()
>   *-- Do not import
>ELSE
>   *-- Import
>ENDIF
>
>This work great. The only issue is that the LOCATE command take about 0.2 seconds on average. That may not seem very long, but for 42 000 records, that take more than 2 hours for the whole file.

I was going to make a similar suggestion to Mark's and David's, but I think mine is slightly different so I'll still suggest it! :) I'd be interested if you have an opportunity to time-test all the approaches and report your results ...

This code assumes TheTempTable has the right field names to match up with TheLiveTable, but fields in the right order to match up with the SDF file:

CREATE CURSOR TheTempTable (Field Definitions) && Or USE predefined temp table.
*SELECT TheTempTable && If neccessary based on option above.
APPEND FROM TheSdfFile SDF
DELETE FOR SEEK(TheKey, "TheLiveTable", "TheIndex")
PACK
SELECT TheLiveTable
APPEND FROM DBF(TheTempTable) && Dbf() not necessary if USEd predefined temp table.

At any rate, if you're reading from the SDF file a line at a time using FGETS(), almost any scheme employing APPEND, massage data, APPEND into Live, will be faster.

If your current scheme is doing APPEND, and LOCATING in the live table for the fields in the appended temp table, all of these APPENDing schemes may work at a similar speed to yours. I think the speed difference between all the suggestions which have been made will be smallish. Please let me know if you can test them all!

Cheers,
Rich Addison, Micro Vane, Inc., Kalamazoo, MI
Relax, don't worry, have a homebrew.
- Charlie Papazian, The New Complete Joy of Home Brewing
Previous
Reply
Map
View

Click here to load this message in the networking platform