Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Adding Large Amounts of Data?
Message
From
14/08/1998 13:04:59
 
 
To
14/08/1998 12:03:13
General information
Forum:
Visual FoxPro
Category:
Troubleshooting
Miscellaneous
Thread ID:
00126944
Message ID:
00126965
Views:
27
>I am loading a large amount of data from a text file.
>The data is APPENDed to a cursor and then that
>cursor is SCANed and then INSERTed into one of
>two tables. One of the fields is used in a lookup table.
>
>The lookup process was initially a SEEK() and I
>changed it to a SET RELATION. The two targe tables
>have an integer primary key. I was using a stored
>procedure to increment the PK, but am now just
>incrementing a variable in an effort to speed things up.
>
>The data being loaded from the temp cursor has about
>750,000 records. I am using a custom progress bar
>that tells me the estimated time lef to complete the task.
>
>The first 10% of the data was loaded very fast and had
>an estimated time to complete of 2 hours. The process
>is now 28% complete and the estimated time is approaching
>7 hours remaining.
>
>Is there any tricks to get the same performance at the
>beginning of the file as the rest of the file? Perhaps
>something like FLUSHing the data? The two target tables
>(added to via INSERT INTO) already had nearly 500,000
>records each when the program started so it is not the
>fact that the target tables where initially empty.
>
>I am runnig this on a NT server using VFP 5.0.
>
>Any ideas on how increase the speed?...

Two quotations taken from the VFP Online Documentation:

"When adding indexes to your tables, you must balance the benefit you get in retrieval times against a performance loss when updating the table. As you add more indexes to your table, updates and inserts to the table are slower because Visual FoxPro needs to update each index."

"If you need to append a large number of records to an indexed table, it might be faster to remove the index, append the records, and then recreate the index."
Previous
Reply
Map
View

Click here to load this message in the networking platform