Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Adding Large Amounts of Data?
Message
From
14/08/1998 13:38:40
 
 
To
14/08/1998 12:03:13
General information
Forum:
Visual FoxPro
Category:
Troubleshooting
Miscellaneous
Thread ID:
00126944
Message ID:
00126973
Views:
24
Tom,

You can save a little time by skipping the step where you do the append from into a temporary table. Use low level functions and open the text file and insert directly into one of your two tables. This will save some time.

Definetly look through your code for areas to gain performance, I think the seek() function is probably faster then set relation. Is your custom progress bar using the ActiveX control, if so drop it. 750,000 records isn't that much...( relative here, what is your record length). I process about 2gigs of text files from a mainframe a day in Vfox on an NT server do custom processing and send them to 3000 clients via email or ftp. I can tell you VFox performance is terrible for text files larger then 2gigs. We are in the process of moving all parsing routines to C++ and having the mainframe create dbf (2.x) files for us natively on the disk instead of ASCII files. If your ASCII data file is greater then 2gigs you may be wasting your time trying to tune it in VFox. I'm sure it can be done, but it isn't that hard to write a service for NT in C. I would think as long as your record length on 750,000 records is less then 200 bytes VFox should be able to rip through it extremly quickly.

Tune your server:
What is your server doing ? Are other people hitting the box, what else is running?, Is this box a PDC or BDC ? We run a gig of RAM on our NT box that parses text files. What kind of disk caching / mirror / RAID array are you hitting we had some problems with Foxpro and NT doing some nasty disk buffering that really slowed us up.

HTH

Bob Horkay
Previous
Reply
Map
View

Click here to load this message in the networking platform