Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Processing large volumes of data
Message
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Divers
Thread ID:
00419465
Message ID:
00419673
Vues:
21
Shane, that's an awful long time for only 200,000 records. We do imports of ASCII files that big all the time and it generally takes little more than 5 - 10 minutes. Even 500,000 records only takes about 25-30 minutes.

I'm curious what your methodologies are:
- Are you populating numerous tables?
- Is there a great deal of string parsing?

I'll take your word on the fact that making the code more efficient is difficult, but it seems to me that there should be something relatively simple that could be done to speed thinks up.

Notes to your ideas:

>
>Some ideas:
>1. Split the load file between multiple computers

-- This probably would not help a great deal. Could actually be longer with file locking issues.

>2. Sort the load file and the table by loan number and use scan/skip

-- I'm not sure what you mean by this, but my instinctss say that there might be a flawed process right here

>3. Try to make code more efficient (difficult)

-- see above

>4. Get the client to use a faster PC.

-- would probably help, but I suspect it won't make a large enough difference to comppensate for a growing dataset

>5. What else can I do?
I try to stay online, to get better information.
I try to get away from the office once in a while..
But they KEEP... PULLING.. ME.. BACK.. IN!!!
Précédent
Répondre
Fil
Voir

Click here to load this message in the networking platform