Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Programatically reading/writing LARGE text files
Message
De
21/08/1999 21:07:27
 
 
À
21/08/1999 19:10:08
Information générale
Forum:
Visual FoxPro
Catégorie:
Autre
Divers
Thread ID:
00256287
Message ID:
00256355
Vues:
51
>>I want to build a VFP program that will determine, and allow setting about 30 variables that will be used to add, delete, or modify lines in what is effectively an INI file. The problem is that this file can range up to about 50 or 60 MegaBytes in size. I read in the VFP help file on System Capacities that the Maximum # of characters per character string or memory variable is 16,777,184 … about 1/3 the size of my file.
>>
>>I plan to read the file size, and use a straight forward approach (reading it into memory and manipulating it there) if it's small enough, but now I need an approach that will work with files I find that are larger than about 16 MB.
>
>Wow, this seems to be a change from VFP5. I just looked at the VFP5 help and it says 2GB. But you're right, the VFP6 help says 16MB. I wonder why they did that?!?! I just tried loading a 20MB file with FILETOSTR and it loaded OK. It did take a long time though, probably accessing the swap file. But it does seem like it will handle longer that 16MB...

It will, and it is slower because it pulls the whole file into memory at one time; if you don't have enoguh physical RAM available, it'll use virtual memory (hit the swap file.) It does prove out that a memvar can get very large, though. The problem here is that everything has to be in memory at once, not just some small fraction of the file at any given time, and I'm not sure what VFP is going to do as far as trying to keep other memory-resident stuff in physical RAM. I suspect it isn't going to have much control over it.

My bet is that the system will thrash badly (by thrash, I mean that the system is going to spend an inordinate amount of time moving things back and forth between physical and virutal RAM.) If you do a string operation like:

cHugeMemVar = STRTRAN(cHugeMemVar,'FooBar','MumbleBletch')

you're going to have at least two full copies of the memvar content in memory at the same time. Since we were talking about a file in the 64MB range, that'd be 128MB of virtual memory in use for the two copies, and assuming that you don't run out of swap file space, it's gonna bounce parts in and out of the swap file a whole lot.

Looks like maybe a non-native VFP approach to this problem is in order; I'd look at either using the low-level file I/O APIs directly, or maybe using Scripting.FileSystemObject's file reading and writing cabailities if the file behaves like a text file, or maybe writing something in another language to do the job and calling it from VFP.
EMail: EdR@edrauh.com
"See, the sun is going down..."
"No, the horizon is moving up!"
- Firesign Theater


NT and Win2K FAQ .. cWashington WSH/ADSI/WMI site
MS WSH site ........... WSH FAQ Site
Wrox Press .............. Win32 Scripting Journal
eSolutions Services, LLC

The Surgeon General has determined that prolonged exposure to the Windows Script Host may be addictive to laboratory mice and codemonkeys
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform