Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
CURSOR Experiencing Memo Bloat
Message
From
18/07/2006 00:33:47
 
General information
Forum:
Visual FoxPro
Category:
Troubleshooting
Miscellaneous
Thread ID:
01136610
Message ID:
01137143
Views:
27
Hi David. pmfji - I have a file which does not get updated often, has only a few hundred records, but does contain a memo field which can hold a large amount of text, +/-5Mb. Just to be clear, are you saying that it might be beneficial to pad my text with chr(0) out to some maximum possible size and then to ensure that each replace always pads out to the same size? i.e. ensure that each memo field is always, for example, 6Mb in my case?



>John, (a later update that I couldn't post because Neil got a reply in while I was editting the post)
>
>You have a mistaken assumption. Whenever the memo size increases beyond the allocated block VFP is going to reallocate a new block of fpt to hold the content. This behavior goes back to VFP6 which is the oldest version I still have installed.
>
>
>set blocksize to 32
>create table employee ( id i, notes m )
>insert into employee values ( 1, replicate( "x", 2000016 ) )
>use in employee
>
>create cursor sizes ( iMemosize i, iFPTSize i )
>
>FOR ix = 1 TO 100
>   use employee exclusive in 0
>   lnSize = max( 1, int( rand() * 2000000 ) )
>   replace notes WITH REPLICATE('a', lnSize ) in Employee
>   use in employee
>   adir( laFile, "employee.fpt" )
>   insert into sizes values ( lnSize, laFile[2] )
>ENDFOR
>
>select sizes
>brow last nowait
>
>
>As long as the memo size is the same or smaller number of blocks it is put back into the same place. Whenever it increases in size the old allocation is trashed and it is completely reallocated to new space at the end of the fpt. It does this so that it doesn't have waste time to check the block allocation of every record and memo.
>
>This wouldn't be necessary if VFP stored 3 integers in the memo header: deleted, current size, allocated space. Heck it'd be nice if VFP would reuse deleted memo space too, but that isn't in the cards either.
>
>This is a little easier to grasp if we look at a more controlled set of memos:
>
>
>set blocksize to 64
>set exclusive on
>
>* creates two memos the first needs 2 64 byte blocks and the second only needs 1
>
>create table bloat ( id i, notes m )
>insert into bloat values ( 1, replicate( "x", 100 ) )
>insert into bloat values ( 2, "second memo" )
>use in bloat
>
>hexedit( "bloat.fpt" )
>
>* update the first memo, size is still under 64 * 2 - 8, so the a's write over the x's
>
>use bloat
>replace notes with replicate( "a", 110 )
>use
>
>hexedit( "bloat.fpt" )
>
>* this memo now exceeds the two blocks it has allocated, so it requires 3 blocks at the end of the file.
>
>use bloat
>replace notes with replicate( "b", 140 )
>use
>
>hexedit( "bloat.fpt" )
>
>* again the size is under 3 blocks so the c overwrites the first b
>
>use bloat
>replace notes with replicate( "c", 1 )
>use
>
>hexedit( "bloat.fpt" )
>
>
>run the above code through each hexedit() call at a time and examine what goes on inside the fpt.
>
>Since VFP only stores the current length of the memo it doesn't know the total space "reserved" in the fpt. This sort of logic is used:
>
>
>if ( ceiling( len( mNewMemoContent ) / ( blocksize - 8 ) ) > ceiling( len( mOldMemoContent ) / ( blocksize - 8 ) )
>   * allocate a new block
>else
>   * still fits where it was
>endif
>
>
>In other words if your memo has 3 blocks allocated, then goes down to 2 and then back up to 3 VFP doesn't know that it can still put it back into the current space it needs to put it at the end of the file.
>
>Despite all your other "I can't possibly change this code" statements, you seriously need to rethink your algorithm and change it to something that doesn't rely on memo fields if you are constructing a memo that is always increasing in size.
>
>>I have always relied on the fact that memo bloat does not exist in any cursor or table that is opened exclusively. This issue has made me confirm that many, many times now. I've created a table (and also a cursor) with a single memo field in it. I have then made a tight loop, replacing the contents hundreds of thousands of times with no problem. So either the memo bloat can occur if "foxpro is really busy" (which I find really hard to believe) or there is something else at work here...
In the End, we will remember not the words of our enemies, but the silence of our friends - Martin Luther King, Jr.
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform