Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
CURSOR Experiencing Memo Bloat
Message
From
15/07/2006 08:07:51
Neil Mc Donald
Cencom Systems P/L
The Sun, Australia
 
 
To
14/07/2006 21:17:21
General information
Forum:
Visual FoxPro
Category:
Troubleshooting
Miscellaneous
Thread ID:
01136610
Message ID:
01136656
Views:
12
Hi,
You only get memo bloat if the text etc being written to the memo is greater than what the memo contents already are. In this case the memo record is deleted and the new data is written to the end of the file.

The workaround I would suggest is to work out what the largest amount of memo data would be, then write dummy data to the memo at 120% of this value.

This should stop the bloat as the data being written would always be less than the current memo size.

If this doesn't stop the bloat there is some other process at work.

Hope this helps.

I would use the NOFILTER clause as it is the only way of ensuring that the cursor is created, as sometimes VFP will only apply a filter to the original file and not create the cursor.

>I have an issue that nobody is going to believe without screenshots, which I can provide if necessary. I have a cursor that is experiencing memo bloat. The cursor is created in a program that is part of an established commercial software package. I cannot replicate the problem outside of the software package in any code snippet. Here are the facts:
>
>1. The cursor is confirmed to be exclusive via LIST STATUS. Of course, a cursor can be nothing but exclusive, but I was looking for any clues.
>2. The cursor is created by a SELECT query, has one record, and is created by using both the READWRITE and NOFILTER clauses.
>3. The cursor, by design, has a single memo field in it that is replaced many times (we'll take 1000 as an example). The memo file bloat size is directly related to the size of the text being replaced multiplied by the number of replacements. The "average" size of the text is around 2 MB, so as you can see, we reach the 2 GB limit for a memo file size VERY quickly. The program ultimately bombs very early on because of this.
>4. The database and memo files associated with the cursor are both .TMP files automatically generated by FoxPro. Throughout the entire program they are confirmed to never be re-created (the file names stay the same).
>5. This is VFP 9 with the latest service pack.
>
>Now for the kicker:
>
>1. Putting a suspend in the replacement loop and executing the following line will SHRINK the memo file size to 2 MB:
>
>SELECT * FROM INTO CURSOR READWRITE NOFILTER
>
>While this is a "fix", the NOFILTER clause is extremely slow and causes two new .TMP files to be created each and every time. If this line is called many times, performance tanks. I've put a patch in where this line is called every 25 replacements, but the logic is sketchy, at best. If the replacement string happens to be large, 25 is too high of a number. If the replacement string is very small, 25 is way too low of a number. If no solution to this is found, I will do some calculations based on some average sizes at runtime, but that would be extremely nasty.
>
>2. This concept is difficult to convey, but it has now kept me awake for nights... If I suspend program execution in the loop and then, from the command window, launch a program which does a replacement of the field 100,000 times in a tight loop with a memory variable that is about 2 meg, the memo file NEVER GROWS. It stays at the size it was when the program was suspended. That could be 20 meg, 200 meg, or 1.2 gigs. The ONLY growth is experienced during program execution.
>
>My suspicion is that the bloat is the result of many parts of the software environment working together. For example, the variables that I use in the "replace" are local variables -- I tried to research that a bit to no avail. The program can call itself recursively, so often the call stack is 10 deep -- that seemed to go nowhere. It has to be a combination of many variables, but the software is a major package and is thus, huge.
>
>Please don't take this the wrong way, but I am not looking for a "workaround". The problem has been pared down to its core, but you are going to have to take my word that it needs to work the way it does. I know how to make it not bloat many different ways -- all of which hurt performance drastically. What I am looking for is anyone who would have a clue as to how a cursor in "memory" could EVER experience memo bloat.
>
>Any help would be greatly appreciated.
Regards N Mc Donald
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform