[Subtitle: How VFP's cache is mapped onto the network tables]

Jim wrote :

>I assume that VFP stores this data in its "cache" and that it either doles out a copy of the specific record in process or points to the specific record's starting position within its cache. It seems reasonable to assume that it definitely uses a copy when buffering is in effect.

The table buffering not taken into account, but the usual record locking within the multi user system to be taken as in effect :

Personally for me, this has always been the big puzzle with regard to the solution of certain corruptions, and for sure performance considerations. I take both of these not the explicit subject here. Instead I sure would like those who really know about the internals of VFP, to respond to the next :

For me it is 100 % sure that with the normal VFP DML commands, VFP's data engines can't be flawed when it comes to corruption, or decent multi user behaviour in general. At the same time however, I can reason how things could work, and these scenarios lead me to flawing easily. However, even knowing the (theoretical) scenarion's, I can't let it flaw.
All of this has to be looked at in the light of corruptions being there anyway, but which can't be explained so far. And for me counts : as long as something can't be explained, I won't mark it as a bug. That is, I won't tell what's the cause of it. And as we all know, say, 95 % is caused by improperly working client software (MS and Novell equally).

I have some perceptions of the "cacheing" operations of VFP, I'd like to be shot at. Or better, to be worked out as to how it really is.
It is the most easy to describe a scenario as my perception :

1.
At the server we have a VFP table. It comprises of, say, 4KB blocks (clusters). At some moment in time, 100 blocks are in there. The last block just cannot hold a new record to be added.

2.
User A wants to change a record in block #100. He performs an RLock(), causing the block to be refreshed in his PC.
Never mind preceeding SEEK's etc.; the RLock() has to be performed in order to receive the latest version of the block. In the end it is about the latest version of the record, but the block is received in the mean time.
Supposed 3,700 records are in the 100-block file, and this is about record #3698.

3.
User B wants to add a new record, and all he does is an Append Blank.
This implies a lock of the header of the table, so not two users at any same time will write to the header. Hence, the append blank from User B will result in incrementing the record count, in this case from 3,700 to 3,701.
After the header is updated, it is unlocked again.

Now all my assumptions start ...

4.
User A performs a replace at record # 3,698. He is not performing an unlock yet. Hence, the new data from record #3,698 will stay in User A's cache only. No fresh copy of the block is obtained, because it is not necessary; the record is already locked.
It is here where I can be dead-wrong.

5.
At the append blank of User B, User B will obtain a fresh copy of block #100, because its last portion contains the beginning of his new record 3,701. User B also receives block #101, because in there is the remainder of record 3,701 (the record contains blanks only at this moment).
Note that User B will not obtain the record #3,698 as how User A already changed it. User B will see the old contents of it (when he'd look for that).
Also note that User B is legitimatly able to obtain a fresh copy of block #100 (as he just did), but that "fresh copy" means : as how it is on the server's (virtual) disk.
It is my statement that by, for example, performing an RLock() on record #3,699, all records from block #100 will be refreshed to the status as how it's on the server's (virtual) disk. A user could perform any RLock() upon a record in block #100 (say records 3,663 u/i 3,700), but for record #3,698. Hence, this record will fail to get locked at this point in time, because User A has got it locked for himself. And, because the lock won't be done, the block will not refresh either.

Note that in between this all, the VFP activity on the Set Refresh is operating, and that that might cause a refresh of the block afterall. But again, the contents of record # 3,698 as how User A sees it right now, will never be obtained for that contents : it's just not on the server's (virtual) disk.

6. User B, who just appended the new record, is performing an Rlock() to record #3,701, for his due replace.
In between the append blank and the replace there is a point in time we can let it all flaw. That is, when no file locking (FLock()) is used, within this period the appended #3,701 can be obtained by someone else, and can be replaced by someone else. But, this will be shooting the "system", whereas by no means logic can be found in such an operation.

7.
At this stage, User A unlocks record #3,698 and because of that it will be flushed to the network's disk. "It" means the whole block #100, because transfer is at the block level.
How will this physically work ? It hardly can be true that some "dumn" file system is able to merge the new block #100 from User A with the current new block #100 as implied bu User B. Or can it ?
Two options exist :
a. The block is re-fetched to the PC first, and all the intelligence necessary is applied by VFP;
b. The block is not re-fetched, and the file system is performing a merge.
I can only fo for a.
Assumed that a. is the case, it won't be allowed to receive record #3,701 because it is locked. Furthermore, what about the #3,701 being spread over block #100 and #101 ? Will User A receive Block #101 anyway, anticipating on it now logically being connected to #100 ? Would a Go Bottom imply the presence of #3,701 anyhow ? I think so ... But, User A can't go around with replacing #3,701 because it is locked by User B.

8.
Now User B is performing the replace, and when the assumptions so far are right, it will imply a re-fetch of block #100 and #101 first. The data from #3,701 is now applied to both block #100 and #101.
User B is now performing an unlock for #3,701.

9.
As I assume it, now User A is according my statements at the end of #7, capable of replacing #3,701, overriding the contents as just applied by User B. But again, this is not applying proper app logic, because first User A should RLock() #3,701. When he does that, he will obtain the fresh contents of block #100 and #101, record #3,701 being part of it.

All poper working depends on the re-fetch of the data, even when the record concerned was locked already. I personally wonder whether this is true ...

The above looks like a nasty thing, and whether my assumptions are right or not, including this "knowledge" I was never able to let things flaw with tests anticipating on this knowledge.
Though this is about the simple situation not having indexes around, including indexes give me the same result so far. It's just 100 % decent. Or ?

Peter