Hi Mike,

You'll find some commentary interspersed. . .

>Heya Jim.
>
>>>I should have been a tad more precise in my comment...my "since the record number is supplied with each index entry (thus providing the offset for the bit in question anyway)" was meant to imply that the bit represented the record's POSITION (ie the Recno is the 'offset' into the bitmap). So the bitmap would of course be in record number order BUT the building of the bitmap could be done from/for any particular index using the offsets to set the appropriate bit ON.
>But what about the index VALUE??? What good is a bitmap of records 'belonging' to an index if the VALUE of the record in question's index is not also there? This is where I suspect that b-tree has some relevance.
><<
>
>However, performance is poor when an order is involved as your code tries to step through records. Without an order performance is better implying that the bitmap is not built according to the index order. This does give the ability to change order while retaining the filter.
>
I have no trouble with the concept/fact that performance *is* affected for non-SQL commands by a SET ORDER. I feel differently about the same in SQL and I think you confirm that below.
I must admit, though, that recent reports seem far more extreme than I ever imagined. This is the type of thing, along with the DELETED() TAG thing, that has me worrying.


>>>But my main concern is a POSSIBLE change in Rushmore's processing which may have significant impact on existing application performance as well as how we deploy it in the future.
>**IF** it has changed (and I've gotta believe that it has, given what is being reported recently compared to the long-standing 'standards' religiously practised by most) then I have some real problems and I think that we all should feel similarly.
><<
>
>Are you referring to the concept of having a tag on deleted()? This is the only change I'm aware of. Although I had a long discussion with a lady and Tamar Granor on Compuserve about the lady's so-called standard of using SET ORDER TO prior to running an SQL command. She swore this was a George Goley-ism. She finally conceded that this practice was useless since timings we ran back then suggested that VFP re-opened the tables in other work areas prior to running the SQL.
>
>In any case, that shows me that as we learn more about the product, that its not so much the product has changed, just our understanding of it. The deleted() tag used to be a standard. Someone wrote an article in FPA recently debunking this belief. I refused to believe it originally, but I'm growing as a human being <g>.
>
Well my worry is that you may possibly be being too "kind" about this. As I said in another reply in this thread, this DELETED() thing has a long history and it surely would have been debunked *much* earlier if its earlier performance was similar to that reported recently. Virtually every time that such a TAG was recommended to someone here on UT, for instance, the report back was that it nicely solved the performance problem. Looks like now it would surely be different!
It is good to "grow as a human being", that's for sure. And if the present method is the way it is always going to be in future, then we can all grow. But... what if this is an error??? What if this (new???) method actually ends up hurting lots of existing applications' performance (after all, I would guess that the vast majority of existing applications have both a TAG on DELETED() *and* only a small potion (if any) of actually deleted records in their tables)??? And it seems that, contrary to my personal practise, most people *do* operate with SET DELETED ON.

>If Rushmore is reading the index tags and building bitmaps, how long does what I call the Rushmore Effect take? If you say SELECT * FROM CUSTOMER WHERE CUST_NAME = "NELSON" and you have a tag on CUST_NAME, this will optimize partially. Let's assume the Rushmore Effect takes 1 second. If set deleted is on, the Rushmore Effect takes an additional 1 second to filter deleted records. If there are no deleted records, we just wasted that 1 second.
>
Except that it appears (from the FPA article) that on a large table your Rushmore effect will be considerably longer! It is this part that is worrisome!

>However, if SET DELETED is ON, SYS(3054) (which by the way is increasing our understanding of what is optimizable) reports partial optimization. That's not to say fastest performance in my example. I've taken to having a deleted tag only while developing the application on my test data. That way I always see full optimization (unless I screw up <g>). I'm certain there will be so few deleted records that I could dispense with a deleted tag. Especially since the user's query will be specifying the record(s) to pull (rarely including sets of records that might have deleted records in their number)
>
>There's another gotcha, if SET DELETED is ON form loading is supposed to be slower (I think it was Jim Slater said this).
>
>>>I remember that you had, when we worked together, two primary interests in FP, those being SQL and data access performance. So I know that you were far deeper into the Rushmore theories than I ever was. I wonder if you might have some old benchmark/test results which could shed some light on this situation???
><<
>
>Thanks for the kind words, I needed some butter anyway <g>. Drew had some performance testing code around somewhere. I'll try to dig that up and post it later.
>
>Want to get together for lunch sometime? Do you ever talk to anyone else from STN?

I'll be happy to have lunch any time. Best to use PRIVATE message to set this up though.

Cheers,

Jim N