Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Scan....EndScan
Message
From
09/02/2004 13:09:46
Hilmar Zonneveld
Independent Consultant
Cochabamba, Bolivia
 
 
To
09/02/2004 13:00:25
General information
Forum:
Visual FoxPro
Category:
Other
Title:
Miscellaneous
Thread ID:
00875534
Message ID:
00875540
Views:
10
>Can someone explain what should be happening and the best recommendation on handling this process better. I am scanning a table with 7 plus million records. I would like to decipher only records that begin with particular NSN's and insert those tables into my prod table. Code is listed as:
>
>************
>sele fed_temp
>scan for SUBSTR(NSN,1,2) = '23' ;
>.or. SUBSTR(NSN,1,2) = '24' .or. SUBSTR(NSN,1,2) = '25' ;
>.or. SUBSTR(NSN,1,2) = '26';
>.or. SUBSTR(NSN,1,2) = '28' ;
>.or. SUBSTR(NSN,1,2) = '29' .or. SUBSTR(NSN,1,2) = '30' ;
>.or. SUBSTR(NSN,1,2) = '31' .or. SUBSTR(NSN,1,2) = '39' ;
>.or. SUBSTR(NSN,1,2) = '40' .or. SUBSTR(NSN,1,2) = '41' ;
>.or. SUBSTR(NSN,1,2) = '42' .or. SUBSTR(NSN,1,2) = '49' ;
>.or. SUBSTR(NSN,1,2) = '51' .or. SUBSTR(NSN,1,2) = '52' ;
>.or. SUBSTR(NSN,1,2) = '53' .or. SUBSTR(NSN,1,2) = '58' ;
>.or. SUBSTR(NSN,1,2) = '59' .or. SUBSTR(NSN,1,2) = '62' ;
>.or. SUBSTR(NSN,1,2) = '60' .or. SUBSTR(NSN,1,2) = '61' ;
>.or. SUBSTR(NSN,1,2) = '63' .or. SUBSTR(NSN,1,2) = '67' ;
>.or. SUBSTR(NSN,1,2) = '68' .or. SUBSTR(NSN,1,2) = '69' ;
>.or. SUBSTR(NSN,1,2) = '70' .or. SUBSTR(NSN,1,2) = '72' ;
>.or. SUBSTR(NSN,1,2) = '71' .or. SUBSTR(NSN,1,2) = '73'
>
>scatter memvar
>
>sele prod_table
>insert into prod_table from memvar memo
>
>sele fed_temp
>
>endscan
>
>Can anyone: 1.) explain what exactly the record pointer is doing by doing the scan this way

VFP might use Rushmore Optimization, which means you have no guarantee that the records will be processed in any particular order.

>2.) Recommend a better way to cipher through 7 million records and get only the 1 or 2 million that i want....

However, Rushmore Optimization will only work in this case, if you happen to have an index on SUBSTR(NSN,1,2). You might be better off if you SET EXACT OFF, and use the condition: NSN = '24'... instead.

Also, the condition can be simplified with BETWEEN() and/or INLIST().

For instance:
... for inlist(left(NSN, 2), "24", "25", ...)
Or:
... for between(left(NSN, 2), "24", "26");
  or between(left(NSN, 2), "28", "31");
  or ...
This might again make an index on left(NSN, 2) advisable.
Difference in opinions hath cost many millions of lives: for instance, whether flesh be bread, or bread be flesh; whether whistling be a vice or a virtue; whether it be better to kiss a post, or throw it into the fire... (from Gulliver's Travels)
Previous
Reply
Map
View

Click here to load this message in the networking platform