Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Pulling Data from a Huge Cursor
Message
From
04/03/2014 06:55:55
 
 
To
02/03/2014 13:01:13
Mike Yearwood
Toronto, Ontario, Canada
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Environment versions
Visual FoxPro:
VFP 9 SP2
OS:
Windows Server 2012
Network:
Windows 2008 Server
Database:
MS SQL Server
Application:
Web
Miscellaneous
Thread ID:
01595090
Message ID:
01595720
Views:
57
Thanks Sir, Tonight I will incorporate all changes you and Thierry Nivelet Sir has suggested and let you know the performance.

Regards
Harsh


>Hi Harsh
>
>I have no problem helping you. You are attempting to do the work and are asking for better ways to do it. You do not seem to be saying "here is my problem, write me the code for free".
>
>I know if someone is asking for help it is because they don't yet know how to do it better and want to learn. They may also know a lot, but seek a better idea.
>
>You are trying to process a page at a time. I really want you to provide a few sample pages and a few sample sets of records so I can write code using that. Then I can post code for you to try. You can learn from the code and others can provide more input.
>
>>Hi,
>>Kindly Consider the following code. The Code Is Extracting Data Between Two '%%Page: 1' From a Cursor.
>>
>>There are over 1800000 Records and I need to Extract Data From Start of '%%Page: 1' to One Prior Record Before next '%%Page: 1'. Than again From Next '%%Page: 1' to One Prior Record Before next '%%Page: 1'.
>>
>>But Gradually The Extraction becomes Slow and becomes very consuming.
>>
>>Is there can be any SQL for such Purpose that May serve better
>>
>>Note:- Please Do Not suggest Indexing as I will not serve my purpose, perhaps.
>
>I think you need to know more about how FoxPro thinks so you might be better not putting limits on the answers.
>
>I absolutely insist you have an index on left(alltrim(pos),9). VFP will use that index to find the page records very fast. Since you are processing so many records, you need all the speed you can get.
>
>Every time a LOCATE FOR executes it takes a chunk of time. Generally speaking, if you have the correct indexes it will take less time.
>
>FoxPro has SQL in it, but RECNO() is not a SQL concept. It is not optimizable and will not be fast for SQL. Better to use FoxPro commands instead.
>
>>
>>
>>SELECT ALLBILLS
>>SELECT COUNT(POS) FROM Allbills WHERE LEFT(ALLTRIM(pos),9)='%%Page: 1' INTO ARRAY nNumberofbillsincurrentfile
>>nBillend=0
>>FOR I=1 TO m. nNumberofbillsincurrentfile
>>SELECT AllBILLS
>>LOCATE FOR LEFT(ALLTRIM(POS),9)='%%Page: 1' AND RECNO()>m.nBillEnd && Bill Starts From Here and This is the First Page
>>nPageonestart=RECNO()
>>LOCATE REST FOR LEFT(ALLTRIM(POS),9)='%%Page: 2' AND RECNO()>m.nPageonestart && Page One Ends Here
>>nPageoneend=RECNO()
>>LOCATE REST FOR LEFT(ALLTRIM(POS),9)='%%Page: 1' AND RECNO()>m.nPageoneend && Next Bills Starts From Here
>>nBillEnd=RECNO()
>
>GO m.nPageOneEnd
>lcFileName = STREXTRACT(POS,'(',')')
>
>>LOCATE REST FOR STREXTRACT(POS,'(',')')='Account Number' AND RECNO()>m.nPageoneend AND RECNO()<m.nBillend
>>SKIP
>
>
>Don't use oFilename. o is usually used for an object variable type. This will be very confusing. It should be cFileName
>>cFileName=STREXTRACT(POS,'(',')')
>>cFileName='D:\HARSH\'+oFileName+'.Ps'
>
>Are you near the record you want below after the SKIP above? LOCATE FOR cannot optimize the following without an index. But if the next few records after the above contains the string you seek, you could do LOCATE REST FOR and it will get there really fast.
>
>>LOCATE REST FOR ALLTRIM(POS)="20.699 184.238 MV (. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .)S"
>>Replace POS WITH '20.699 184.238 MV (. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . )SH'
>
>If the above is really what you mean to do, it seems the only difference is adding an H to the end of POS. Here's how:
>REPLACE POS WITH ALLTRIM(POS) + "H"
>
>
>SELECT cannot use RECNO() to optimize. DO NOT ADD AN INDEX ON RECNO()!!! Very dangerous.
>>SELECT POS FROM Allbills INTO CURSOR Splitfile Where RECNO()=>m.nPageoneend AND RECNO()<m.nBillEnd Readwrite
>
>I think you might try..
>GO m.nPageoneend 
>COPY REST TO splitfile.DBF WHILE RECNO()<m.nBillEnd
>USE splitfile in select("splitfile")
>SELECT splitfile
>
>>Set Textmerge ON
>>Set Textmerge To (m.cFileName) Noshow
>>SCAN
>>\\<< TRIM(POS) >>
>>\
>>ENDSCAN
>>Set Textmerge To
>>Set Textmerge OFF
>
>
>
>*DO NOT EVER USE RECNO() with SQL commands.
>>DELETE FROM Allbills Where RECNO()=>m.nPageonestart AND RECNO()<=m.nBillEnd
>
>Do this instead.
>
>SELECT ALLBILLS
>GO m.nPageOneStart
>DELETE REST WHILE RECNO()<=m.nBillEnd
>
>
>>WAIT WINDOW I Nowait
>>ENDFOR
>>
Harsh
Previous
Reply
Map
View

Click here to load this message in the networking platform