Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
How to read a flat file LINE by LINE ?
Message
From
20/04/2002 10:16:06
Cetin Basoz
Engineerica Inc.
Izmir, Turkey
 
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
00647169
Message ID:
00647201
Views:
11
>I understand how to read a complete text file in a variable but I need to read it line by line. I am playing around with FGETS but without success :-(
>
>thanks for any hint
>Eric

Eric,
With VFP6 if file is lte 16Mb and linecount is lte 65000 then :

lnLines = alines(arrLines, StrToFile(cFileName))
for ix=1 to lnLines
* ...
endfor

is the usual and faster way.
Otherwise what's the problem with fgets() ? if you encountered an error (fgets() and fread() has a bug that chars they could read at once are limited. In VFP6 up to SP3 fgets() can't read beyond 256 and after SP3 8192). FRead() limit is higher and AFAIK 64K. I use my own SafefGets() :
#Define QUOTES '%%%'
#Define CRLFSYMBOL '~'
#Define READMAX 65535

lcFile = 'sampler.txt'
If .T. && Create test file
  lnHandle=fcreate(lcFile)
  For ix = 1 to 10
    =fwrite(lnHandle, trans(ix)+','+QUOTES+'Memo started.'+CRLFSYMBOL+CRLFSYMBOL+;
      replicate(replicate( chr(ix-1+asc('A')), 100 )+CRLFSYMBOL,655)+;
      CRLFSYMBOL+'Memo ended.'+QUOTES+CHR(13)+CHR(10) )
  Endfor
  =fclose(lnHandle)
Endif
lnSTart=seconds()
Create cursor myCursor (RecId i, myMemo m)
? ProcessFile(lcFile,'myLineProcessor',''), seconds()-lnSTart
Select myCursor
Browse

Function ProcessFile
Lparameters tcFile, tcProcessorFuncName, tcParamList
Local lnHandle, lnFileSize, lcTempFile, lnLinesToProcess, lnLinesRead
lcTempFile  = sys(2015)+'.tmp'
lnLinesToProcess = UnifyCRLF(tcFile,lcTempFile) && Make sure only chr(10) is CRLF
lnHandle = fopen(lcTempFile)
If lnHandle < 0 && This should never happen - unless current dir was readonly on purpose
  Return ferror()
Endif
lnLinesRead = 0
* Progress Bar setup if any here - lnTotLines is the number of lines to read
* If last record had no NL than would be one less than actual
Do while !feof(lnHandle)
  * Read and Process line
  Do (tcProcessorFuncName) with (SafeFgets(lnHandle)) &tcParamList
  lnLinesRead = lnLinesRead + 1 && Progress Bar update if any here
  Wait window nowait 'Processed '+padr(lnLinesRead,5)-'/'-padr(lnLinesToProcess,5)
Enddo
=fclose(lnHandle)
Erase (lcTempFile)
Return lnLinesRead

Function UnifyCRLF
Lparameters tcOrgFile, tcTempFile
Local hin,hout, lnTotLines, lcRead
hin = fopen(tcOrgFile)
hout = fcreate(tcTempFile)
lnTotLines = 0
Do while !feof(hin)
  lcRead = chrtran(fread(hin,READMAX),chr(13)+chr(10),chr(10))
  lnTotLines = lnTotLines + occurs(chr(10),lcRead)
  Fwrite(hout,lcRead,READMAX)
Enddo
Fclose(hin)
Fclose(hout)
Return lnTotLines

Function SafeFgets
Lparameters tnHandle
Local lcRetLine, lnCurPos
lcRetLine = ''
lnCurPos = fseek(tnHandle,0,1)    && Save current pos
Do while !feof(tnHandle) and at(chr(10), lcRetLine) = 0 && Till eof() or NL encountered
  lcRetLine = lcRetLine + fread(tnHandle,READMAX) && Read in safe READMAX bytes blocks
Enddo
If at(chr(10), lcRetLine) > 0
  lcRetLine = Left(lcRetLine, at(chr(10), lcRetLine)-1)
  =fseek(tnHandle,lnCurPos+len(lcRetLine)+1,0) && Set position to just after NL
Endif
Return lcRetLine

Function myLineProcessor && Line parser specific to target file
Lparameters tcLine
* For sampling only, memos could be over 64K that's not considered here
Local lnRecId, lcMemoText, lnMemoStart, lnMemoEnd
lnRecId = val(tcLine)
lnMemoStart = at(QUOTES,tcLine,1)+3
lnMemoEnd   = at(QUOTES,tcLine,2)
lcMemoText = strtran(substr(tcLine, lnMemoStart, lnMemoEnd-lnMemoStart),CRLFSYMBOL,chr(13)+chr(10))
Insert into myCursor (RecId,myMemo) values (lnRecId,lcMemoText)
Alines() is faster because despite sample presented here it uses C behind the scenes I believe (I could bet:). Actually the version I use also utilizes a FLL feeding the lines to process :)
Cetin
Çetin Basöz

The way to Go
Flutter - For mobile, web and desktop.
World's most advanced open source relational database.
.Net for foxheads - Blog (main)
FoxSharp - Blog (mirror)
Welcome to FoxyClasses

LinqPad - C#,VB,F#,SQL,eSQL ... scratchpad
Previous
Reply
Map
View

Click here to load this message in the networking platform