>So I ended up setting up some tests here locally comparing SUBSTR() to some .NET based equivalents. And the performance difference even there is astounding on largish strings.
>
>SUBSTR() results are not pretty:
>
>

>CLEAR
>
>* 1mb string
>LOCAL lnX, lcString
>lcString = REPLICATE("1234567890",100000)
>
>lnLength = LEN(lcString)
>? "Iterations: " + TRANSFORM(lnLength,"9,999,999")
>
>IF .T.
>lnSecs = SECONDS()
>
>FOR lnX = 1 TO lnLength
>   lcVal = SUBSTR(lcString,lnX,1)
>ENDFOR
>
>*** 35 seconds
>? "SUBSTR: " +  TRANSFORM(SECONDS() - lnSecs )
>
>ENDIF
>
>IF .T.
>do wwDotNetBridge
>LOCAL loBridge as wwDotNetBridge
>loBridge = GetwwDotnetBridge()
>
>lnSecs = SECONDS()
>
>* Returns a COM Array object
>loStringArray = loBridge.InvokeMethod(lcString,"ToCharArray")
>lnLength = loStringArray.Count
>FOR lnX = 0 TO lnLength -1
>	lcVal = CHR(loStringArray.Item(lnX))
>	*lcVal = loBridge.GetIndexedproperty(loStringArray.Instance,lnX)
>ENDFOR
>
>* ~ 2.5-2.9 seconds (still not very quick really) - in .NET same iteration takes 0.03 seconds
>? "wwDotnetBridge ToCharArray(): " + TRANSFORM( SECONDS() - lnSecs ) 
>
>ENDIF
>
>
>IF .T.
>
>lnSecs = SECONDS()
>
>STRTOFILE(lcString,"c:\temp\test.txt")
>lnHandle = FOPEN("c:\temp\test.txt",0)
>
>DO WHILE !FEOF(lnHandle)
>  lcVal = FREAD(lnHandle,1)
>ENDDO
>FCLOSE(lnHandle)
>ERASE("c:\temp\test.txt")
>
>* 3.6 seconds 
>? "FREAD : " + TRANSFORM( SECONDS() - lnSecs ) 
>
>ENDIF
>

>
>This sample runs with a 1mb string:
>
>* SUBSTR: 35 seconds
>* ToCharArray (.NET): 2.5 secs
>* FREAD(): 3.6 seconds
>
>Unfortunately it's not as simple as all that though - smaller operations are considerably faster with SUBSTR() so there are cut off points where performance with one or the other is better than the other. Somewhere ~100k SUBSTR() starts being really slow.
>
>Still - it's good to know if large byte by byte parsing is needed there are alternatives.
>
>I think using a string buffer in C code probably would be faster than the .NET solution as it would avoid the COM interop calls. Be interesting to see how that pans out.
>
>Wrote up a blog post here:
>
>Workaround for horrible SUBSTR() performance on large FoxPro Strings
>
>+++ Rick ---
>
>>>>Faststring.dll won't store its strings in VFP. You'll have to use FTRANSFORM(), FLEFT(), FRIGHT(), or FSUB() to return a string usable in VFP.
>>>
>>>I get that, but unfortunately I don't think that helps when retrieving individual values out because API calls (or FLL calls) have their own overhead in VFP. Have you done any performance comparison walking through a string with SUBSTR() compared to FSUBSTR()?
>>...
>>>Curious to see if what you've built here might address that although I'm not holding my breath due to the overhead of the API calls that may end up killing any native performance gains. That was my experience at least with the DECLARE DLL interface - maybe FLL is better there.
>>...
>>>Anyway - I'd be interested to see some benchmark comparisons between the native FoxPro functions and your replacements although I think Fox is pretty damn good with almost everything except SUBSTR(). I can't think of other scenarios where I've needed something to provide faster string support.
>>
>>Mostly ACK. Lack of Array-like direct access to string elements worst aspect of vfp string handling.
>>Penultimate for me the lack of StringBu???er class/functions, forcing new memory assignment - most of the old dogs code long string results built from huge number of concats as pipe into LLF or array of smaller srings to build first, which then are concatenated at the end with few line calls each handling many array elements.
>>
>>One edge case comes up for me: if you work with an array of tinyints and already have many fields, using a string as storage enables you to exceed field count limits on a record and avoiding another record just for such small helper fields. If you need 2 byte integers or larger, perf difference not as maked, but the runtime transform of asc/chr helps a lot.
>>I can imagine working on such a category group array via C faster, esp. if the string can stay locked in the C memory space for less work. Still something to both code AND call respecting C memory gotchas, as any error checking/trapping will beat the intent of max perf accomplished. As parameter passing is also a definite slow operation in vfp/fll, doing much work on a string set as "default" in C memory space might give a measurable edge.
>>
>>Would be a messy API (simulating OOP setup and teardown via functions only) needing lots of docs and dire warnings
>>
>>my 0.0022 (been a long time I needed to visit that machine room)
>>thomas

Hello Rick.. the solution seems simple.. just chunk it:
At least for your reading scenario this makes the trick ;
- currently working on a core i3 laptop takes 1.189 secs.

* 1mb string
Local lnX, lcString
lcString = Replicate("1234567890",100000)

lnLength = Len(lcString)
? "Iterations: " + Transform(lnLength,"9,999,999")

	lnSecs = Seconds()

FOR n = 0 TO 999
	j = SUBSTR(m.lcString,(n*1000)+1,1000)
	For lnX = 1 To 1000
		lcVal = Substr(m.j,lnX,1)
	Endfor
endfor

	? "SUBSTR: " +  Transform(Seconds() - lnSecs )