Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Speeding up string conversion
Message
De
16/03/2005 09:23:51
 
 
À
16/03/2005 09:06:28
Information générale
Forum:
Visual FoxPro
Catégorie:
Codage, syntaxe et commandes
Divers
Thread ID:
00996197
Message ID:
00996308
Vues:
18
>>>>I need to convert binary strings to ascii by changing nonprintable non-ascii
>>>>charaters to a \\nnn strings where nnn is octal representation.
>>>>
>>>>I created the following routine. For 381 KB byte strings, the conversion takes 5 (!) minutes and task manager shows that my application takes a lot of MBs memory.
>>>>
>>>>Any way to speed up this conversion ?
>>>>
>>>>
FUNCTION toBYTEA( cStr )
>>>>
>>>>LOCAL i, j, cRes, nChr, cDig
>>>>
>>>>cRes = ''
>>>>FOR i=1 TO LEN(m.cStr)
>>>>  nChr = ASC( SUBSTR( m.cStr, m.i,1))
>>>>  * The following four lines cna be removed if this can speed the conversion.
>>>>  IF BETWEEN( nChr, 32,126 ) AND !INLIST( nChr,39,92)
>>>>    cRes = m.cRes + SUBSTR( m.cStr, m.i,1)
>>>>    LOOP
>>>>    ENDIF
>>>>
>>>>  cDig = ''
>>>>  FOR j=1 TO 3
>>>>    cDig= CHR(ASC('0')+ m.nChr%8) + m.cDig
>>>>    nChr = INT(m.nChr/8)
>>>>    ENDFOR
>>>>  cRes = m.cRes + '\\'+ m.cDig
>>>>  ENDFOR
>>>>RETURN m.cRes
>>>>ENDFUNC
>>>
>>>This thread is a interesting point for publishing the only solution
>>>that can reduce the VFP long string processing, without write a C code.
>>>The solution is not immediate, because it is RECURSION.
>>>The problem of VFP with a strings and native command is that into
>>>a native command is not possible pass a var by reference ( a mem pointer ),
>>>then VFP copy the string into a engine buffer.
>>>With Recursion you can create of the virtual pointer, breaking the string in substring,
>>>this have another important effect, VFP move and work on short string, then the
>>>job is done in first level cache
>>>( the only area of memory where the cpu efficiency works is sufficient ).
>>>
>>>On next code, STRCONV() return a the expected time for a C implementation
>>>Then:
>>>
>>>#DEFINE MAXLEN 2048
>>>
>>>X=''
>>>FOR K=0 TO 255
>>> X=X+CHR(K)
>>>NEXT
>>>ACTIVATE WINDOW "Debug Output"
>>>DEBUGOUT toBYTEA(REPLICATE(m.x,100))== toBYTEA3(REPLICATE(m.x,100))
>>>FOR j=2 TO MAXLEN STEP 0
>>>	T0=SECONDS()
>>>	=toBYTEA(REPLICATE(m.x,m.j))
>>>	t1=SECONDS()-T0
>>>	T0=SECONDS()
>>>	=toBYTEA2(REPLICATE(m.x,m.j))
>>>	t2=SECONDS()-T0
>>>	T0=SECONDS()
>>>	=toBYTEA3(REPLICATE(m.x,m.j))
>>>	t3=SECONDS()-T0
>>>	T0=SECONDS()
>>>	=STRCONV(REPLICATE(m.x,m.j),15)
>>>	t4=SECONDS()-T0
>>>	DEBUGOUT m.j,m.t1,m.t2,m.t3,M.T4
>>>	j=j*2
>>>NEXT
>>>
>>>FUNCTION toBYTEA( cStr )
>>>
>>>	PRIVATE i,  cRes, cChr, nChr
>>>
>>>	cRes = ''
>>>	FOR i=1 TO LEN(m.cStr)
>>>		STORE SUBSTR( m.cStr, m.i,1)	TO cChr
>>>		STORE ASC( m.cChr)				TO nChr
>>>		cRes = m.cRes + IIF(BETWEEN( m.nChr, 32,126 ) AND !INLIST( m.nChr,39,92);
>>>	  			,	m.cChr	;
>>>	  			,	'\\'+ CHR(48+ BITAND(m.nChr/64,7))+CHR(48+ BITAND(m.nChr/8,7))+CHR(48+ BITAND(m.nChr,7)))
>>>	ENDFOR
>>>	RETURN m.cRes
>>>ENDFUNC
>>>
>>>FUNCTION toBYTEA2( cStr )
>>>
>>>	STORE STRTRAN(STRTRAN(m.cStr,[\],[\\134]),['],[\\047]) TO cStr
>>>	PRIVATE nChr
>>>	FOR nChr=0 TO 255
>>>		IF m.nChr=32
>>>			STORE 126 TO nChr
>>>			LOOP
>>>		ENDIF
>>>		STORE STRTRAN(m.cStr;
>>>				,	CHR(m.nChr);
>>>				,	'\\'+ CHR(48+ BITAND(m.nChr/64,7))+CHR(48+ BITAND(m.nChr/8,7))+CHR(48+ BITAND(m.nChr,7)));
>>>			TO cStr
>>>	NEXT
>>>	RETURN m.cStr
>>>ENDFUNC
>>>
>>>FUNCTION toBYTEA3( cStr,lenght )
>>>	IF PARAMETERS()=1
>>>		* this is a slow op
>>>		STORE LEN(m.cStr) TO lenght
>>>	ENDIF
>>>	IF m.lenght>100
>>>		* this is a slow op
>>>		RETURN	toBYTEA3(LEFT( m.cStr ,BITRSHIFT(m.lenght+1,1)) ,BITRSHIFT(m.lenght+1,1))	;
>>>			+	toBYTEA3(RIGHT(m.cStr ,BITRSHIFT(m.lenght,1))   ,BITRSHIFT(m.lenght  ,1))
>>>	ENDIF
>>>	PRIVATE cRes, cChr
>>>	STORE [] TO cRes
>>>	FOR lenght=1 TO m.lenght
>>>		* this is a slow op
>>>		STORE SUBSTR( m.cStr, m.lenght,1)	TO cChr
>>>		* this is a fast op
>>>		cRes = m.cRes + IIF(BETWEEN( ASC( m.cChr), 32,126 ) AND !INLIST( ASC( m.cChr),39,92);
>>>	  			,	m.cChr	;
>>>	  			,	'\\'+ CHR(48+ BITRSHIFT(ASC(m.cChr),6))+CHR(48+ BITAND(BITRSHIFT(ASC(m.cChr),3),7))+CHR(48+ BITAND(ASC(m.cChr),7)))
>>>	ENDFOR
>>>	* this is a slow op
>>>	RETURN m.cRes
>>>ENDFUNC
>>>
>>>
>>>Fabio
>>
>>hi Fabio,
>>
>>A1 = dog slow
>>A3 = recursion, faster
>>A2 = the fastest ( it resembles the code I have posted)
>
>Mistery.
>My results:
>
>.T.
>         2          0,000          0,010          0,000          0,000
>         4          0,010          0,010          0,000          0,000
>         8          0,010          0,030          0,010          0,000
>        16          0,020          0,060          0,020          0,000
>        32          0,051          0,120          0,030          0,000
>        64          0,170          0,240          0,080          0,000
>       128          0,601          0,491          0,160          0,000
>       256          1,923          1,072          0,320          0,000
>       512          7,271          2,583          0,591          0,000
>      1024         22,112         16,484          1,171          0,000
>
Maybe the processor cache ?? or vfp version. I'm using vfp6sp5
debugout trans(m.j) + ' ' +transf(m.t1) + ' ' +transf(m.t2) +' ' + transf(m.t3) + ' ' +transf(M.T4)


2 0.040000000000873 0.020000000004075 0.029999999998836 0
4 0.089999999996508 0.02100000000064 0.069999999999709 0
8 0.180000000000291 0.040000000000873 0.129999999997381 0
16 0.351000000002387 0.069999999999709 0.279999999998836 0
32 0.710999999995693 0.120000000002619 0.550999999999476 0
64 1.472000000001572 0.241000000001804 1.100999999995111 0
128 2.993999999998778 0.480999999999767 2.192999999999302 0
256 6.889999999999418 0.972000000001572 4.376000000003842 0
512 18.29599999999482 2.364000000001397 8.762000000002445 0
Anyway, one of the suggestions I made this morning is faster, see latest replies
Gregory
Précédent
Répondre
Fil
Voir

Click here to load this message in the networking platform