Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Quick way to find number of lines in 16GB file
Message
De
03/06/2020 17:40:44
 
 
Information générale
Forum:
Windows
Catégorie:
Informatique en général
Divers
Thread ID:
01674670
Message ID:
01674690
Vues:
48
Did you try to load directly ? If so, error or exponential slowdown ?

>>>Open with VFP using low-level functions FOPEN() and FREAD(). Then count the number of lines in a loop. Don't use buffering. FREAD will read upto each CR+LF.
>>
>>I'd read the file in some sizable chunks - perhaps 64K at a time - and keep counting the CRs in it.
>
>The number of lines I already got with the GitBash command (took a bit, but I know the file is around 80mln rows). I then split it and currently loading using SSIS. Loaded up to the middle of number 3 (it loads using character ordering, so it already loaded 1 and 11-19 and also 2 and 21-29)
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform