Plateforme Level Extreme
Abonnement
Profil corporatif
Produits & Services
Support
Légal
English
Quick way to find number of lines in 16GB file
Message
 
 
À
03/06/2020 03:38:37
Dragan Nedeljkovich (En ligne)
Now officially retired
Zrenjanin, Serbia
Information générale
Forum:
Windows
Catégorie:
Informatique en général
Divers
Thread ID:
01674670
Message ID:
01674687
Vues:
48
>>Open with VFP using low-level functions FOPEN() and FREAD(). Then count the number of lines in a loop. Don't use buffering. FREAD will read upto each CR+LF.
>
>I'd read the file in some sizable chunks - perhaps 64K at a time - and keep counting the CRs in it.

The number of lines I already got with the GitBash command (took a bit, but I know the file is around 80mln rows). I then split it and currently loading using SSIS. Loaded up to the middle of number 3 (it loads using character ordering, so it already loaded 1 and 11-19 and also 2 and 21-29)
If it's not broken, fix it until it is.


My Blog
Précédent
Suivant
Répondre
Fil
Voir

Click here to load this message in the networking platform