Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Quick way to find number of lines in 16GB file
Message
 
 
To
03/06/2020 03:38:37
Dragan Nedeljkovich (Online)
Now officially retired
Zrenjanin, Serbia
General information
Forum:
Windows
Category:
Computing in general
Miscellaneous
Thread ID:
01674670
Message ID:
01674687
Views:
46
>>Open with VFP using low-level functions FOPEN() and FREAD(). Then count the number of lines in a loop. Don't use buffering. FREAD will read upto each CR+LF.
>
>I'd read the file in some sizable chunks - perhaps 64K at a time - and keep counting the CRs in it.

The number of lines I already got with the GitBash command (took a bit, but I know the file is around 80mln rows). I then split it and currently loading using SSIS. Loaded up to the middle of number 3 (it loads using character ordering, so it already loaded 1 and 11-19 and also 2 and 21-29)
If it's not broken, fix it until it is.


My Blog
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform