Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Creating table from CSV
Message
From
07/11/2016 14:23:20
Dragan Nedeljkovich (Online)
Now officially retired
Zrenjanin, Serbia
 
 
To
06/11/2016 07:44:56
Thomas Ganss (Online)
Main Trend
Frankfurt, Germany
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
01642781
Message ID:
01642834
Views:
60
>Speaking of Memos: the base utility class responsible for importing has the option to create an intermediate cursor where all text fields are memos, so that a check can be made that new import does not contain text fields needing more chars than was previously deemed necessary.
>
>Esp. importing from gov data such field len sometimes were changed and gov programmers will offer such info in their internal dokumentation, available via mail as soon as anyone asks again after change ;-)

I'm fighting such sheets daily. My way is to use LibreOffice to import them (the csv, the xlsx - whatever) and export them as tab delimited. Then aLines() into an array, where the first element contains the column names.

Process the column names. Split that first line with aLines(aFldN, ..., chr(9)) - name the empty ones fldNNN, trim initial non-chars, replace inadmissible chars (spaces, punctuation etc) with underscores, reduce multiple underscores with singles.

Then go line by line and create an empty object per line (again aLines with tab as delimiter), with values assigned to properties of that object. They are all strings, of course.

Then do something with that object.

The point is that I never need to get a table from a sheet. I need to get the data from the sheet into my tables. So the processing, in my case, isn't creation of a table, it's using the values on the row object's properties to create records in my tables - which is different from case to case. And often getting the same file from the same source twice means rewriting some pieces of the code, because they have changed something - added a column, removed four columns, changed the date format in one column (but not others), switched from first and last name in separate columns into full name in single column, introduced separator rows between groups, extra header row etc etc.

Why LibreOffice? Because Excel has problems - doesn't recognize tab delimited, doesn't even ask which format is a file, tries to be smart about mixed columns (I have a collection of stupid assumptions it makes), sometimes thinks that phone numbers, if not formatted, must be displayed in exponential notation... I've lost patience. Libre isn't perfect (I saw two 9-digit phone numbers, separated with a comma and a space, displayed as something n.nnnnnnnnnnnE+17) but it gives me far less headache. I just wish it had a macro recorder so I could automate the "save as csv using tab as a delimiter and don't offer to convert to .odt format, I know what I'm doing, thank you" part.

And it is smart enough to solve most of the cases when commas were inside a text, so it doesn't convert those into tabs and I get a rather regular tab delimited file.

And at least the memos get wrapped in quotation marks, so when they contain CRLFs, I can do this:
lcc=filetostr("\\vboxsvr\ku_2015\there\snafu.cnv")
#define hQ chr(34)
DO while occurs(hq, lcc)>1
	lcQ=strextract(lcc, hq, hq,1,4)
	lcq2=chrtran(lcq, 0h0d0a22, "||")
	lcq2=strtran(lcq2, "||", [|])
	lcc=strtran(lcc, lcq, lcq2)
ENDDO

strtofile(lcc, ""\\vboxsvr\ku_2015\there\snafixed.cnv"")
Then when converting back, convert the pipe character back into a CR inside a memo and everything's fine.

back to same old

the first online autobiography, unfinished by design
What, me reckless? I'm full of recks!
Balkans, eh? Count them.
Previous
Next
Reply
Map
View

Click here to load this message in the networking platform