Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Creating table from CSV
Message
From
08/11/2016 07:40:41
Thomas Ganss (Online)
Main Trend
Frankfurt, Germany
 
 
To
07/11/2016 14:23:20
Dragan Nedeljkovich (Online)
Now officially retired
Zrenjanin, Serbia
General information
Forum:
Visual FoxPro
Category:
Coding, syntax & commands
Miscellaneous
Thread ID:
01642781
Message ID:
01642863
Views:
58
Here it is not daily, but a few each month.My first step is reading into a table, often as type delimited with those stupidly named further "named" parameters. Reasons for skipping Excel often similar to yours. With the table I can generate desceriptive stats and SQL min/max
values or len() with ease, which often point to problems. In vfp an object is only a scatter away from a record, but as most of the data from the input file will land in other tables for me staying at record level is ok - but the strongest reason for working at materialzed table level is backup of distinct processing steps - if any error occurs, it WILLL be costly and finger-pointing as well as steps to eliminate from later work is needed,
sometimes a correcting run, sometimes only a guard for next tables ;-)



>I'm fighting such sheets daily. My way is to use LibreOffice to import them (the csv, the xlsx - whatever) and export them as tab delimited. Then aLines() into an array, where the first element contains the column names.
>
>Process the column names. Split that first line with aLines(aFldN, ..., chr(9)) - name the empty ones fldNNN, trim initial non-chars, replace inadmissible chars (spaces, punctuation etc) with underscores, reduce multiple underscores with singles.
>
>Then go line by line and create an empty object per line (again aLines with tab as delimiter), with values assigned to properties of that object. They are all strings, of course.
>
>Then do something with that object.
>
>The point is that I never need to get a table from a sheet. I need to get the data from the sheet into my tables. So the processing, in my case, isn't creation of a table, it's using the values on the row object's properties to create records in my tables - which is different from case to case. And often getting the same file from the same source twice means rewriting some pieces of the code, because they have changed something - added a column, removed four columns, changed the date format in one column (but not others), switched from first and last name in separate columns into full name in single column, introduced separator rows between groups, extra header row etc etc.
>
>Why LibreOffice? Because Excel has problems - doesn't recognize tab delimited, doesn't even ask which format is a file, tries to be smart about mixed columns (I have a collection of stupid assumptions it makes), sometimes thinks that phone numbers, if not formatted, must be displayed in exponential notation... I've lost patience. Libre isn't perfect (I saw two 9-digit phone numbers, separated with a comma and a space, displayed as something n.nnnnnnnnnnnE+17) but it gives me far less headache. I just wish it had a macro recorder so I could automate the "save as csv using tab as a delimiter and don't offer to convert to .odt format, I know what I'm doing, thank you" part.
>
>And it is smart enough to solve most of the cases when commas were inside a text, so it doesn't convert those into tabs and I get a rather regular tab delimited file.
>
>And at least the memos get wrapped in quotation marks, so when they contain CRLFs, I can do this:
>
>
lcc=filetostr("\\vboxsvr\ku_2015\there\snafu.cnv")
>#define hQ chr(34)
>DO while occurs(hq, lcc)>1
>	lcQ=strextract(lcc, hq, hq,1,4)
>	lcq2=chrtran(lcq, 0h0d0a22, "||")
>	lcq2=strtran(lcq2, "||", [|])
>	lcc=strtran(lcc, lcq, lcq2)
>ENDDO
>
>strtofile(lcc, ""\\vboxsvr\ku_2015\there\snafixed.cnv"")
>
>Then when converting back, convert the pipe character back into a CR inside a memo and everything's fine.
Previous
Reply
Map
View

Click here to load this message in the networking platform