Level Extreme platform
Subscription
Corporate profile
Products & Services
Support
Legal
Français
Is CSV the best format to import data?
Message
 
 
To
25/08/2017 04:32:16
General information
Forum:
Visual FoxPro
Category:
Other
Miscellaneous
Thread ID:
01653669
Message ID:
01653746
Views:
53
>>>>Hi,
>>>>
>>>>I will be discussing (later this morning) with a company who is supposed to export the data from their SQL Server database with the goal that I will import it in my SQL Server database. Initially the agreement was that they will provide me with the CSV format files. My questions is, is this "the best" or "sufficient" that I can then import the data into my database? Or should I ask for another format? (the SQL Server backup is out of the question because their SQL Server database has their other customers' data).
>>>
>>>I prefer tab delimited, because of commas embedded in text fields, which don't necessarily get escaped and one can't be sure that all text fields are wrapped in quotes. Also, another nice feature is to have LF for line breaks inside the text fields (aka memos in foxspeak) but CR for record delimiters.
>>>
>>>This means I usually get these exports as excel sheets and then export them into this format. I even wrote a LibreOffice macro for that :).
>>
>>Thank you. I agree with you that the commas in the text field could be a problem. But the meeting already took place and they will send me the files in CSV format. I hope I won't have too many problems.
>
>If there are text field included you'd be well advised to get them to strip any commas in fields. Much easier done when exporting than when trying to import .
>But as you'e already agreed you could import first into into a temporary table where each row is one record and do a quick check on the number of commas in each row. that way you could import all the rows without extra commas and handle the problem rows (if any) in a separate process.
>Apologies if you've already thought of things like that.

Yes, there are fields that have text in them. And stripping the commas would change the content. And that would not go well. I think I should have chosen a different format than CSV; the best would be, as Fabio suggestion, a dummy SQL Server.
I will talk today with the project manage to see if they are willing to revise the spec. I am not very optimistic that they would. In this case, I will have a nightmare on my hands. Not that this will be the first time :)
Thank you.
"The creative process is nothing but a series of crises." Isaac Bashevis Singer
"My experience is that as soon as people are old enough to know better, they don't know anything at all." Oscar Wilde
"If a nation values anything more than freedom, it will lose its freedom; and the irony of it is that if it is comfort or money that it values more, it will lose that too." W.Somerset Maugham
Previous
Reply
Map
View

Click here to load this message in the networking platform