I am using Django import-export to import a csv file. It works great if I open the csv in TextEdit and save it but if I try to import a csv file without saving it in TextEdit first it doesn’t read it correctly. Any ideas on what would cause this?
Thanks!
My initial guess would be that there are some types of control characters or special characters in the file that TextEdit is filtering out or converting for you.
Is there any indication of an error when you try to import it? (Check your Django logs as well as your console log if you’re running this in runserver.
What is happening that you’re describing as “doesn’t read it correctly”?
What is creating the file that you’re trying to use? (It could be the source is causing the problems.)
The only indication of an error is that it doesn’t import the first column. And instead of importing all rows it only imports the first row with the first column left blank.
The csv doesnt really contain any odd characters. a row would look like:
B000E7SU3I,“Toro 29210 43-Gallon Gardening Spring Bucket”,“Patio, Lawn & Garden”,Toro,Amazon,“Small Oversize”,2,0,6.84976248034,45205,29.99,100,13.99,2999,763,4.4,15,1135,-5,-79,“Nov, 2020”,0.13
I am running this locally and in production. Locally i get no errors in the console log and i do not have django logging.
“Only importing the first row” - whenever I see that with reference to a file-import issue, I immediately think of “newline” problems.
I would examine the original files with something that will let you look at the hex data within the file to see what you’re dealing with among non-visible characters.
I’d also check your TextEdit settings to see if it’s set up to automatically convert line-ending characters from one format to another.
Also, I’d check to see if whatever-it-is that is creating these files specifies what they’re using for newline characters - all of this with an eye toward adjusting your import process to accept what you’re receiving.
(If you’re not aware of this, be aware that by default, each of Windows, Linux, and Mac use different characters / character sequences to mark the end of a line of text. Moving files from one environment to another frequently involves converting these markers.)
I was thinking something along the same lines but couldn’t narrow it down. The csv is being generated by Helium 10 and saved to a Windows-based computer, then forwarded to me (on a Mac) to troubleshoot the issue. I may have fixed it last night, but who knows…
And I was not aware that each system uses different characters - thank you for the heads up and direction. I taught Paramedicine up until about a year ago and this is all still pretty new to me.
Thanks again, Ken!