Author Topic: rows not importing (Read 6619 times)

lesley · April 28, 2009, 11:43:37 pm

using drupal 6 & civicrm2.2.2

I'm going nuts trying to import individual contacts.

If my csv file has 150 contacts only 63 rows will import.

I reduced csv file to 40 contacts, removed any empty columns-
only first 15 rows will import. In case one row was causing a problem I deleted rows 13 to 18
and still only first 15 rows imported.
when I check the csv created in sites\default\files\civicrm\upload (exportfilename21_730816116f5427cd5ac5b1377bd88cc7.csv) it has all 40 rows
when I check the database table (civicrm_import_job_9bf4edfcd940669ce9dde3eca0e6e2dd) it has only 15 rows.

there are no individual contacts in the database (only orgs) and I am doing no duplicate checking.

I tried the 40 record csv in the demo site and only the first 15 rows imported - which I guess suggests some problem with particular rows in my csv file
BUT I can pretty much grab any rows to make a file of 40 and only the first 15 will import.
A file of 18 rows will import OK suggesting timeout issue? but the why would 63 out of 150 work OK?

sigh - any hints appreciated

Donald Lobo · April 29, 2009, 07:02:01 am

Since you have reproduced this issue on demo, can you please file an issue and attach the file to the issue along with steps to reproduce

thanx

lobo

bmodesign · May 04, 2009, 03:10:15 pm

I had a similar issue of not importing every row in a csv file. I was working in Excel, saving as a csv.

I found somewhere that if you open the csv in Open Office, and open it as Unicode UTF-8, and re-save it, every row uploaded fine.

So, I sorted and edited in Excel, because I'm used that that program, saved, re-opened in Open Office, Saved, then uploaded to CiviCRM and got every row.

Hope this helps.
Bmodesign

lesley · May 06, 2009, 12:32:13 am

bmodesign

THANKYOU THANKYOU THANKYOU

worked perfectly
I promise you my firstborn child! (it's ok I'm 50)

cheers
Lesley

bmodesign · May 07, 2009, 08:00:25 am

Oh wow! I didn't realize what I was getting myself into on this Forum!

Glad to hear it worked so well!

Good luck to you Lesley.
Bmodesign

Kiran Jagtap · May 08, 2009, 05:21:36 am

hi,

for tracking purpose : http://issues.civicrm.org/jira/browse/CRM-4429

kiran

kdas · May 08, 2009, 05:04:48 pm

Would like to thank you all. I wasted 3 days trying all the tricks as Import stops after certain records. This was not happening in 2.1.4.
Used the workaround mentioned above to open the csv in Open Office (free download) and saved and Import worked great!

benmoss · June 12, 2009, 02:19:06 pm

For the record, if you just save the file in Excel as a "UTF-16 Unicode Text (.txt)" and then save it back to a CSV and import, it seems to do the same magic. Saves having to install Open Office if you don't already have it.

Mt_Roxx · July 30, 2009, 05:38:11 pm

My client works in MS Office 2007, and saved it down to MS Office 2000 for me. I tried everything I could think of and then some to get it to upload more than 15 rows at a time. The method of saving as a UTF and back to CSV did not work for me.

The suggestion of downloading Open office, open and save as a csv file - and then importing that file worked!!!!!! THANK YOU OH THANK YOU THANK YOU AGAIN FOR SHARING YOUR FIX!!!!

Blake · August 11, 2009, 02:21:14 pm

This is incredible! If only I had realized MS Excel was full of crap 8 months ago! Yesterday I began to think Excel was saving bad csv files, and this confirms it. Now to delete MS Office from my computer all together!

Somehow 2.1 was less sensitive than 2.2, so I didn't initially think it was Excel.

I'm so happy, I think I'll go fishing or go to a baseball game!

arrow · September 15, 2009, 07:44:10 pm

>I found somewhere that if you open the csv in Open Office, and open it as Unicode UTF-8, and re-save it, every row uploaded fine.

This worked for me as well. Thank you, Bmodesign! What a relief.

cgnicholas · September 20, 2009, 12:38:34 pm

Hmm - unfortunately, I have the same situation, but not the same fix.. :-(

I get to step 3, i.e. things upload, it tells me how many records have problems, asks if I would like to import them (YES PLEASE...PRETTY PLEASE!!) and then...BAM! get bumped back to step one, with now message whatsoever.

I can see the import job as well in mysql, and it has all 488 rows.

I also wrote my own csv file parser, and created a separate database table, and tried the SQL importer; no luck.. :-(

Created: http://issues.civicrm.org/jira/browse/CRM-5086 with my file...

thanks for any and all help!
Chris Nicholas

Donald Lobo · September 20, 2009, 12:46:05 pm

Please reproduce this issue on http://sandbox.civicrm.org/ (if u r using 3.0). In general we like issues to be reproduced on the sandbox or the demo servers (for 2.3) before filing an issue

lobo

cgnicholas · September 20, 2009, 08:55:38 pm

sorry about that!

OK - tried it on the sandbox site; I checked "skip duplicate records", but still getting:

"Sorry. A non-recoverable error has occurred.

DB Error: already exists

Database Error Code: Duplicate entry 'Dept of State, Office of the Geo' for key 2, 1062

Return to home page."

I saved the mappings as 'ungiwg_test_mapping'

again, many thanks for building an awesome product...
Chris

Donald Lobo · September 20, 2009, 09:29:38 pm

1. if u select "Update", it should work

2. In general choosing Institutional Name for external identifier might be problematic in the long run. External ids are typically unique identifiers that names in general dont meet

lobo

CiviCRM Community Forums (archive)

News: