CiviCRM Community Forums (archive)

In one of our instances of the database, when we do an import by uploading a .csv file the import will say there are only 15 records to import, even where there are many more.  I've experimented a great deal and can't figure out what the problem might be:

- the problem happens regardless of what data fields are being used (as best I can tell)
- the problem happens regardless of whether or not I use a saved import template or select the matching fields each time.

Does anyone have any ideas of things I might do to try to diagnose the problem?

Thanks,
Colleen

I've usually had a weird character in my data when this happens [e.g. someone copy/pasted an email into a memo field], i.e. it's likely the php csv parser that can't figure out where the line ends. Most of the time, you can just go in an edit that field [you may not even see the bad characters, but sometimes you can guess, e.g. things like em-dashes or other non-ascii characters].

I had this problem too. The original file had about 120 records and on the first import it imported 70 of them but these were in 2 alphabetically sequential groups.

So I deleted those that were already imported from the csv file. That left 49.

2nd import it imported the first 15 only.

I deleted those from the file. 3rd import imported only the first 15 of these.

I deleted those from the file. 4th import imported only the first 15 of these.

That left only 4 which were imported on the 5th import.

I'm sure glad I didn't have a larger file.

The number 15 does seem to be significant.

I've had the same problem when importing email lists from another email program into civi

Here are a few notes on what I found helpful (maybe they are obvious to everyone else!)
- Some of the email addresses weren't valid. Maybe they were missing a "@" symbol, had a "," in it, etc.... I don't know why my old email program didn't catch those!
- Sometimes when you are dealing with bulk records you may have blank fields. Try to fix those, remove them or put those records at the bottom of your csv file
- Civi will show you what errors it finds in your import file. Remedy those and try again. The rest of your valid fields should show up to import

I am interested if anyone has had the same problems or found other potential fixes

did u try importing your data on the demo server? if not, please do so and let us know how that goes

if confidentital data, scramble the data before importing

lobo

I'm seeing the same issue. Version 3.1.4
Importing from an Excel generated CSV file. First try it imported 75 records form the 650 in the file, seemingly at random. All subsequent attempts it only wants to import 15 records.
This is a major pain. what should be a matter of a few minutes work has so far taken me over 2 hours, with probably another 2 until - hopefully - I have my data imported.

Update!. Having just read the thread at http://forum.civicrm.org/index.php/topic,13168.0.html I gave it a go - i.e. added a column of zeros at the end of my data, and it works! The moral of the tale - don't trust Excel to generate anything like a clean well structured CSV file.

In the cold light of day it might be worthwhile putting something about this issue into the online documentation and FLOSS manual - and if possible offering a foolproof method for creating a clean CSV file for import.

hi

I also had this problem. As mentioned above: filemaker is the solution
i exported my access database as excelfile, loaded this into filemaker (trial), exportet as csv and civicrm got 100 per cent of the data

cheers
conte

Just a note any one viewing this thread, if you "ARE" using Excel to create CSV Files, when you do the save as, make sure you select CSV (DOS).

After Export to CSV, I always open with notepad, then re-save, then import.

D