CiviCRM Community Forums (archive)

I just installed a new Drupal/CiviCRM 1.8 test site and I see the Administer CiviCRM > Global Settings > Upload Directories have changed from previous versions.

Now - 1.8
Temporary Files   ./templates_c/en_US/upload/
Images            ./templates_c/en_US/persist/contribute/
Custom Files      ./templates_c/en_US/custom/

Before - 1.7?
Temporary Files   /var/www/www.example.com/files/civicrm/upload/
Images            /var/www/www.example.com/files/civicrm/persist/contribute/
Custom Files      /var/www/www.example.com/files/civicrm/upload/custom/

I've been having lots of trouble with a CiviCRM 1.7 site (may have been 1.6 originally) (also problems with MyISAM tables instead of InnoDB)

When did these change? Which are correct for 1.8? Why did I see nothing about this in the upgrade docs? What would be affected? Importing?

The ./templates_c directories are what I found when I did a clean install of CiviCRM 1.8 with which I have successfully imported contacts and contibutions.

Last night I tried the same imports on our main website using the same cleanly installed CiviCRM database, unsuccessfully. Even a single simple contact wouldn't import. There were no error messages or log messages to alert me that anything was amiss and CiviCRM went through it's expected routine and reported x number of contacts successfully imported, except they weren't.

I found the import files still in files/civicrm/upload

Which made me wonder when I saw the different entries and it worked flawlessly on import. This is on a different server (one I built). RAM and disc are in short supply on the server where the main website is hosted.

Neil

Neil:

Can you try this on a fresh clean install of civicrm v1.8 and see if import works.

I just tried importing a simple first/last/email file in 1.8 (php5) and it worked fine

Also, can u try to reproduce this on demo

lobo

The first clean Drupal/CiviCRM install the defaults in Global Settings > Upload Directories were

Temporary Files   ./templates_c/en_US/upload/
Images            ./templates_c/en_US/persist/contribute/
Custom Files      ./templates_c/en_US/custom/

This worked fine.

The second clean install I saw this in the default Global Settings > Upload Directories

Temporary Files   ./upload/
Images            ./persist/contribute/
Custom Files      ./custom/

This didn't work at all so I put in the first set but that didn't work either, go figure, they worked the first time.

It worked when I replaced them with the familiar (from civicrm.settings.php files)

Temporary Files   /var/www/web.doc.root/files/civicrm/upload/
Images            /var/www/web.doc.root/files/civicrm/persist/contribute/
Custom Files      /var/www/web.doc.root/files/civicrm/upload/custom/

When I saw the first set of directories I thought they may have changed to accommodate the automated installer (which I haven't tried, I have installed the CfC standalone) or multi languages.

The variability is not reassuring!

I'm having problems importing contributions, I'm getting errors about duplicates which don't exist. The files (multiple copies) are all ending up in files/upload. It doesn't seem to clean up at all after a failed import. If I'm matching to contacts using cid should I disable match to contacts with email address? Any problem with a contribution import seems to render it impossible to import the file subsequently. I just nuke the whole mess (including Drupal) and start from scratch.

Contacts on the other hand import very fast and reliably.

Neil

can u check and ensure that your civicrm.settings.php has the following line:

ini_set('auto_detect_line_endings', '1');

Can u try to load the same file on demo (or a subset or randomize the contents) and see if it works. I suspect it will and most likely it is the line endings issue on your system

For the mysql log, did u restart the mysql server after u added the log line

lobo

I think I've found the problem, and have import pretty much working now...

It looks like there were some fields in my csv file that CiviCRM didn't like, even though it didn't report any errors. I went through the csv again carefully to make sure it was as clean as possible (BTW I assume commas within fields can cause demarcation problems - should this be highlighted in the docs?). Then I started again by trying to import just First Name, Last Name & Email from all records. That worked (don't know why my earlier, similar test failed), so I tried a larger subset, and that worked too. So I'm narrowing in on identifying the fields from our spreadsheet that cause problems for CiviCRM.

Looks like CiviCRM's error-checking isn't catching these conditions, and somehow is able to report success prematurely. Will try to see what it might be about our problem fields that it doesn't like.

Meanwhile I suggest that anyone doing an import check the source file with great care, and if they're having trouble try this narrowing procedure of starting with a stripped-down subset and, assuming that works,  gradually expanding the number of fields per record until the problem fields are found.

(BTW I checked php.ini & it was set to allow Mac line endings.)