CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site
This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Import (Moderator: Yashodha Chaku) »
  • Memory Limit on Relationship Imports
Pages: [1]

Author Topic: Memory Limit on Relationship Imports  (Read 653 times)

PrincessAng417

  • I post occasionally
  • **
  • Posts: 50
  • Karma: 2
  • CiviCRM version: 4.5.4
  • CMS version: Drupal 7.34
Memory Limit on Relationship Imports
August 13, 2014, 07:03:52 am
I have been successfully importing Civi Relationships for several weeks on a D7.28/Civi 4.4.5 installation.

As of last week, I am hitting memory issues when trying to import.

My CSV files have only two columns: the external ID of the first contact, and the Civi ID of the second contact.  I have been successfully importing these relationships, maybe 100-150 records or so, at a time for weeks.  Now, I can't even get a CSV file with two records in.

I am hitting: "Fatal error: Out of memory (allocated 121634816) (tried to allocate 48 bytes) in xxx/sites/all/modules/civicrm/packages/DB/mysql.php on line 383"

Or, sometimes, I just white screen.

My hosting company increased the memory available on our VPS account; no success: at one point, we were getting a 500 Server error instead, but still no success.  They believe the problem is not a resources issues, but instead that the import script is entering a loop.

I cannot figure out why we had been successful (for weeks) performing these imports, and all of a sudden we are at a stop.  The timeout errors occur within the first few seconds.  Is there some sort of cache or reserved memory block or something that is “clogged”?

JonGold

  • Ask me questions
  • ****
  • Posts: 638
  • Karma: 81
    • Palante Technology
  • CiviCRM version: 4.1 to the latest
  • CMS version: Drupal 6-7, Wordpress 4.0+
  • PHP version: PHP 5.3-5.5
Re: Memory Limit on Relationship Imports
August 13, 2014, 07:48:28 am
I wonder if there's something weird in the data set.  Does it matter which two records you import?
Sign up to StackExchange and get free expert CiviCRM advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

JonGold

  • Ask me questions
  • ****
  • Posts: 638
  • Karma: 81
    • Palante Technology
  • CiviCRM version: 4.1 to the latest
  • CMS version: Drupal 6-7, Wordpress 4.0+
  • PHP version: PHP 5.3-5.5
Re: Memory Limit on Relationship Imports
August 13, 2014, 07:49:35 am
Also, have you turned on debugging and backtrace in Administer menu > System Settings > Debugging and Error Handling?  If so, does it yield additional information?  If you have a backtrace, please paste it here.
Sign up to StackExchange and get free expert CiviCRM advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

PrincessAng417

  • I post occasionally
  • **
  • Posts: 50
  • Karma: 2
  • CiviCRM version: 4.5.4
  • CMS version: Drupal 7.34
Re: Memory Limit on Relationship Imports
August 13, 2014, 09:22:14 am
Nothing odd in the data set... I had even gone in and deleted an existing relationship and tried to re-import what originally went through successfully.  It did not go in the second time.

Having said that...

I have now tried to turn on the Debugging and Error Handling... and the import went through??  I turned off the debug settings, and I got a second file of 25 records to go through.

However, it is taking several minutes, and I am not getting any feedback of how far along the import is-- it just flips to the successful import screen.

To add mystery to the whole thing (but may be a clue in what is going on here), the error file that is being generated each time is kicking back with "Record imported successfully but unable to parse the street address: ..."  The street address of the 2nd contact has a PO Box address which is being referenced in the error file.  The 2nd contact, a servicing organization, is the same contact for each record that is being imported.

Why would it be trying to parse the street address of an already-existing contact, when all I am trying to import is the relationship between the two already-existing contacts, by ID numbers?  Again, the file only has two columns.  I do have the parser set to "update", but there are no address fields in the mapping.

And why on earth did it work after turning on the debugging?

PrincessAng417

  • I post occasionally
  • **
  • Posts: 50
  • Karma: 2
  • CiviCRM version: 4.5.4
  • CMS version: Drupal 7.34
Re: Memory Limit on Relationship Imports
August 14, 2014, 01:13:29 pm
Well, it's getting stranger.

I can now import 13 (not 12, not 14, but 13) relationships in at a time.  I am assigning the relationship to a Household contact.  Two columns in the CSV... one is the Civi ID of the target contact (Household), and the second column is the Civi ID of the servicing contact (Organization).  The Household, Organization, and all of the Individuals within a Household are all existing contacts.

I am still getting address parsing issues on the error file.

However, one Individual contact in each Household is *being changed into a Household contact*.  For sure, I thought this had to be an error on import... but I checked and we are absolutely selecting "Household" and "Update" on the import settings.

This is, again, something we have been doing for weeks with no issue.  I thought maybe there was a cache issue... cleared all caches.  I went into the database, and sure enough... these contacts that were Individuals, who were not referenced at ALL on the import CSV, are now categorized as Households. 

What is happening??

PrincessAng417

  • I post occasionally
  • **
  • Posts: 50
  • Karma: 2
  • CiviCRM version: 4.5.4
  • CMS version: Drupal 7.34
Re: Memory Limit on Relationship Imports
August 14, 2014, 01:16:01 pm
And just another piece of information... the Households and their respective Individuals are not even close to consecutive in the database.  Their Civi IDs are thousands of records apart.

JonGold

  • Ask me questions
  • ****
  • Posts: 638
  • Karma: 81
    • Palante Technology
  • CiviCRM version: 4.1 to the latest
  • CMS version: Drupal 6-7, Wordpress 4.0+
  • PHP version: PHP 5.3-5.5
Re: Memory Limit on Relationship Imports
August 14, 2014, 01:29:29 pm
Ah - now I think we're on to something.

The Individual getting changed to a Household on import is a known (fixed) bug.  I can't help but wonder if the rest of what you're describing is related to that bug.  I just looked at your profile, and it says you're using CiviCRM 4.2.0.  Is that correct?  If so, I would definitely upgrade to Civi 4.4.6, which fixes CRM-14681.  If you're still having the problem on upgrade, at least we've eliminated several variables - but I'd be willing to bet this presents itself very differently on a newer version!
Sign up to StackExchange and get free expert CiviCRM advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

PrincessAng417

  • I post occasionally
  • **
  • Posts: 50
  • Karma: 2
  • CiviCRM version: 4.5.4
  • CMS version: Drupal 7.34
Re: Memory Limit on Relationship Imports
August 14, 2014, 02:02:57 pm
I'm sorry; I'm on D7.28/Civi 4.4.5. 

I see that CRM-14681 did affect 4.4.5, so I will upgrade this weekend and give it another try.  Here's hoping that's it!

PrincessAng417

  • I post occasionally
  • **
  • Posts: 50
  • Karma: 2
  • CiviCRM version: 4.5.4
  • CMS version: Drupal 7.34
Re: Memory Limit on Relationship Imports
August 20, 2014, 02:39:20 pm
Happy to report that upgrading to 4.4.6 seems to have fixed all of the import issues we were having.

JonGold

  • Ask me questions
  • ****
  • Posts: 638
  • Karma: 81
    • Palante Technology
  • CiviCRM version: 4.1 to the latest
  • CMS version: Drupal 6-7, Wordpress 4.0+
  • PHP version: PHP 5.3-5.5
Re: Memory Limit on Relationship Imports
August 20, 2014, 02:52:54 pm
Glad to hear it, and thanks for reporting back so that other folks in the same situation can benefit!
Sign up to StackExchange and get free expert CiviCRM advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help
Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Import (Moderator: Yashodha Chaku) »
  • Memory Limit on Relationship Imports

This forum was archived on 2017-11-26.