CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Installing CiviCRM »
  • Drupal Installations (Moderator: Piotr Szotkowski) »
  • CiviCRM contact to drupal user sync
Pages: [1]

Author Topic: CiviCRM contact to drupal user sync  (Read 2163 times)

jellobrain

  • I post occasionally
  • **
  • Posts: 30
  • Karma: 0
  • charmed, i'm sure.
    • jellobrain
  • CiviCRM version: 3.3.2
  • CMS version: drupal 6.x
  • MySQL version: 5.1
  • PHP version: 5.2
CiviCRM contact to drupal user sync
March 14, 2011, 12:36:08 pm
I am using drupal 6.20 and CiviCRM 3.3.2.

I have an installation of drupal with 26,200 users in the user table.  When I click to 'Synchronize Users to Contacts' (http://netsquared.org/civicrm/admin/synchUser?reset=1) and click okay, the process always ends in an error usually related to memory allocation (even after increasing the memory limit to 192M) - usually a timeout kind of thing after about 10minutes. 

What I notice is that although it times out on the browser, the database is still performing the opperations (the number of entries in the civicrm_uf_match table fluctuates for a while afterwards).

Once it stabilizes (I've done this a few times) it gives a different end number in the civicrm_uf_match table:
20,126
19,994
19,468
20,086

Once that table stabilizes, it has never had the same amount of entries as is in the users (drupal) table.

Can you help me understand if/how these errors/discrepancies are occurring and if there is a need to do anything about them?  Is there any documentation that I can read that explains what is going on and might give me insight in how to address it? 

Thanks!
__

jellobrain.com

jellobrain

  • I post occasionally
  • **
  • Posts: 30
  • Karma: 0
  • charmed, i'm sure.
    • jellobrain
  • CiviCRM version: 3.3.2
  • CMS version: drupal 6.x
  • MySQL version: 5.1
  • PHP version: 5.2
Re: CiviCRM contact to drupal user sync
March 21, 2011, 12:54:03 pm
I am stuck here, and just wanting to understand if my lack of response is because I am not posting in the right place or correctly...

Anyone?
__

jellobrain.com

jellobrain

  • I post occasionally
  • **
  • Posts: 30
  • Karma: 0
  • charmed, i'm sure.
    • jellobrain
  • CiviCRM version: 3.3.2
  • CMS version: drupal 6.x
  • MySQL version: 5.1
  • PHP version: 5.2
Re: CiviCRM contact to drupal user sync
March 25, 2011, 07:43:44 pm
I completely REMOVED CiviCRM and started from scratch, and as soon as I fill the civicrm_uf_match table, it starts growing and shrinking again. 

Is this supposed to happen?  It isn't happening on any of my smaller CiviCRM installations...

Here is the saga that I tried before conceding to starting from scratch: (from an internal post)
the question i am asking myself is what is it going to take to kill the beast. here is the chronicle of my final (and i think somewhat heroic) effort to kill this process without having to completely reinstall civicrm:

1) with the same conditions as described in my last comment below (civicrm modules folder renamed, CMSUser.php changes as per the civi-forum mentioned below, run cron and clear all caches again, set php execution_limit via .htaccess file to 3600, rebooted mysql server, etc, etc)

2) i remove the civicrm_uf_match table completely from the database (after backing it up, of course) for 15 hours (54000 seconds).

3) when i reload that table, i begin checking the numbers in that table, and immediately they start fluctuating again.

short of a conversation with someone at civi who will pay attention to my ticket/s on their forums, i am not sure what to do. i have spoken with philip as well (who came up with the forum link below that i used to change the CMSUser.php file), and i am curious what he would think at this point.
__

jellobrain.com

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: CiviCRM contact to drupal user sync
March 25, 2011, 10:00:51 pm

the syncuser stuff was not written to process a large number of users. For 26K users, i would recommend either:

a. writing a custom script to do the same and potentially doing it in batches (using drupal batch api?)

b. modifying the syncuser script to do small chunks at a time and doing it smartly so it can make progress and/or call itself back to process the next chunk

doing the latter will enable future users to sync large number of users. Would be great if net2 can work on this and contribute it back to core

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

jellobrain

  • I post occasionally
  • **
  • Posts: 30
  • Karma: 0
  • charmed, i'm sure.
    • jellobrain
  • CiviCRM version: 3.3.2
  • CMS version: drupal 6.x
  • MySQL version: 5.1
  • PHP version: 5.2
Re: CiviCRM contact to drupal user sync
March 26, 2011, 06:24:49 am
hey lobo, thanks for your response.  i actually, as i said in my last entry, did do it in small batches (according to the link above), and the synching part itself went great.  the issue is that there might have been old processes that were still alive from the first time we tried. 

i am gatherring from your respoinse that this civicrm_uf_matches table is indeed not supposed to continually fluctuate.  any ideas how to destroy old processes from my first try that are not seeming to have died?  at this point, i completely removed and reinstalled civicrm on both machines.  the interesting part to me is that this last time, i only ran the synching in dev, and then copied that database over after doing a very simple installation (without configuring anything) on production. 

The point being that the new civicrm_uf_match table in production is fluctuating as well.   I never actually ran the synching in production this time around (instead i did it in dev and then copied the database over) so I am assuming that these are old processes that still have not died.

Any ideas?
__

jellobrain.com

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: CiviCRM contact to drupal user sync
March 26, 2011, 09:45:44 am

U'll need to get a sysadmin or someone who can check the log files and see whats happening and why. Cant figure it out from this info. I suspect someone will need to take a look at what processes are running etc

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Installing CiviCRM »
  • Drupal Installations (Moderator: Piotr Szotkowski) »
  • CiviCRM contact to drupal user sync

This forum was archived on 2017-11-26.