CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Core CiviCRM Functions (Moderator: Yashodha Chaku) »
  • Altering user match rules
Pages: [1]

Author Topic: Altering user match rules  (Read 4497 times)

Chris Burgess

  • Ask me questions
  • ****
  • Posts: 675
  • Karma: 59
Altering user match rules
June 17, 2007, 09:17:08 am
Pretty sure I found this once before, but it's 4am now and my brain is a little sizzled!

I've written a script which bulk imports our existing user DB from the previous system (ran on Interbase DB). This is working ok, but we have contacts who share an email address. This causes CiviCRM to barf with "Unrecoverable error" due to a key conflict in civicrm_uf_match.

I'm using Drupal. Here's a precis of my script:

Code: [Select]
$old_users = Old_UserDB::getAllUsers() ; // simplified
foreach( $old_users as $member_id => $details ) {
  // create or update a drupal user account for each member
  if ( !$user = user_load(array('uid' => $member_id)) ) {
    $user = '' ; // array('uid'=>$member_id) ;
  }
  $account = array( 'uid' => $member_id, 'name' => $name, 'email' => $email ) ;
  $account = user_save($user, $details) ; // now $account is a drupal user stdClass
  // update some extra member contact details
  $cid = crm_uf_get_match_id($account->uid) ;
  $contact =& crm_get_contact(array('contact_id' => $cid)) ;
  $params = array( 'first_name' => $details['first_name']) ; // simplified
  $contact =& crm_update_contact($contact, $params) ;
}

I think that's enough to cover what I'm doing, though the actual script does quite a bit more (it adds separate locations for their Home and Work details, etc etc.

In order to make this import all our users, I need to disable the match coded in which identifies any two contacts with the same email address as being the same contact. (We have married members who share this detail.) This is not configurable via "Duplicate Matching", I tried already to unset that rule but it didn't help.

Can anyone give me a pointer here? I'd be really happy to share my bulk user import script once I've got past this hurdle.

Thanks!
@xurizaemon ● www.fuzion.co.nz

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Altering user match rules
June 17, 2007, 11:54:25 am

1. You should be able to make the dupe match clause empty (i.e. have no first / last name / email) and it should work (in which case its supposed to not do any dupe matching)

2. If the above fails, can u please reproduce on demo, and file an issue

3. Alternatively, u can hack the code at: CRM/Core/BAO/UFGroup.php:

   public static function findContact( &$params, $id = null, $flatten = false )

and return null for that function in all cases (thats what item 1 on this list is supposed to do)

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Chris Burgess

  • Ask me questions
  • ****
  • Posts: 675
  • Karma: 59
Re: Altering user match rules
June 17, 2007, 05:36:52 pm
Thanks Lobo. I must have described my problem incorrectly, because it doesn't seem to be that duplicate match which is the problem.

I took your advice and unset the dupe match rule, but that didn't change things; neither did setting CRM_Core_BAO_UFGroup::findContact() to return null. I may have confused things with my original description of the problem. It was 4.30am  :P

The third and fourth users are a husband and wife who share an email address. Their Drupal user gets created by the user_save() call, preserving their old username.

Then the script fails with,
Code: [Select]
Array (
    [callback] => Array (
            [0] => CRM_Core_Error
            [1] => handle
        )
    [code] => -5
    [message] => DB Error: already exists
    [mode] => 16
    [debug_info] => INSERT INTO civicrm_uf_match (uf_id , contact_id , domain_id , email ) VALUES ( 3066 ,  4 ,  1 , 'z@xxx.nz' )
                          [nativecode=1062 ** Duplicate entry '4' for key 3]
    [type] => DB_Error
    [user_info] => INSERT INTO civicrm_uf_match (uf_id , contact_id , domain_id , email ) VALUES ( 3066 ,  4 ,  1 , 'z@xxx.nz' )
                        [nativecode=1062 ** Duplicate entry '4' for key 3]
    [to_string] => [db_error: message="DB Error: already exists" code=-5 mode=callback
                        callback=CRM_Core_Error::handle prefix=""
                        info="INSERT INTO civicrm_uf_match (uf_id , contact_id , domain_id , email )
                               VALUES ( 3066 ,  4 ,  1 , 'z@xxx.nz' )  [nativecode=1062 ** Duplicate entry '4' for key 3]"]
)

The error above happens on calling crm_uf_get_match_id($account->uid) for a Drupal user whose email address is shared with another Drupal user. crm_uf_get_match_id() led me to CRM_Core_BAO_UFMatch::getContactId($ufID).

Thanks again for the quick reply, Lobo. It's Chris Burgess here - we met in Nelson a couple of weeks back. Unsure if you recognise the username  :)[/code]
« Last Edit: June 17, 2007, 05:49:50 pm by xurizaemon »
@xurizaemon ● www.fuzion.co.nz

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Altering user match rules
June 17, 2007, 06:42:57 pm

hey chris:

did not recognize the user name :)

so drupal should not allow the same email for two accounts? I'm not sure how you are getting around that issue (or if it just fails to catch that scenario), but CiviCRM synchronizes with drupal using the email address, and hence it is triggering the error below. So to some extent it is a valid error :)

Not sure if we can get around this issue, its quite core to how we sync. So the solution is: don't create two drupal accounts with the same email address (its invalid anyway).

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Dave Greenberg

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 5760
  • Karma: 226
    • My CiviCRM Blog
Re: Altering user match rules
June 18, 2007, 06:48:01 am
In case it's not obvious - your CiviCRM contacts CAN have "shared" email addresses. (They shouldn't even show up as potential duplicates if first and last name are different.)
Protect your investment in CiviCRM by  becoming a Member!

Chris Burgess

  • Ask me questions
  • ****
  • Posts: 675
  • Karma: 59
Re: Altering user match rules
June 18, 2007, 09:58:01 pm
I can see why CiviCRM uses emails to try and identify potential matches, but don't yet see why a non-unique identifier is used for UF lookups when the Drupal/Joomla ID should do. Maybe I'll understand once I've dug a little deeper.

(FX: digging sounds)

It seems (from a look around the Drupal site), Drupal doesn't technically *require* unique email addresses per user account; address uniqueness is only validated during the registration phase, and an administrator is able to create or import multiple accounts per email address. You can happily create multiple accounts per address using user_save() as I did above.

Joomla meanwhile, has a configuration option for unique email accounts per user ($mosConfig_uniqueemail in 1.0), which is on by default but not required. (Checked on current version 1.0, can't find the same setting in 1.5.)

So ... my question is, if Drupal and Joomla don't actually require unique emails per account, should I be looking at getting a fix in that removes this requirement from CiviCRM, or using a workaround?

For a workaround, I could provide internal-only email addresses that alias to the first "real" CiviCRM email detail for these members who duplicate the address. That's OK - I can just create a fake domain like "@members.my.org", then create a DB lookup alias table that aliases MEMBERID@members.my.org => result of DB lookup (if anything), and bounces when they don't have a working email.

Then any secondary user using the same email will have their Drupal account point to a unique alias, and their CiviCRM data can list the correct member email. That ought to work around it.

Thanks for the tips, Lobo & Dave. I have a couple of options to consider.
@xurizaemon ● www.fuzion.co.nz

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Altering user match rules
June 19, 2007, 01:14:24 am
Quote from: xurizaemon on June 18, 2007, 09:58:01 pm
I can see why CiviCRM uses emails to try and identify potential matches, but don't yet see why a non-unique identifier is used for UF lookups when the Drupal/Joomla ID should do. Maybe I'll understand once I've dug a little deeper.

primarily because identifiers do not mean a lot when u r trying to sync up two different data sources :). CiviCRM tries to match a drupal/joomla user to an existing contact in your CRM db via the email address before it creates a new contact.

Quote from: xurizaemon on June 18, 2007, 09:58:01 pm
It seems (from a look around the Drupal site), Drupal doesn't technically *require* unique email addresses per user account; address uniqueness is only validated during the registration phase, and an administrator is able to create or import multiple accounts per email address. You can happily create multiple accounts per address using user_save() as I did above.

Joomla meanwhile, has a configuration option for unique email accounts per user ($mosConfig_uniqueemail in 1.0), which is on by default but not required. (Checked on current version 1.0, can't find the same setting in 1.5.)

So not sure how drupal/joomla operate in the above scenario to find an user account by email and/or email a missing password. We were not aware that multiple users could share the same email address. However in this day and age, email addresses are cheap :), so not sure if we should really encourage sharing?

Quote from: xurizaemon on June 18, 2007, 09:58:01 pm
So ... my question is, if Drupal and Joomla don't actually require unique emails per account, should I be looking at getting a fix in that removes this requirement from CiviCRM, or using a workaround?

For a workaround, I could provide internal-only email addresses that alias to the first "real" CiviCRM email detail for these members who duplicate the address. That's OK - I can just create a fake domain like "@members.my.org", then create a DB lookup alias table that aliases MEMBERID@members.my.org => result of DB lookup (if anything), and bounces when they don't have a working email.

Then any secondary user using the same email will have their Drupal account point to a unique alias, and their CiviCRM data can list the correct member email. That ought to work around it.

Thanks for the tips, Lobo & Dave. I have a couple of options to consider.

I dont think the fix to CiviCRM will be trivial. I'll think a bit about this over the next few days, but we dont have a whole lot of information to match between a drupal user record and a civicrm contact record. A unique email is so so nice and convenient :) Your workaround seems a bit too complex and might not be worth it? Another option is to avoid creating a drupal user account for folks who share the same email address (or just create it for the first person on the list). Do u need a drupal account for everyone?

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Chris Burgess

  • Ask me questions
  • ****
  • Posts: 675
  • Karma: 59
Re: Altering user match rules
June 19, 2007, 07:25:30 pm
Quote from: Donald Lobo on June 19, 2007, 01:14:24 am
CiviCRM tries to match a drupal/joomla user to an existing contact in your CRM db via the email address before it creates a new contact.

Agreed. However as email addresses are non-unique, and we have a user database with a large number of family members. Trying to match them is a good thing, I agree, but I think that requiring they be unique is a limitation (which I'm prepared to put some energy into resolving, and would prefer to do this in a way that benefits all CiviCRM users rather than making a workaround just for us).

It looks like we may have 10 - 20% of our members with a shared email account - lots of family memberships I guess, combined with some repeat registrations which would be useful to identify (easier done once we've imported the contact DB into CiviCRM). We also have several hundred users who lack any email contact whatsoever, and they need to be handled nicely too.

Quote from: Donald Lobo on June 19, 2007, 01:14:24 am
So not sure how drupal/joomla operate in the above scenario to find an user account by email and/or email a missing password. We were not aware that multiple users could share the same email address. However in this day and age, email addresses are cheap :), so not sure if we should really encourage sharing?

Emails are cheap, but it's my job to work with the data as it exists, not to suggest several hundred members sign up for gMail because I couldn't find a way to deal with them sharing an address ... after all, matching surnames and phone numbers aren't stopping me from getting the data in :)

Joomla 1.0 allows multiple emails if you configure the site that way; in 1.5 this is no longer an option (an imposed restriction on user migrations). J1.5 is still in beta, of course.

Quote from: Donald Lobo on June 19, 2007, 01:14:24 am
I dont think the fix to CiviCRM will be trivial. I'll think a bit about this over the next few days, but we dont have a whole lot of information to match between a drupal user record and a civicrm contact record. A unique email is so so nice and convenient :) Your workaround seems a bit too complex and might not be worth it? Another option is to avoid creating a drupal user account for folks who share the same email address (or just create it for the first person on the list). Do u need a drupal account for everyone?

Yes, our current plan is to have a Drupal user and CiviCRM contact for each member. That way they can log into the site and adjust their own user preferences.

I agree that unique emails are convenient, but they are not always guaranteed.

Matching emails may indeed pose a problem when resetting passwords, but that problem is outside CiviCRM's domain. CiviCRM doesn't handle that so it's up to Drupal/Joomla to handle that sensibly. Another means of resetting passwords is for the administrator to do it via the GUI (extra workload, but also how a good portion of our password resets are currently done).

I understand that, given the limited amount of other user info in Drupal/Joomla by default, emails are an obvious first check to find a matching user. I don't think it follows that we should tie the email address to the UF match, and therefore make it impossible to create multiple UF matches if their email addresses match.

Hope I'm not rocking the boat too much, Lobo! Keen to chat via Skype or gTalk/AIM/etc if you have thoughts about this and want to, my username is the same all the world around.
@xurizaemon ● www.fuzion.co.nz

Chris Burgess

  • Ask me questions
  • ****
  • Posts: 675
  • Karma: 59
Re: Altering user match rules
June 19, 2007, 07:29:30 pm
PS. Have fun in SanFran!
@xurizaemon ● www.fuzion.co.nz

cap10morgan

  • I post occasionally
  • **
  • Posts: 56
  • Karma: 9
Re: Altering user match rules
June 21, 2007, 09:18:19 am
dlobo and I discussed this in the IRC channel:

I and the rest of the Code for Change team have implemented a standalone (no CMS required) version of CiviCRM and an OpenID login system to go with it. However, OpenID doesn't always provide a valid email address back to the system requesting authentication, and we don't want to require one.

So dlobo and I talked about doing something like this: create a cms_identifier field in civicrm_contact that can have a UF-defined unique string in it. For OpenID systems (Drupal 6 & Standalone), this would be the OpenID url (which is the "username" more or less for OpenID) or it could be the email address for another UF. Or all UFs could just use uf_username.site_domain in there, which would basically be an OpenID.

That last option would get around the non-unique email address problem since the usernames have to be unique across a domain anyway.

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Core CiviCRM Functions (Moderator: Yashodha Chaku) »
  • Altering user match rules

This forum was archived on 2017-11-26.