CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Developer Discussion (Moderator: Donald Lobo) »
  • How to handle contacts for drupal users that never logged in
Pages: [1] 2

Author Topic: How to handle contacts for drupal users that never logged in  (Read 6124 times)

grahamgilchrist

  • I post occasionally
  • **
  • Posts: 70
  • Karma: 3
How to handle contacts for drupal users that never logged in
October 27, 2010, 04:33:53 am
Hi Guys,
Bit of a general query really, but it seemed to relate to custom coding so I put it here (please feel free to move to a more appropriate forum). I couldn't find this info elsewhere in the forum or in the docs/book.

We run civicrm+drupal.
We have contacts added manually by administrators, as well as web users. When web users sign up to the drupal site, we are using a custom drupal module to add some fields to the registration form corresponding to custom contact fields in CiviCRM (in a similar way to profiles).

To avoid spam, and prove identities etc. we have the user options in drupal set to 'require e-mail verification', so when someone signs up, they receive an authorisation email to check they are who they say they are (or at least own that email address).

Q1: CiviCRM creates a contact for each drupal user at the point of registration, before they have confirmed their email address and logged in for the first time.
If a contact already exists in CiviCRM, and someone signs up to the drupal site with the same email address, does the account get immediately linked?

Q2: What happens if there is more than one contact with this email address?

Q3: What happens if someone fakes a signup with another person's email address? If it gets linked before the user has verified it, then any info put in the profile on the signup page might overwrite the correct info for that contact

Q4: What happens if an administrator manually edits the contact info between signup and first login. Do the profile fields from the signup form overwrite the manually entered data?

Q5: About 25% of our web users that sign up never log-in. We presume this is due to spam, incorrect email addresses and various other reasons. Since CiviCRM creates contacts for them as soon as they register, we end up with a lot of contacts which aren't 'real' contacts, either because their email addresses don't exist or they were spammers. This tends to clog up the database and doesn't give a true account of the number of contacts we are engaging with.
Do other people have this problem?
How do you deal with it?

Q6: As a result of these problems, is there any way I can get Civi to only create the webuser->contact link when the user first logs in? Or disable the auto-linking on registration so that I can link it myself on login with a drupal module?

Does anyone have any other suggestions for how I could sort out this workflow?

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: How to handle contacts for drupal users that never logged in
October 27, 2010, 08:06:32 am

hey graham:

this has come up in the past and there currently is no non hackish workaround

the easiest way is potentially:

1. Do not use a CiviCRM Profile for user registration

2. Hack civicrm.module to ignore the register / insert / validate ops of the hook_user (line 384 - line 412 of drupal/civicrm.module)

3. Use a profile on My Account, and collect contact information there

Alternatively the hook code can be modified to perform the operation ONLY if the user is in an unblocked state

wanna take a look and contribute a patch?

lobo


A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

grahamgilchrist

  • I post occasionally
  • **
  • Posts: 70
  • Karma: 3
Re: How to handle contacts for drupal users that never logged in
October 28, 2010, 06:12:40 am
Unfortunately our organisation has reporting requirements that means we must collect location information (only on a rough regional basis) from all contacts. The only way to guarantee getting this information is to put it on the registration form, otherwise once logged in you have no way of forcing them to fill it in.
I guess we could have a check that runs on every page load to see if they have filled the data in but this would cause a performance hit, and would be quite annoying for the first time they log-in.

I'd be willing to try compiling a patch, since I am having to code around this issue anyway.

I need a bit more info to understand how it works properly first though...

1) Are you saying that using a profile on the registration form definitely inserts it's data as soon as the contact is created/linked on registration. And that it overwrites what is already there?

2) The code which matches a civicrm user to drupal user in in Core/BAO/UFMatch.php
What happens if there are two civicrm contacts with the same email address as the new drupal user. How does it choose?

3) I'm trying to work out how the hook_user in civicrm.module works. Could it be as simple as changing:

Code: [Select]
   case 'login':
        require_once 'CRM/Core/BAO/UFMatch.php';
        return CRM_Core_BAO_UFMatch::synchronize( $user, false, 'Drupal',
                                                  civicrm_get_ctype( 'Individual' ), true );
    case 'register':
        $config = CRM_Core_Config::singleton( );
        if ( $config->inCiviCRM ) {
            return;
        }
 
        if ( empty( $_POST ) ) {
            return civicrm_register_data($edit, $user, $category, true, false );
        } else {
            return civicrm_register_data( $edit, $user, $category, false, true );
        }
        break;

to this:

Code: [Select]
   case 'login':
        $config = CRM_Core_Config::singleton( );
        if ( $config->inCiviCRM ) {
            require_once 'CRM/Core/BAO/UFMatch.php';
            return CRM_Core_BAO_UFMatch::synchronize( $user, false, 'Drupal',
                                                  civicrm_get_ctype( 'Individual' ), true );
        } else {
           //not entirely sure what this if (empty()) does?
           if ( empty( $_POST ) ) {
               return civicrm_register_data($edit, $user, $category, true, false );
           } else {
               return civicrm_register_data( $edit, $user, $category, false, true );
           }
         }
 

    case 'register':
        break;

I'm not entirely sure what the if (empty($_POST)) is for?

4) This would fulfill my requirements, but could there be a legitimate use case where you definitely wanted the contact link/create to happen on registration as it does now? If so, this would need an option in the admin config screens somewhere (or just under an admin page for this drupal module).
If we have the option to change, then when the setting is changes from 'link on login' to 'link on registration' would we need to update everyone who hadn't logged in yet with a sync account?

grahamgilchrist

  • I post occasionally
  • **
  • Posts: 70
  • Karma: 3
Re: How to handle contacts for drupal users that never logged in
November 08, 2010, 03:45:52 am
Any opinions on the above suggestions?
I am testing my above ideas now, but It's a feature that we would find really useful, as we require email validation and collecting extra data on user registration.

I can patch the civicrm.module easily enough to only associate contacts and users on registration, but is this going to affect some other use-case and break things for other people?

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: How to handle contacts for drupal users that never logged in
November 08, 2010, 06:55:36 am

u'll lose the location info if u dont process it during registration (unless u save it in some other table)

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

grahamgilchrist

  • I post occasionally
  • **
  • Posts: 70
  • Karma: 3
Re: How to handle contacts for drupal users that never logged in
November 08, 2010, 07:01:24 am
Thanks for the info.
Is it not the case that drupal saves any extra fields added to the registration form in the user object, which should then be a simple effort to extract them at login.
I will do some tests to check we can make this work.

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: How to handle contacts for drupal users that never logged in
November 08, 2010, 08:13:56 am
Isn't it possible to have a hook that is run when user are unblocked ?

In that case, I'd create a hook after the contact creation from the registration, add a tag "unconfirmed", and remove it in the drupal hook when the user is unblocked.

X+

(we are doing something quite similar for the petitions as an option)
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

grahamgilchrist

  • I post occasionally
  • **
  • Posts: 70
  • Karma: 3
Re: How to handle contacts for drupal users that never logged in
November 08, 2010, 08:50:49 am
Hi Xavier, thanks for the reply
Quote from: xavier on November 08, 2010, 08:13:56 am
Isn't it possible to have a hook that is run when user are unblocked ?

Yes, but this would also run *every* time the user was blocked/unblocked. For instance, we sometimes have to use the block feature for staff who have left but we need to retain their accounts, or users who have specifically caused a problem (spam etc.)
Unblock can be triggered by an administrator to activate the user/contact link, whilst login can only be activated by the end-user, so unblock would be better...
I guess one could set a flag so it only happened on the first unblock, which effectively works the same as the first login flag.

Quote from: xavier on November 08, 2010, 08:13:56 am
In that case, I'd create a hook after the contact creation from the registration, add a tag "unconfirmed", and remove it in the drupal hook when the user is unblocked.

I don't know that this solves the problem.
There are two issues I see that need to be resolved:

1) If you create the user->contact link on registration before you have validated the user's email address, you can end up associating the wrong user and contact. A malicious user could sign up with another contact's email, and be linked to that contact. Since they would never receive the validation email, the contact would be forever linked to the wrong user, preventing the real user from logging in and confusing the civiCRM staff.

2) Even worse, if any form data is collected on the registration page (profiles etc) this would be immediately saved to the incorrect contact's profile, possibly overwriting correct data. Another problem is that staff may then enter data for a contact which has registered but not yet logged in, and when this user logs in, the manual data is overwritten when it may be the more accurate info.

My suggestion would be not do anything on user registration, apart from saving extra data fields from profiles etc. into the drupal user object. I think this happens automatically if you use the form_user_register_modify hook (which is what our custom module does). I haven't delved this far yet to see how civiCRM handles profiles on the registration form, but I am assuming it uses this hook.

Then on first login (or first unblock as Xavier suggested), the user->contact link is created, and then the extra field info is retrieved from the user object and saved to the contact. Finally you then empty the drupal user object to clean up.

Can anyone think of any use cases where this does not work? i.e. the users->contact link must be created immediately?
« Last Edit: November 08, 2010, 08:54:34 am by grahamgilchrist »

petednz

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4899
  • Karma: 193
    • Fuzion
  • CiviCRM version: 3.x - 4.x
  • CMS version: Drupal 6 and 7
Re: How to handle contacts for drupal users that never logged in
November 08, 2010, 10:26:39 am
HI Graham - your explanations pretty much reflect a post we made many moons ago, particularly your 1) and 2) below . Our clients weren't in a position to contribute anything towards this sort of improvement.
Sign up to StackExchange and get free expert advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

pete davis : www.fuzion.co.nz : connect + campaign + communicate

grahamgilchrist

  • I post occasionally
  • **
  • Posts: 70
  • Karma: 3
Re: How to handle contacts for drupal users that never logged in
November 09, 2010, 03:40:52 am
Quote from: peterd on November 08, 2010, 10:26:39 am
HI Graham - your explanations pretty much reflect a post we made many moons ago, particularly your 1) and 2) below . Our clients weren't in a position to contribute anything towards this sort of improvement.

Well it's good to know it's not just us :)

I am in a position to do some work on this, and could contribute some patches if this would be of use, as it would be beneficial to our organisation too.
Before I go ahead though, I think as a community, we ought to decide the best way to approach this. I don't want to go ruining things which are there for a specific reason!

I have some questions about the current process that hopefully someone can answer

1) Current 'registration page' profiles modify the registration form using hook_user in civicrm.module. I can't find which function in the current civicrm.module actually saves the profile data to civicrm - is it done automatically in the UFMatch::synchronize function?
2) How does the UFMatch::synchronize function choose to link a user to a contact? I know it uses the default strict dedupe rule (which is by email only), but what happens if there are two matching contacts? (i.e. two with the same email address - we have several of these)
3) The 'insert' switch in hook_user checks to see if it is on a civicrm page by calling civicrm_on_user_page(). What is this for? The only thing I can think this might be used for when civicrm will be creating drupal users. When does this happen

Key decisions to be made to progress with this:

a) Are there any use-cases where you would definitely want a contact for every single user, as soon as they had registered (but not verified their email and logged in)?
b) Do we therefore need an admin option which allows you to choose between 'link contacts on register' and 'link contacts on login', or is this too confusing?
c) If we changed things so that contacts were created/linked on login/unblock, not all users would have a related contact. Would this break any code or other assumptions made anywhere else?

Proposal:

Make changes to civicrm.module that do the following:

  • Alter the hook_user function (http://api.drupal.org/api/drupal/developer--hooks--core.php/function/hook_user/6) in civicrm.module so that nothing happens on registration submit. i.e. remove this:
Code: [Select]
    case 'insert':
        $config = CRM_Core_Config::singleton( );
        if ( $config->inCiviCRM ) {
            return;
        }

        // did civicrm generate this page, or is it via a user hook?
        if ( civicrm_on_user_page( ) ) {
            return civicrm_register_data( $edit, $user, $category, false );
        } else {
            require_once 'CRM/Core/BAO/UFMatch.php';
            CRM_Core_BAO_UFMatch::synchronize( $user, false, 'Drupal',
                                               civicrm_get_ctype( 'Individual' ) );
        }
        break;
to this:
  • Drupal should save anything modified using hook_form_user_register_alter() (including profile data) to the user object in the DB
  • We now have a drupal user, but no civicrm contact. When the drupal user validates his/her email, they then login
  • Add to hook_user a switch for 'login' which checks whether the user has a civicrm contact match, and if it doesn't, links the user and contact, then pulls the profile info out of the user object and puts it into the contact.
    i.e. add this:
Code: [Select]
    case 'login':
        $config = CRM_Core_Config::singleton( );
        if ( $config->inCiviCRM ) {
            require_once 'CRM/Core/BAO/UFMatch.php';
            return CRM_Core_BAO_UFMatch::synchronize( $user, false, 'Drupal',
                                                  civicrm_get_ctype( 'Individual' ), true );
        } else {
                require_once 'CRM/Core/BAO/UFMatch.php';
                CRM_Core_BAO_UFMatch::synchronize( $user, false, 'Drupal',
                                               civicrm_get_ctype( 'Individual' ) );
               //save profile data here if we haven't already
        }
    }
    break;

    What do people think?

    xavier

    • Forum Godess / God
    • I’m (like) Lobo ;)
    • *****
    • Posts: 4453
    • Karma: 161
      • Tech To The People
    • CiviCRM version: yes probably
    • CMS version: drupal
    Re: How to handle contacts for drupal users that never logged in
    November 09, 2010, 07:59:19 am
    Hi,

    I have several use case of willing to register the contact when you create the user:

    1) If you have a profile on a membership/event registration with the option to create an account, I definitely want to register the contact and the associated payment in Civicrm, even if the never confirm their account

    2) to be able to use civimail to send an email "hey, you haven't confirmed your account"

    Another option: create the contact when creating the account, but with the status "deleted", and you "undelete" it when the account is confirmed.

    While we are on that region: I'd love to see the source set "account created online" when creating the contact, so you can more easily see where the contact comes from.

    X+

    -Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

    petednz

    • Forum Godess / God
    • I’m (like) Lobo ;)
    • *****
    • Posts: 4899
    • Karma: 193
      • Fuzion
    • CiviCRM version: 3.x - 4.x
    • CMS version: Drupal 6 and 7
    Re: How to handle contacts for drupal users that never logged in
    November 09, 2010, 11:12:14 am
    Quote
    2) How does the UFMatch::synchronize function choose to link a user to a contact? I know it uses the default strict dedupe rule (which is by email only), but what happens if there are two matching contacts? (i.e. two with the same email address - we have several of these)
    Not sure if the above is totally correct but what I am adding may not be news (or even right). The match in the UFMatch table is on email, but the strict rule can be changed to whatever you want eg first+email or first+last+email, provided of course those fields are exposed in a Profile used for Drupal User Create
    Sign up to StackExchange and get free expert advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

    pete davis : www.fuzion.co.nz : connect + campaign + communicate

    Donald Lobo

    • Administrator
    • I’m (like) Lobo ;)
    • *****
    • Posts: 15963
    • Karma: 470
      • CiviCRM site
    • CiviCRM version: 4.2+
    • CMS version: Drupal 7, Joomla 2.5+
    • MySQL version: 5.5.x
    • PHP version: 5.4.x
    Re: How to handle contacts for drupal users that never logged in
    November 09, 2010, 11:24:50 am

    might be easier to discuss this on IRC :)

    Unfortunately in D7 the user hook has been broken up into smaller chunks (a good thing), but the code will need to be modified for that release (when it comes out)

    lobo
    A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

    grahamgilchrist

    • I post occasionally
    • **
    • Posts: 70
    • Karma: 3
    Re: How to handle contacts for drupal users that never logged in
    November 11, 2010, 03:25:41 am
    Quote from: xavier on November 09, 2010, 07:59:19 am
    I have several use case of willing to register the contact when you create the user:

    1) If you have a profile on a membership/event registration with the option to create an account, I definitely want to register the contact and the associated payment in Civicrm, even if the never confirm their account

    2) to be able to use civimail to send an email "hey, you haven't confirmed your account"

    Hmm, well obviously this needs more thought.
    My latest idea is to keep most of the module code 'as is', so as not to break any existing usage scenarios, but add a check in the registration hook to see if the request has come via the user registration form (and not via drupal API, admin interface etc.)
    It only then runs the ufMatch sync if this is not the case (i.e. not for user signup), otherwise it saves the form data.

    We then repeat the registration form code on a login hook, which does the contact sync, then gets the form data from the user object and adds it to the user account.

    Quote from: xavier on November 09, 2010, 07:59:19 am
    Another option: create the contact when creating the account, but with the status "deleted", and you "undelete" it when the account is confirmed.
    This seems a good idea, but isn't there a 'purge all deleted users' option somewhere? If you run this there could the danger of accidentally deleting these users?

    Quote from: xavier on November 09, 2010, 07:59:19 am
    While we are on that region: I'd love to see the source set "account created online" when creating the contact, so you can more easily see where the contact comes from.

    I don't think this would be that difficult? Just a contact API call after the user has been synced to a contact?

    Quote from: Donald Lobo on November 09, 2010, 11:24:50 am
    might be easier to discuss this on IRC :)

    Cheers Lobo. I'll have a go but IRC is difficult for me as I am in and out of meetings often so can't stay around for long! The forum is also better for posting code snippets etc.

    Quote from: Donald Lobo on November 09, 2010, 11:24:50 am
    Unfortunately in D7 the user hook has been broken up into smaller chunks (a good thing), but the code will need to be modified for that release (when it comes out)

    Yes, but I don't think this will affect this problem much? The existing module would have needed to be ported to D7 anyway, and the API isn't *that* different...

    grahamgilchrist

    • I post occasionally
    • **
    • Posts: 70
    • Karma: 3
    Re: How to handle contacts for drupal users that never logged in
    November 11, 2010, 08:55:11 am
    Hi everyone.

    I've made some modifications to civicrm.module to try to solve this issue (see attached new version). I added a flag to the user-register form to indicate that it was the source of the registration (rather than an API call or admin screen). Then when the user is created, the contact sync only happens if this flag is not present. I then added a check to the login which syncs the user to CiviCRM if it doesn't already exist. This keeps the existing functionality, but only changes things if the user create route comes via the user/register website form.

    I have attached my hacked version in case it helps anyone else in this situation, and in case some of it can be absorbed into the main CiviCRM codebase.

    Unfortunately, this version does not save CiviCRM profile data.
    On our site I have a custom module which collects extra info much like the profiles system (but with some custom UI that profiles could not provide). My module works using the standard drupal form API by injecting extra fields into the registration form. These become automatically saved in the user object and retreived and synced to CiviCRM on login.

    CiviCRM by contrast, uses Quickform to process user registration profile fields. They are added to the user registration form as raw html, and not using the Drupal form API, thus Drupal knows nothing about them and does not save the data in the user object/DB. To get this working, one would have to manually save the profile fields to another table in the DB, and then retrieve them on login.

    New suggestion:
    Perhaps we are going about this the wrong way.
    Since the drupal user registration match uses the 'strict' individual rule, we could modify this rule to never match any contacts and thus 'force' a new user for every contact. You lose some of the automated linking of users->contacts, but this can be performed during regular contact merge/dedupe runs. It also means the profile data won't overwrite anything important.
    The main problem for me with this approach is that the contact import, event registration and contribution system all use the 'default' strict rule. We really need imports to match properly against email addresses.

    So, it would be really useful to be able to specify seperate strict dedupe rules for website registration, event registration, contribution pages and contact import. I know this has come up before, so would this be a good idea to try to patch this functionality? It would just mean changes to a few admin screens and a couple of extra fields stored wherever the dedupe rules are kept (DB or $config object?)

    What do you think of this idea?

    Thread summary
    Since this has got rather long, I thought I would summarize.

    Problem:
    Many sites require email verification from website signups to prove identity. Until the user has first logged in, you do not know that they have received the verification email, and are who they say they are. CiviCRM syncs users->contacts on user registration, which means a user could set up an account for an email address they do not own, and it would be linked to CiviCRM, effectively 'stealing' that contact.
    If any profile fields are used to collect additional information from the user, these would overwrite the valid entries with whatever the 'fake' user had entered.

    Solutions:
    It seems there is no 'catch-all' solution to this problem. It is reasonably easy to fix the 'fake' synchronisation problem by just doing the sync on login rather than registration (see attached module).
    The complex part is what do do with the profile data.

    If there is a long time between registering and logging in, the profile fields may have changed and not save properly.
    If there is a long time between registering and logging in, the contacts details may change (say moving address) and then when they login the old out-of date data from registration may overwrite the newer correct data.

    The ideal fullproof process seems quite complex:
    -website registration
    -drupal account created
    -submitted civicrm profile saved somewhere
    time passes
    -user gets email and logs in
    -load saved profile.
    -Remove fields in stored profile that no longer exist.
    -For each field, only update that field in CRM if it hasn't been changed since date of registration. This is the main problem, as we do not have field level change logging in CiviCRM. It also gets quite complex, because we can't update part of an address for example...
    -remove saved profile

    As you can see, some of this is probably not possible in current CiviCRM. Here is a summary of the various suggestions in this thread for how to approach this issue:

    1) Add contacts immediately on registration but as 'deleted'
    Good idea, but these contacts may get removed between registration and login using the 'permanent delete' facility. What happens if contact undelete is not turned on? Could they be added as an extra level of 'hidden' which would need an access permission to view? Then how doe sit it interact with all the components, de-dup etc.

    2) Add contacts from website signup on login instead of registration (save in Drupal user object) (attached module)
    Requires user-registration profiles to use drupal form API. Doesn't handle accounts made with event registrations or contributions

    3) Add contacts from website signup on login instead of registration (save to CiviCRM DB)
    Requires DB table to store profile fields. Could we serialize the $_POST object and then use quickform to process on login? Can quickform even work this way without a direct submit? Doesn't handle accounts made with event registrations or contributions

    4) Modify dedupe rules system to accept different strict rules for import and website registration
    Automatic sync lost. New contacts are created for each user so need to run de-dupe every so often.

    I would appreciate the thoughts of anyone else with this problem and how they solved it or what approach they would prefer to use? This is starting to drive me slightly crazy!

    Pages: [1] 2
    • CiviCRM Community Forums (archive) »
    • Old sections (read-only, deprecated) »
    • Developer Discussion (Moderator: Donald Lobo) »
    • How to handle contacts for drupal users that never logged in

    This forum was archived on 2017-11-26.