Export producing multiple records based on how many phone numbers some one has

[petednz]

Hi - using Find Member or Adv Search there is a problem - for me at least - in that the Search finds eg 11 contacts but the export file contains 14 records because it replicates the record for any person who has more than one 'phone' field - i suspect it will do this also for other fields where there is more than just the primary data
eg in your sandbox - Find Members >> Memb Type = General >> 11 records found including Mrs Demo Demo

>> Export and I end up with Mrs Demo Demo being listed three times because three of her phone fields have got data in them (in this case on 000 but that isn't the problem).

This is causing us serious problems with exporting lists for use - can this be fixed easily?

[petednz]

I note that a similar issue is affecting 'Batch Update by Profile' whereupon the list that appears when I hit 'continue' is also showing repeated contacts, which relates to the number of phone numbers in their Primary Location (though even here I note that person A has PHone Fax and Mobile and is listed 3 times, person B has Phone and Mobile and is listed twice, but person C also has Phone and Mobile but on her Contact view I spot that her Mobile is not in Bold where as the other numbers of the other people are in Bold - what is going on?)

[petednz]

And I will note that having tried the 'batch update' to see what would happen I got an error
which may be to do with these duplicates - or maybe not -
Error Details:

Array
(
    [callback] => Array
        (
           

• => CRM_Core_Error
[1] => handle
        )

   

Code: [Select]

=> -5
    [message] => DB Error: already exists
    [mode] => 16
    [debug_info] => INSERT INTO civicrm_value_1_officers_and_roles  SET position_or_role = '', branch_officeholder = '', domain_id = 1, entity_id = 12906  [nativecode=1062 ** Duplicate entry '1-12906' for key 2]
    [type] => DB_Error
    [user_info] => INSERT INTO civicrm_value_1_officers_and_roles  SET position_or_role = '', branch_officeholder = '', domain_id = 1, entity_id = 12906  [nativecode=1062 ** Duplicate entry '1-12906' for key 2]
    [to_string] => [db_error: message="DB Error: already exists" code=-5 mode=callback callback=CRM_Core_Error::handle prefix="" info="INSERT INTO civicrm_value_1_officers_and_roles  SET position_or_role = '', branch_officeholder = '', domain_id = 1, entity_id = 12906  [nativecode=1062 ** Duplicate entry '1-12906' for key 2]"]
)


[Donald Lobo]

that was because the demo demo user has multiple primary phone numbers which in turn triggered a bad export. There was an issue with this in prior upgrade scripts (which has been fixed in beta3)

i do think there is a bug in the code where we add a new phone number / email and also make it primary. If we can figure out how to reproduce this, we can fix it Any ideas on how u got 3 primary phone numbers on demo demo?

if u edit and save the contact, we fix the db and the primary issues in the save process. Yes ugly workaround for now

lobo

[petednz]

Hi - okay so it is the multiple primary that is the issue - i didn't create Mrs Demo Demo so can't comment but obviously lots of our records have this problem. And I note that, as you say, if I open and save a record, the multiple BOLD numbers on the Contact record change to just showing one.
So, ugly, but can it be automated. I note that when the contacts were not 'mapping' there was an option to run 'something' that caused every record to basically recalculate its co-ordinates - would a cron job on the Address records be an option or am i thinking out of the wrong end of my spine?
I will see if Chris can shed any light on why ours have so many with multiple primary - but have lost sight of hiim to BIRT for the recent past 

[Donald Lobo]

should be quite simple to fix the email / address / phone tables via a php script that interacts with the db. All it needs to do is for each contact ensure that only one email / address / phone is primary (too bad we cant put an easy db constraint for this and trap all errors).

i'll chat with chris and get him going on the script.

lobo

[petednz]

Good luck getting hold of chris - but seriously - has this bug got anything to do with this inexplicable problem. Or have I hit another one - or is it a clue to the first one.

So i just did an import and successfully allocated lots of 'office roles' to members (yes i am moving away from relationships!) - so I then 'exported' my smart group - which shows the import was good - other than the export produced 3 x Steffan - because of this bug - so, then i open his summary and hit save so he only has the one primary - and reconduct my search hoping to show that the export now only has one of him and guess what - no steffan at all in the search results - and when I go and check his record his 'office roles' are all cleaned out.

And just to verify, I opened another contact from my search result who I know was being exported twice, hit save, and bingo, my custom data for officers has been cleaned out. Now I am beginning to worry that it may be cleaning out other data too (which may explain concerns that some of our data is getting lost). Of course I can't tell what I have lost without doing some more digging (and people have gone home) but will try and find another example I can mess with.

[Donald Lobo]

This is a bit scary and we'd like to track it down and fix asap

Considering this might be painful to recreate, i can logon to your test servers and do some diagnosis there. Can you give chris all the details. We'll need a few test records that follow the pattern, so we can replicate etc

thanx

lobo

[Donald Lobo]

This was a nasty bug (IMO) and has been present since CiviCRM v1.8. It is now fixed in v2.0

the relevant issue is here: http://issues.civicrm.org/jira/browse/CRM-2763

lobo

[petednz]

Nothing like a nasty bug to get the immune system pumping - thanks for your superb help sorting this - and nice to know I didn't really break things - any thought re clarifying how this issue may have affected data as I suspect users may have incurred the downside without even realising it.

[Donald Lobo]

Here's the issue:

If you were using tabbed custom groups in v1.8 / v1.9

AND

if the tabbed group had a multi-select OR checkbox custom field

THEN

on any contact edit, the values would be reset for ALL the above multi-select / checkbox custom fields

unfortunately i cant think of any way folks can recover this lost data in a easy manner. I'm a bit down that we did not catch this issue earlier

lobo

[petednz]

Thanks Lobo - amazed how quickly you got it sorted once we found it  and of course we only found it because of a self-inflicted (possibly)problem with these multiple exports.