CiviCRM Community Forums (archive)

I've found a way of excluding known non-duplicates which doesn't require adding data to all contacts.

• Create a custom data group with a single field
• Change the dedupe rules to include that field, but with a NEGATIVE weight (ie, so a match on that field causes the overall match to be less than the threshold)
• Using that rule, when two or more contacts match, but are shown to be non-duplicates, set their custom data field to the same value
• That group of contacts will not appear in any future search for duplicates

Ken

Ken - this seems like an interesting approach, and probably quite useful for other folks.

Would be cool if you could add this as a {tip} to the wiki section on Finding Dupes!

http://wiki.civicrm.org/confluence/display/CRMDOC/Find+Duplicate+Contacts

Excellent idea! Thanks a lot, Ken.

Peter,

I updated the page http://wiki.civicrm.org/confluence/display/CRMDOC/Find+Duplicate+Contacts back on August 20 as Dave G suggested. Sorry for not posting back here to advise.

We haven't had a huge deal of experience with this approach, as we are a small outfit who has to live with some degree of ambiguity: we generally only clean up data when a need presents itself, and when the team has time (which isn't often).

I like your Relationship approach as it relies on an existing feature rather than requiring new custom data. The custom data approach presents itself on every contact record (noisy) while the relationship only appears where it needs to (efficient). (Though perhaps the contra there is that being in-your-face, the custom data approach might be easier to train people in?)

How would this scenario work?

Say "John Smith" 1 and "John Smith" 2 have a relationship "share the same name". Another "John Smith" comes along: will he be a candidate for deduping? I think he will, as he's in the group you propose (persons not in a "shares name" relationship). If #3 is a duplicate he gets merged (ouch!). If he's not, we create a relationship with one of the first two.

If we realise we've made a mistake (they are duplicates), we can remove the relationship from the duplicate and then merge it with the original (I think we need to do things in that order because if we merge while the relationship exists, we run the risk of either having a relationship from and to the same contact, or 2 relationships between the same pair.)

Finally, how do you create the group "not in a 'share name' relationship"? My mind's gone blank on that one.

Smart thinking, oh fellow Antipodean!

Ken

(we will beat you guys in the Rugby one day, you know! People have told me I'm a patient guy, and I might need that!)