CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Core CiviCRM Functions (Moderator: Yashodha Chaku) »
  • De-duping help for a newbie
Pages: [1]

Author Topic: De-duping help for a newbie  (Read 594 times)

chp

  • I’m new here
  • *
  • Posts: 20
  • Karma: 0
  • CiviCRM version: 4.1
  • CMS version: Drupal
  • MySQL version: unk
  • PHP version: unk
De-duping help for a newbie
May 21, 2012, 07:25:19 am
Hi all,

CiviCRM 4.1, freshly installed
Hosted on Bitnami using a VM with Drupal 7
Browsing with Safari on a Mac

I'm having lots of trouble with CiviCRM at this point.  I've imported 4335 records into a test site (import seemed to hang on the "importing records" screen, but when I manually navigated back to the dashboard, the records were there - errors were reported probably relative to blank fields or records???).  Although I chose to de-dupe with the Indiv-->Strict filter, I still had all the dupes imported (for example, 102 copies of one individual). 

I then tested the de-dupe procedure, again using the Indiv-->Strict filter and it too hangs, now on a blank screen (Safari thinks it's done loading but the screen is blank).  When I manually navigate to the records, I again see all 4335 records, none merged.  Finally, I found the one record that has 102 copies, then selected them (with the checkbox) and tried Merging from the actions menu.  Alas, this approach only lets me merge two records at a time - that'd take me a LONG time to do. 

Is there a way to merge dupes when records have multiple copies, not just two?  This is from a Quickbooks export of transactions so there were many records for each individual.  I know I can export a contact list from QB to avoid this problem when I do the real import, but I'm trying to understand what is possible and what isn't in CiviCRM.

Thanks so much!
Carl

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: De-duping help for a newbie
May 21, 2012, 08:58:27 am

4.2 (alpha in the next few weeks) has batch merge functionality that you can use to do an automatic batch merge (use with care)

however, we do recommend that if you can, you should do de-dupe outside of civicrm. Much better to start with nice clean data

lobo
 
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

chp

  • I’m new here
  • *
  • Posts: 20
  • Karma: 0
  • CiviCRM version: 4.1
  • CMS version: Drupal
  • MySQL version: unk
  • PHP version: unk
Re: De-duping help for a newbie
May 21, 2012, 08:26:01 pm
OK, thanks for the heads up about 4.2 Lobo.  And I get that batch merging comes with cautions.  I can do de-duping outside of CiviCRM for this batch of contacts.   Only thing is, when you have a few thousand dupes or more, it doesn't really matter where you do the merge/purge, the user will still have to make compromises.  But if it means not having to do a few thousand merges by hand, I for one am willing to make those compromises.  Time will heal any data wounds suffered that way!

New question:  so instead of doing the merge/purge, is there a way to batch delete contacts?

Thanks again,
Carl

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: De-duping help for a newbie
May 22, 2012, 06:44:24 am

Might want to read the civi-book: http://book.civicrm.org/user

You can do a mass delete from the search screen. Its one of the actions. However its not optimized and does the delete one contact at a time

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

chp

  • I’m new here
  • *
  • Posts: 20
  • Karma: 0
  • CiviCRM version: 4.1
  • CMS version: Drupal
  • MySQL version: unk
  • PHP version: unk
Re: De-duping help for a newbie
May 23, 2012, 05:21:23 am
Thanks again, Lobo.  I'll try that out.  I was looking for delete and hadn't thought to look through the search section of the guide to find that.  I must say, though, from the perspective of someone with a lot of history with fundraising/contact management software, that while CiviCRM seems quite robust, batch processing such as deleting seems like a very basic function that's missing.  I assume there's a place to make that feature request, but is there a reason it hasn't been included?

Lastly, I tried a search with a wildcard (%) in the last name field but got no records found.  Any wisdom about why that would be?  No other search criteria other than the group the records are in.

Carl

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Core CiviCRM Functions (Moderator: Yashodha Chaku) »
  • De-duping help for a newbie

This forum was archived on 2017-11-26.