CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site
This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Core CiviCRM Functions (Moderator: Yashodha Chaku) »
  • Dedup Running Forever
Pages: [1]

Author Topic: Dedup Running Forever  (Read 848 times)

Hershel

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4640
  • Karma: 176
    • CiviHosting
  • CiviCRM version: Latest
  • CMS version: Mostly WordPress and Drupal
Dedup Running Forever
January 23, 2012, 01:08:14 pm
On a D7 D4.0.8 site with ~70,785 contact records, when we run dedup on the whole set, we see

Code: [Select]
INSERT INTO dedupe( id1, id2, weight ) SELECT t1.id id1, t2.id id2, 5 weight FROM civicrm_contact t1
running for 76 minutes on a VPS with 2G RAM and not yet finishing. Anyone ever seen anything like this? Or have any ideas how to get around this?
CiviHosting and CiviOnline -- The CiviCRM hosting experts, since 2007

See here for the official: What to do if you think you've found a bug.

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: Dedup Running Forever
January 23, 2012, 02:36:03 pm

what version? One potential workaround is to invoke dedupe on a much smaller set of contacts (i.e. limit it to a group)

4.1 has made a lot more improvements to dedupe performance (thx to NYSS). Wanna upgrade the site and see how it does on 4.1

lobo
A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Hershel

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4640
  • Karma: 176
    • CiviHosting
  • CiviCRM version: Latest
  • CMS version: Mostly WordPress and Drupal
Re: Dedup Running Forever
January 24, 2012, 05:00:56 am
I wrote "D7 D4.0.8" but I meant to write "D7 C4.0.8"

Anyhow, thank you for the idea to upgrade. The initial upgrade erred on:

Code: [Select]
ALTER TABLE `civicrm_contact` CHANGE `contact_sub_type` `contact_sub_type` VARCHAR( 255 ) CHARACTER SET utf8 COLLATE utf8_unicode_ci NULL DEFAULT NULL COMMENT 'May be used to over-ride contact view and edit templates.'
 [nativecode=1205 ** Lock wait timeout exceeded; try restarting transaction]
[to_string] => [db_error: message="DB Error: unknown error" code=-1 mode=callback callback=CRM_Core_Error::handle prefix="" info="ALTER TABLE `civicrm_contact` CHANGE `contact_sub_type` `contact_sub_type` VARCHAR( 255 ) CHARACTER SET utf8 COLLATE utf8_unicode_ci NULL DEFAULT NULL COMMENT 'May be used to over-ride contact view and edit templates.'
[nativecode=1205 ** Lock wait timeout exceeded; try restarting transaction]"]
So we increased the innodb_lock_wait_timeout value to 600 seconds, restored the DB, ran the upgrade successfully and the initial dedupe tests now run to completion, with no errors!

Thank you very much. :)
CiviHosting and CiviOnline -- The CiviCRM hosting experts, since 2007

See here for the official: What to do if you think you've found a bug.
Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Core CiviCRM Functions (Moderator: Yashodha Chaku) »
  • Dedup Running Forever

This forum was archived on 2017-11-26.