CiviCRM Community Forums (archive)

Hello,

     I'm running into a problem with Finding Duplicate Individuals and wanted to share my experience and get advice.

Configuration Details

Intel Dual Core P4 3Ghz CPU
1Gb DDR RAM

Fedora Core 6
Drupal 5.3
CiviCRM 1.9

Table sizes (rows)
civicrm_contact - 263,571
civicrm_individual - 240,795
civicrm_location - 250,096
civicrm_email - 247,382

The dedupe rule for Individual is
Email : 11
Threshold: 10

Now i tried to find duplicate individuals with the deduping feature and it broke down 4 times.

Fatal error: Maximum execution time of 30 seconds exceeded in /work/projects/testsites/civicrm1/www/sites/all/modules/civicrm/CRM/Dedupe/Finder.php on line 98
Fatal error: Maximum execution time of 1200 seconds exceeded in /work/projects/testsites/civicrm1/www/sites/all/modules/civicrm/CRM/Dedupe/Finder.php on line 98
Fatal error: Maximum execution time of 3600 seconds exceeded in /work/projects/testsites/civicrm1/www/sites/all/modules/civicrm/CRM/Dedupe/Finder.php on line 98

The above 3 were fixed by increasing the max_execution_time in php.ini

finally after increasing the time to 1 day it broke down for lack of memory
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 999700 bytes) in /work/projects/testsites/civicrm1/www/sites/all/modules/civicrm/CRM/Dedupe/Finder.php on line 98

the memory_limit is 128M.
Has anyone seen such problems before, is it advisable to bump up the memory from 128Mb to 256Mb ?
Any other ways to get this up and running ?

Thanks.

Yashesh Bhatia