CiviCRM Community Forums (archive)

This topic has come up before (http://forum.civicrm.org/index.php/topic,24704.0.html)  and no resolution was posted, I am hoping we can work to draft a solution as I know the community would benefit.

A bit of background on my system:
I have root access to a dedicated server with:

• 16GB RAM
• 8 XEON Processor Cores
• Apache/2.2.25 - running in FastCGI mode
• MySQL 5.5.32-cll
• PHP v. 5.3.27 (cli)

I am running Drupal v7.23 and CiviCRM 4.3.5 on a new system. No additional contrib modules installed.

• For testing purposes I have CiviCRM debug mode turned on
• I have enabled Drupal watchdog logging
• I have display backtrace enabled

Now for the details of my issue:

Every time I attempt to import more than approximately 750 contacts I am able to complete step 1 (telling it to import) and step 2 (mapping fields). After mapping fields I hit continue and my system "thinks" for about 10 to 15 seconds after which I am redirected back to step 1 like I was starting the process from the beginning.

No messages what-so-ever are written to the:

• - Drupal Log
• - /var/log/messages
• - the MySQL logs
• - /var/log/httpd/error_log

I mean it's like the transaction is not even happening.

I see the MySQL buffers fill (via phpMyAdmin's tools) but nothing is written to the logs.

It HAS to be a memory issue, well, that's my thinking. Like a buffer pool is filling up and when it reaches its max value it just dies, but I can't ID the process. This is KILLING ME!!!! I have a sneeking suspicion that I am missing some basic parameter, but I can't figure it out.

Can someone please provide me with some sizing/tuning advice?


I have included some (what I think are relevant) system settings. 

Relevant Apache Settings (I think)
RewriteEngine on
Timeout 300
<IfModule mod_fcgid.c>
    FcgidMaxRequestLen 1073741824
</IfModule>


Relevant PHP Settings (let me know if I am missing any)
max_execution_time = 600
max_input_time = 600
memory_limit = 2048M
upload_max_filesize = 1024M
default_socket_timeout = 60
mysql.connect_timeout = 60

MySQL Settings (I threw everything but the kitchen sink at MySQL):

• innodb_file_per_table = 1
• default-storage-engine = MyISAM
• general_log
• slow_query_log
• key_buffer_size = 64M
• query_alloc_block_size = 131072
• wait_timeout = 3600
• tmp_table_size = 512M
• join_buffer_size = 64M
• query_cache_limit = 64M
• max_user_connections = 100
• sort_buffer_size = 32M
• thread_cache_size = 32
• table_cache = 8193
• query_cache_size = 128M
• query_cache_type = 1
• max_connections = 100
• thread_stack = 256K
• query_prealloc_size = 65536
• long_query_time = 5
• read_buffer_size = 2M
• max_allowed_packet= 512M
• bulk_insert_buffer_size = 32M
• myisam_sort_buffer_size = 64M
• connect_timeout=8
• log-queries-not-using-indexes
• open_files_limit=38004
• log_slow_queries = yes
• max_heap_table_size = 128M
• table_open_cache = 16368
• open_files_limit=38572

I LOVE CiviCRM and would like the ability to leverage the import tool and not have to run it from the command line (I need to train users on importing via the UI).

I am praying someone out there can see the trees through the forest and kick me in the right direction.

Thanks in advance for any insight.

My PHP handler options are:
- suphp
- dso
- fcgi
- cgi
- none

I am running fcgi for APC caching support.... I think I can run in dso, which is mod_php mode, and still get support for APC, which I am using for caching....

cool, glad that worked and thx for reporting back

lobo