CiviCRM Community Forums (archive)

I'm at my wits' end, and I'm hoping someone can help.

I have a brand new, out of the box CiviCRM 4.4.5 install hosted in Wordpress 3.9. I'm getting ready to import contacts from different data sources in. So far, I've been able to successfully import and map custom fields against a data set that's roughly 250 columns wide but I have another data set that's roughly 350 columns wide that generates a white screen even if I reduce the import down to a single row under the column headers. The URL it bombs out at is:

.../wp-admin/admin.php?page=CiviCRM&q=civicrm/import/contact&_qf_MapField_display=true&qfKey=db9ab82800c2719f416e6298fdcda0b4_2339

This happens near-immediately if I just make it a two row source (column headers + 1 row of data), so I don't think I'm timing out on anything.

I confirmed that the data source, a CSV file, is UTF-8 encoded. I've also tried changing the line breaks (file is just CR; tried CR+LF, no effect). I'm able to import the CSV using PHPMyAdmin into a table in MySQL, so I know the server is able to read the data without issue. I've also tried clearing out quotation marks from the column headers; this had no effect. I've also tried importing the data using a SQL query against the table I imported via PHPMyAdmin (e.g. "SELECT * FROM mycivi_database.importedTable;") - this also gives me the White Screen of Death.

The really frustrating part is that, even if I enable CiviCRM debugging, even if I enable PHP debugging, even if I enable MySQL debugging, even if I enable Apache LogError debug, I'm not getting any errors, warnings, messages, or anything else indicating what's going on.

The good news is I have root on the box it's being hosted on (CentOS 6.5, Apache 2.2, PHP 5.3.26, MySQL 5.5.36 Community), so I've been able to play with some configuration options. Here's what I have so far:

PHP.ini Settings in VirtualHost Directory
upload_max_filesize=256M
post_max_size=256M
display_errors=On
display_startup_errors=On
html_errors=On
log_errors=On
error_reporting=E_ALL
memory_limit=2048M
max_execution_time=30000
max_input_vars=4000
realpath_cache_size=2M

Current MySQL Settings

max_allowed_packet	1073741824
max_binlog_cache_size	18446744073709547520
max_binlog_size	1073741824
max_binlog_stmt_cache_size	18446744073709547520
max_connect_errors	10
max_connections	151
max_delayed_threads	20
max_error_count	64
max_heap_table_size	33554432
max_insert_delayed_threads	20
max_join_size	1000000000
max_length_for_sort_data	1024
max_long_data_size	1073741824
max_prepared_stmt_count	16382
max_relay_log_size	0
max_seeks_for_key	18446744073709551615
max_sort_length	1024
max_sp_recursion_depth	0
max_tmp_tables	32
max_user_connections	50
max_write_lock_count	18446744073709551615

Apache, meanwhile, is set with a Timeout of 300.

Anyway, I'm running out of possible reasons why the Import page might be bombing out. One idea I had was that maybe the query was too long, but when I enabled Slow Query logging in MySQL and turned max_packet_size to the maximum (1GB), the longest query (UPDATE civicrm_cache SET group_name = 'CiviCRM Session', path='CiviCRM_CRM_Contact_Import_randomtext, data=Wall of text!; ) completed successfully and there were additional CiviCRM queries completed after it.

So, what's going on? Is there a way for me to get this stuff in without splitting the file in half lengthwise and hoping deduplication fills in the remaining columns in the "duplicate" entries? Do I need to spend the next couple of days reading up on the API (I'd rather not if it can be helped)? Is there some server setting I should be trying somewhere that I'm not aware of?

According to civicrm.settings.php:

* EXAMPLE - WordPress Installations:
 * If the path to the WordPress home directory is /var/www/htdocs/wordpress
 * the $civicrm_root setting would be:
 *      $civicrm_root = '/var/www/htdocs/wordpress/wp-content/plugins/civicrm/civicrm/';
 *
 * the CIVICRM_TEMPLATE_COMPILEDIR would be:
 *      define( 'CIVICRM_TEMPLATE_COMPILEDIR', '/var/www/htdocs/wordpress/wp-content/plugins/files/civicrm/templates_c/' );
The good news is the ConfigAndLog directory is indeed in the same area. The bad news is, there's nothing in there, even after I enabled debug and backtracing just now and re-ran the import that keeps bombing out on me.

As for Apache, this particular server has CPanel and WHM installed, which is configured to log to /usr/local/apache/logs - I'm not seeing any errors, even with LogLevel=debug. I do see some debug-related messages so I know nothing's overriding that setting anywhere, but if I have a tail -f going on error_log while I'm performing the import, nothing new shows up on screen.

At this point, I think I'm being more stubborn than anything - I mean, something has to be causing it to error out, and I should be able to find out what somewhere, right? So where's the bottleneck?

In the meantime, I'm definitely going to have to break this data down into thinner chunks. Whether I use the CLI or the Import screen, that's going to be a necessity either way, it seems. It's either that or roll my own VM and try to SQL dump everything in or something.

a wsod in most cases means:

a. PHP/Apache is aborting

c. PHP is running into a fatal error (memory limits etc)

with 350 fields, i suspect we are hitting the POST limit maybe of apache?

lobo

Hi Linkx,

I'm pretty sure I recall this being a problem that was fixed somewhere in the 4.3 series.  You don't have your CiviCRM version listed on your forum profile - could you please tell us what version you're usng?