out of memory error at mail Upload step

[chrism]

I have installed CiviCRM 1.7.9379 and am attempted to send a mailing, but after the selecting "Next" at the Upload step I get the following error:

Fatal error: out of dynamic memory in yy_create_buffer() in /home/sites/all/modules/civicrm/packages/Smarty/Smarty.class.php on line 1453

Warning: MySQL client ran out of memory query: SELECT * FROM system WHERE type = 'theme' in /home/includes/database.mysql.inc on line 172

I had a 1.6 version of CiviMail working on the same host previously, and I have an custom install of PHP with memory_limit set to 90M.  The group just consists of a small number of contacts.  Any suggestions as to the cause of this and how to resolve? 

Much appreciated,
Chris

[Piotr Szotkowski]

This is very strange. Can you get access to your MySQL logs? Note that this is an error on a Drupal query…

[Donald Lobo]

Chris:

we've not seen this error in the past, so are a bit lost here. can u get on IRC and we can chat and hopefully make more progress on it

lobo

[chrism]

Thanks Piotr, Lobo,

I investigated a bit more, and discovered that the Upload step failure appears when the number of contacts in the civicrm database increases.  With 800 contacts (7.0MB database) it worked fine, but with 1000 contacts (7.5 MB database) the error message appeared. 

The second time I got a different message:

Fatal error: Out of memory (allocated 31457280) (tried to allocate 40961 bytes) in /home/sites/all/modules/civicrm/packages/Smarty/Smarty.class.php on line 1261

and other times I just got a blank screen, so I think this is a general out-of-memory problem on the server rather than a specific line of code problem.

I inquired into memory limits and SQL logs with my host and got this response:

There are several limits in place that you may be hitting. We have an
apache memory limit, a php memory limit, and a few others, but I think
those two are what's causing the issue.

Both limits are set at 90 megs, but they're a total limit, so any sum of
memory you're using, will get added up. Once you hit that, you'll be
throttled. You'll need to reduce your memory consumption in order to
proceed.

Unfortunately, we do not provide access to mysql logs.

It is surprising to me that a memory error on the Upload step should be dependant on the total size of the database, since the mailing group that I was sending to in all cases had only one member (i.e. shouldn't be a big query).

Chris

[Donald Lobo]

Chris:

i took a look at the queries we generate during the upload postprocess and dont see any queries that are a function of the database. Obviously your experiments below indicate that we seem to be hitting some bug that consumes a fair amount of memory

Not having access to your mysql logs makes things a bit more difficult to diagnose and debug. I'm not sure what we can do at this stage. would be cool to get ssh access to your box so we can instrument the code and see what queries we are
generating and trace the flow

if this is possible, please get on IRC and we can discuss details / get access etc

lobo

[chrism]

An issue has been filed: http://issues.civicrm.org/jira/browse/CRM-1989