CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using CiviMail (Moderator: Piotr Szotkowski) »
  • [SOLVED] CiviMail cron timeout and bounce processing fails after server upgrade
Pages: [1]

Author Topic: [SOLVED] CiviMail cron timeout and bounce processing fails after server upgrade  (Read 2830 times)

bmw

  • I post occasionally
  • **
  • Posts: 103
  • Karma: 4
    • Alcohol Justice - The Industry Watchdog
  • CiviCRM version: 4.5.8
  • CMS version: Joomla! 3.4.0
  • MySQL version: 5.5.42-cli
  • PHP version: 5.3.27
[SOLVED] CiviMail cron timeout and bounce processing fails after server upgrade
November 07, 2013, 10:59:00 am
After a WHM and Litespeed server (http://litespeedtech.com) upgrade by webhost I'm getting some broken processes and errors. They are not sure what variables are needing to be adjusted to fix, and neither do I, directly, but I research and experiment until I get a desired result and not break security best practices.

VPS is running Linux version 2.6.32-042stab078.27 (root@rh6-build-x64) (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4)

First, civimail.cronjob.php would immediately abort on cronjob and commandline. In browser it would timeout in 60 seconds. I finally increased the Litespeed server Tuning settings "Connection Timeout" to 1200 seconds (20 mins). That seems to be excessive to have the connection open for that long. I am trying to find a way to just set that on a process by process basis. Please respond if you have any salient advice.

Second, EmailProcessor.php breaks with this error. I have scoured the forums, search engines and PHP resources to no real avail. I know that this is just a PHP setting to increase the length but can't find the variable. Please offer some advice on this. Thank you in advance.

Code: [Select]
Fatal error: Uncaught exception 'ezcMailTransportException' with message 'An error occured while sending or receiving mail. The IMAP server could not list messages: * BAD maximum line length exceeded.' in /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/packages/ezc/Mail/src/transports/imap/imap_transport.php:1008 Stack trace: #0 /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/packages/ezc/Mail/src/transports/imap/imap_transport.php(1405): ezcMailImapTransport->listMessages() #1 /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/CRM/Mailing/MailStore.php(114): ezcMailImapTransport->listUniqueIdentifiers() #2 /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/bin/EmailProcessor.php(172): CRM_Mailing_MailStore->fetchNext(50) #3 /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/bin/EmailProcessor.php(122): EmailProcessor::_process(true, Object(CRM_Core_DAO_MailSettings)) #4 /home/alcohol/public_html/administrator/compone in /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/packages/ezc/Mail/src/transports/imap/imap_transport.php on line 1008

I don't see a variable for MAX_LINE_LENGTH or MAX_VALUE_LENGTH in php.ini. Could this be entered in .htaccess? Would that work?

UPDATE: I think I narrowed it down to Client Command Line Length Limit. But, why all of a sudden is this being triggered? Is there some adjustment in CiviCRM to override?

Thanks in advance, all.
« Last Edit: November 14, 2013, 06:34:50 pm by bmw »
Bruce Wolfe, M.S.W., CIO
Alcohol Justice, 501(c)3

bmw

  • I post occasionally
  • **
  • Posts: 103
  • Karma: 4
    • Alcohol Justice - The Industry Watchdog
  • CiviCRM version: 4.5.8
  • CMS version: Joomla! 3.4.0
  • MySQL version: 5.5.42-cli
  • PHP version: 5.3.27
Re: CiviMail timeout and bounce processing fails after server upgrade
November 09, 2013, 01:28:40 am
Bump.
Bruce Wolfe, M.S.W., CIO
Alcohol Justice, 501(c)3

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: CiviMail timeout and bounce processing fails after server upgrade
November 09, 2013, 02:13:37 am
Hi,

EmailProcessor.php? it's quite an "old way" of processing bounces. Check out the documentation, you will need to do it the "new way" (with the job system)

Otherwise, not sure if it works with Joomla, but try running on the shell, it avoids all the timeout problems

X+
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

bmw

  • I post occasionally
  • **
  • Posts: 103
  • Karma: 4
    • Alcohol Justice - The Industry Watchdog
  • CiviCRM version: 4.5.8
  • CMS version: Joomla! 3.4.0
  • MySQL version: 5.5.42-cli
  • PHP version: 5.3.27
Re: CiviMail timeout and bounce processing fails after server upgrade
November 09, 2013, 11:15:14 am
Thanks Xavier. I reentered the crontab using the instructions at http://en.flossmanuals.net/CiviCRM/SystemConfiguration/
Again, this is a Joomla installation.

The book is not specific as to which user login/password to use.
Also, is the -sdefault variable needed for Joomla? If so, which domain, of the website or the server?
I've tried all combinations. Does this need to be a root user?


The cron is firing properly but returns the error:
Code: [Select]
X-Powered-By: PHP/5.3.13
Set-Cookie: PHPSESSID=598e2f342632ed3da832cd23f7898831; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-type: text/html

ERROR: You need to send a valid user name and password to execute this file
Bruce Wolfe, M.S.W., CIO
Alcohol Justice, 501(c)3

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: CiviMail timeout and bounce processing fails after server upgrade
November 10, 2013, 02:43:45 am
Hi,

I'm afraid I don't know Joomla enough to be able to help you there. Let's see if someone else can come to the rescue.
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

bmw

  • I post occasionally
  • **
  • Posts: 103
  • Karma: 4
    • Alcohol Justice - The Industry Watchdog
  • CiviCRM version: 4.5.8
  • CMS version: Joomla! 3.4.0
  • MySQL version: 5.5.42-cli
  • PHP version: 5.3.27
Re: CiviMail timeout and bounce processing fails after server upgrade
November 11, 2013, 03:43:06 pm
This is not so much a Joomla issue as it is a communication issue between CiviCRM and IMAP. My IMAP service is Postfix. I have increased all message lengths to no avail.

Anyone have this same issue before? Anyone?
Bruce Wolfe, M.S.W., CIO
Alcohol Justice, 501(c)3

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: CiviMail timeout and bounce processing fails after server upgrade
November 11, 2013, 11:19:01 pm
Quote from: bmw on November 11, 2013, 03:43:06 pm
This is not so much a Joomla issue as it is a communication issue between CiviCRM and IMAP.

Code: [Select]
ERROR: You need to send a valid user name and password to execute this file
This look as an error message from joomla/civi. ie. before it talks to imap

Quote from: bmw on November 11, 2013, 03:43:06 pm
My IMAP service is Postfix. I have increased all message lengths to no avail.

Postfix isn't an IMAP server. you have probably have an imap server as well as postfix (dovecot is a popular one)
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

bmw

  • I post occasionally
  • **
  • Posts: 103
  • Karma: 4
    • Alcohol Justice - The Industry Watchdog
  • CiviCRM version: 4.5.8
  • CMS version: Joomla! 3.4.0
  • MySQL version: 5.5.42-cli
  • PHP version: 5.3.27
Re: CiviMail timeout and bounce processing fails after server upgrade
November 12, 2013, 12:12:03 pm
Xavier,
You mentioned in an earlier post that using EmailProcessor.php is not the new way to set up the cron to fire bounce handling but didn't provide a resource. I responded with a solution but I am not sure this is what you were referring to. Would you be so kind as to provide the process name or a link to what you were referring to? Thank you in advance. To your last response:

Quote from: xavier on November 11, 2013, 11:19:01 pm
Quote from: bmw on November 11, 2013, 03:43:06 pm
This is not so much a Joomla issue as it is a communication issue between CiviCRM and IMAP.

Code: [Select]
ERROR: You need to send a valid user name and password to execute this file
This look as an error message from joomla/civi. ie. before it talks to imap

Actually, this is not an error any longer. There was a typo.
Quote
Quote from: bmw on November 11, 2013, 03:43:06 pm
My IMAP service is Postfix. I have increased all message lengths to no avail.

Postfix isn't an IMAP server. you have probably have an imap server as well as postfix (dovecot is a popular one)

Yes, yes...I stand corrected. It was a very long day. I meant to say Courier. Postfix is what Zimbra is using as an MTA which I was reviewing through this issue.

I am still having this error that I am still unable to fix. I have contacted Zimbra (our email server) tech support and awaiting an answer but if anyone has any insight on to what is causing or how to fix this error message it would greatly be appreciated. I removed the pertinent access info in the cronjob for this posting.


Code: [Select]
[root@host ~]# /usr/bin/wget -O - -q 1 'http://alcoholjustice.org/administrator/components/com_civicrm/civicrm/bin/EmailProcessor.php?name=<Joomla username>&pass=<password>&key=<public_key>'

Fatal error: Uncaught exception 'ezcMailTransportException' with message 'An error occured while sending or receiving mail. The IMAP server could not list messages: * BAD maximum line length exceeded.' in /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/packages/ezc/Mail/src/transports/imap/imap_transport.php:1008

Stack trace:
#0 /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/packages/ezc/Mail/src/transports/imap/imap_transport.php(1405): ezcMailImapTransport->listMessages()
#1 /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/CRM/Mailing/MailStore.php(114): ezcMailImapTransport->listUniqueIdentifiers()
#2 /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/bin/EmailProcessor.php(172): CRM_Mailing_MailStore->fetchNext(50)
#3 /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/bin/EmailProcessor.php(122): EmailProcessor::_process(true, Object(CRM_Core_DAO_MailSettings))
#4 /home/alcohol/public_html/administrator/compone in /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/packages/ezc/Mail/src/transports/imap/imap_transport.php on line 1008
Bruce Wolfe, M.S.W., CIO
Alcohol Justice, 501(c)3

xavier

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4453
  • Karma: 161
    • Tech To The People
  • CiviCRM version: yes probably
  • CMS version: drupal
Re: CiviMail timeout and bounce processing fails after server upgrade
November 13, 2013, 03:24:06 am
http://wiki.civicrm.org/confluence/display/CRMDOC/CiviMail+Installation#CiviMailInstallation-SettingUptheCronJob

Never tried on J!, but the wiki is the most up to date one.

X+
-Hackathon and data journalism about the European parliament 24-26 jan. Watch out the result

bmw

  • I post occasionally
  • **
  • Posts: 103
  • Karma: 4
    • Alcohol Justice - The Industry Watchdog
  • CiviCRM version: 4.5.8
  • CMS version: Joomla! 3.4.0
  • MySQL version: 5.5.42-cli
  • PHP version: 5.3.27
[SOLVED] Re: CiviMail timeout and bounce processing fails after server upgrade
November 14, 2013, 06:32:04 pm
Okay, I found the problem and a correction for the documentation.

It appears that the error message coded in this file:
Fatal error: Uncaught exception 'ezcMailTransportException' with message 'An error occured while sending or receiving mail. The IMAP server could not list messages: * BAD maximum line length exceeded.' in /home/alcohol/public_html/administrator/components/com_civicrm/civicrm/packages/ezc/Mail/src/transports/imap/imap_transport.php:1008

Pertains to the IMAP process retrieving the list of messages in the bounce email account. While the variable $response is not contained "maximum line length exceeded" refers to something in an actual email message headers. I can't tell if it is the subject line or some header created during transport from CiviCRM to SMTP remailer to mail server. I will be experimenting to find out. I'm sure that once I do, I bet some PHP setting will solve it all. But, this is the "problem".

Now, the correction: The documentation for building the cronjob using the command line method will not work under RedHat/PHP 5.3.x/cPanel using the options '-u', '-p' and '-s'. It must be 'name=', 'pass=' ('site=' not required). But, in EmailProcessor.php it calls for the options '-u', '-p' and '-s' that I can see. Did I miss something somewhere?

In any case, the docs should be updated to reflect this in case our new burgeoning installers run into trouble.
Note, that the webhost nor remailer nor mail server tech support had any answers, and for all intents and purposes, neither did CiviCRM. Makes it really hard for us. This was two solid days of labor.

We must share as much of these problems and/or solutions as possible in the community all the across to the webhost and SMTP remailers, too. We can do better at indexing these kinds of issues.
Bruce Wolfe, M.S.W., CIO
Alcohol Justice, 501(c)3

Donald Lobo

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 15963
  • Karma: 470
    • CiviCRM site
  • CiviCRM version: 4.2+
  • CMS version: Drupal 7, Joomla 2.5+
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: [SOLVED] CiviMail cron timeout and bounce processing fails after server upgrade
November 14, 2013, 07:02:28 pm

hey bruce:

thanx for posting back your findings, definitely useful as other folks might hit this issue

note that the wiki is community editable, so please do update the documentation as needed. Note that the name= / pass= etc are typically used when stuff is invoke via the web server (via curl or the browser) and the -u -p etc are used on command line. Note that for email processing / civimail delivery we recommend using cli to avoid the webserver overhead (its a lot faster and uses less memory)

lobo


lobo

A new CiviCRM Q&A resource needs YOUR help to get started. Visit our StackExchange proposed site, sign up and vote on 5 questions

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using CiviMail (Moderator: Piotr Szotkowski) »
  • [SOLVED] CiviMail cron timeout and bounce processing fails after server upgrade

This forum was archived on 2017-11-26.