imap error

[geraldr_sc]

We are using google apps and I started receiving the following error when connecting using CiviMailProcessor.php:

connecting to imap.gmail.com, authenticating as civimail@domain.org and selecting INBOX
Fatal error: Uncaught exception 'ezcMailTransportException' with message 'An error occured while sending or receiving mail. The IMAP server did not accept the username and/or password: * CAPABILITY IMAP4rev1 UNSELECT LITERAL+ IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE.' in /...[deleted].../public_html/administrator/components/com_civicrm/civicrm/packages/ezc/Mail/src/transports/imap/imap_transport.php:515 Stack trace: #0 /...[deleted].../public_html/administrator/components/com_civicrm/civicrm/CRM/Mailing/MailStore/Imap.php(61): ezcMailImapTransport->authenticate('civimailuser@...', 'password') #1 /...[deleted].../public_html/administrator/components/com_civicrm/civicrm/CRM/Mailing/MailStore.php(61): CRM_Mailing_MailStore_Imap->__construct('imap.gmail.com', 'civimailuser@...', 'password', true, NULL) #2 /...[deleted].../public_html/administrator/components/com_civicrm/civicrm/bin/CiviMailProcessor.php(91): CRM_Mailing_MailStore::getStore(NULL) #3 /...[deleted].../public_html/administrator/compone in /home/carolina/public_html/administrator/components/com_civicrm/civicrm/packages/ezc/Mail/src/transports/imap/imap_transport.php on line 515

Any ideas about what is causing this?

I am able to connect to the google apps account using thunderbird with the same account settings without difficulty.

[pdowling]

Just to add.  I'm getting exactly the same error.

Yesterday I was getting a different error that turned out to be port 993 not being open from my host.  That is fixed.  Now I get same exact error as this.  I've definitely checked and rechecked the credentials and info I'm supplying. When I change them I get different errors that make it obvious that they are wrong.

Hopefully there's another gmail person who received and figured out this error?

[pdowling]

geraldr_sc,

You say you "started receiving" the following error.

Was there a time when it was working correctly to process the return channel and now it suddenly isn't?

That would imply either a bug was introduced to civi, or gmail did something to change the way they respond to IMAP.

[geraldr_sc]

Yes. It was working a week ago. I suspected the host or gmail made changes, but I may have inadvertently broken something. I am just not sure how to check. I inserted var_dump($xxx) at various places to check variables, and everything seemed to be correct. My web host is checking the firewall or other settings to see if there is something interfering.

The responseType function is returning a 4, which appears to be RESPONSE_UNTAGGED (line 2521) returning value to line 513. It returns anything that starts with '* ' after falling through tests for the strings 'OK', 'NO', or 'BAD'. Perhaps gmail has changed the return string, which is now '* CAPABILITY IMAP4rev1 UNSELECT LITERAL+ IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS COMPRESS=DEFLATE.' Or that could be the return when the login fails. I don't know enough about IMAP to know what is happening.

[geraldr_sc]

I got it to work with a temporary change in imap_transport.php, line 2521.  I changed this:

protected function responseType( $line)

to this:

protected function responseType( $line, $server='imap.gmail.com' )

I then added the following on line 2537, just before the line 'return self::RESPONSE_UNTAGGED;' in the if clause checking for '* '.
   if ( ($server)=='imap.gmail.com' )
   {
       return self::RESPONSE_OK;
   }
   
So the problem is perhaps that Google apps changed their IMAP response which broke the civimail code that was expecting '* OK'. But it now logs in and processes emails.

[pdowling]

You beat me to the punch!!

I was just going to say it sounded like it was working (and I had even found an example in java showing the return value from gmail with an OK response from a year ago) and just wasn't being reported as working by the component civi is using for imap.

So the next step is to actually debug exactly what are we getting back from gmail so we can add a test for that where the imap server has "gmail" in the server text.

It isn't good enough of course to just force it to OK every time as what happens when it really does fail!

Unfortunately I won't be in a position to look further into this till probably Sunday, but feels like this is now a fairly trivial fix.

Boy, awful annoying if gmail messed with the standard imap response to the point where a heavily used php imap library failed!

[yogibear]

+ 1

Go code warriors

[pdowling]

Well to follow up I did a couple of trivial debugging steps on my live site (didn't want to do too much there!).  I'll have to get my site running locally to really get dirty with debugging and probably won't be able to do that till tomorrow or later this week.

Anyway I put a

Code: [Select]

var_export($response); in the getResponse function at line 2595 of imap_transport.php file just for kicks.

The odd thing?  It returned...

"A0001 OK username@example.org authenticated (Success)"

which suggests that the debug message coming back of "The IMAP server did not accept the username and/or password" doesn't really make much sense.

Anyway, I'll have to get it running locally and really trace it all the way through to see where it is choking.

[demeritcowboy]

Getting the same thing. Supposedly Google made an update on Nov 3 to try to become more compliant with the spec, but it seems to have broken some clients.
Other clients are getting a different error, but it must be related to the login problem too.

http://www.google.com/support/forum/p/gmail/thread?tid=667b59b9e5b94c35&hl=en

[greenmachine]

I've confirmed that this error has now popped up for at least three client sites that are running 2.2x. CiviMailProcessor.php was working fine previously for all three, now reporting the same php error as posted above.

I tried the current version of EZ Components, replacing imap_transport.php with the current EZC imap_transport.php (looks like version 1.6 vs. 1.6.3). However, that does not fix the problem. I didn't see a bug for this in the EZ Components bug tracker, so I created one: http://issues.ez.no/IssueView.php?Id=15837&activeItem=1

[derick]

Hello,

This is now fixed in eZ Components' SVN version from which a beta will be released next Monday.

Derick

[greenmachine]

Thanks for taking the time to post over here, Derick, and for working on a fix.

[pdowling]

Yes, thanks so much Derick.

So just for fun I tried adding exactly the lines in Derick's eZ Components patch, but it failed.  I'm guessing because his patch assumes other code that the civiCRM version of the eZ Mail component doesn't have yet.  The specific error on my side was "No such property name 'serverType'".

Anyway, if the beta which includes this fix perhaps I'll just include that.

How would one tell which version of eZ Components civicrm is currently using?  I couldn't find a readme or changelog or anything in the "packages/ezc/" folder or below.  Some of the files say that it is mail package version 1.6.  So I downloaded the latest unstable eZ Components files.  The changelog there says 1.7.  I wonder what was the reason for removing the changelog for this component from the civicrm folder?

[pdowling]

OK, I posted this as a bug here...

http://issues.civicrm.org/jira/browse/CRM-5431

... with a suggestion that one fix could be to substitute the latest ezc component update beta to be released tomorrow.

[mfigart]

I replaced the ezc components (autoload, base and mail) in my CiviCRM 3.0.2 installation with those from the Nov. 23 ezc beta, as referenced above.  I'm still not getting the imap to function correctly, as it did before the Gmail (Google Apps, in my case) change.  Just thought I should post this. 

By the way, I cleared cache on my Drupal site, and am unaware of anything else I should try to ensure the new code is executing.  If I'm missing something, I'd love to know.

Thanks!

Mark Figart
Digett