Author Topic: Mail becomes 7bit in transit (Read 2953 times)

Chris Burgess · May 18, 2008, 10:52:35 pm

CiviMails sent to a particular set of domains seem to be getting converted to 7bit in transit. This is resulting in them exceeding the line-length limitation in RFC2821 (4.5.3.1).

Mails sent to a couple of test accounts finish their travels with a boundary marker that looks like this:

Code: [Select]

--=_87b0a9a76b3fc3d8f41ef8279631eca0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit

I've tested this with a LOT of domains, and it's pretty consistent - works on Gmail, Hotmail, Yahoo!, works on various ISP accounts I have access to.

However, mails sent to addresses @clear.net.nz or @paradise.net.nz arrive with this boundary marker instead, and with the messages truncated at 1000 characters. (This happens because Dōjō's HTML editor widget doesn't insert any linefeeds, which is perfectly fine behaviour for an HTML editor.)

Code: [Select]

--Boundary_(ID_YIeu1ryfOv9MRWLiaU1ALw)
Content-type: text/html; charset=utf-8
Content-transfer-encoding: 7BIT

These messages also contain a header inserted, which reads

Code: [Select]

Sun-Java-System-SMTP-Warning: Lines longer than SMTP allows found and truncated.

Would I be right in guessing that the latter set of emails have been through some kind of mail scanner which has put them back together wrong?

If this ISP insists on converting all mails to 7BIT, what are my options? (For example: If I sent to them in 7bit anyway, maybe they wouldn't munge it on the way through?)

Chris Burgess · May 18, 2008, 11:23:48 pm

However, I'm open to the possibility that this ISP is closer to the RFC than others.

Here's what I see arrive in Gmail - HTML portion only of the mail source shown below:

Code: [Select]

--=_87b0a9a76b3fc3d8f41ef8279631eca0
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit

This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />And that's a thousand and ninety-nine.

(The above all on one line.)

Here's what I see via Clear / Paradise:

Code: [Select]

--Boundary_(ID_YIeu1ryfOv9MRWLiaU1ALw)
Content-type: text/html; charset=utf-8
Content-transfer-encoding: 7BIT

This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my test text line. It has fifty-six letters.<br />This is my t

If the Gmail version is really in 8bit, shouldn't it be broken into shorter octets?

Chris Burgess · May 19, 2008, 01:15:49 am

I altered CRM/Mailing/BAO/Mailing.php @ line 977 to make the HTML component encoded in 7bit instead of 8bit. However, this caused no visible change in the source of the emails generated (either dumped to file or sent to a remote address).

This makes me wonder if CiviCRM is correctly encoding the message components - I'd appreciate it if someone could confirm that the attached mail is correctly MIME encoded, and if the various body parts are really 8bit encoded as labelled.

Thanks!

Piotr Szotkowski · May 19, 2008, 02:03:03 am

FWIW, you can try simply calling $html = wordwrap($html, 200) where applicable.

Quote from: xurizaemon on May 19, 2008, 01:15:49 am

I'd appreciate it if someone could confirm that the attached mail is correctly MIME encoded, and if the various body parts are really 8bit encoded as labelled.

My Mutt opens this email just fine. Note: it doesn’t contain any non-7-bit characters anyway…

Chris Burgess · May 19, 2008, 02:45:49 am

I thought one of the features of 8bit encoding was that line lengths weren't an issue, because it broke the message into tidy octets anyway?

From what I understand, Mail_Mime ought to be handling things like SMTP compatability for us anyway. Looking at the code in packages/Mail/mime.php, if the message component is 7bit, then it will enforce the setting from $params['7bit_wrap'] which breaks each line at 998 char (+CRLF=1000).

Not so for 8bit - which suggests to me that 8bit has some other means of avoiding messages getting stopped for being >1000char per line.

twowheeler · May 19, 2008, 04:07:36 am

Quote from: xurizaemon on May 19, 2008, 02:45:49 am

I thought one of the features of 8bit encoding was that line lengths weren't an issue, because it broke the message into tidy octets anyway?

No I don't think that's right. 8Bit still needs to be in lines of <1000 chars. The ISP is probably enforcing the rules more strictly than most. The easiest thing in the short run might be to make sure your line lengths stay under 1000 chars. Piotr's little hack should do it.

Quote

Content-Transfer-Encoding: 8bit

8bit encoding has the same line-length limitations as the 7bit encoding. It allows 8bit characters. No encoding or decoding is required for 8bit files. Since not all MTAs can handle 8bit data, the 8bit encoding is not a valid encoding mechanism for Internet mail.

http://msdn.microsoft.com/en-us/library/ms526290(EXCHG.10).aspx

Piotr Szotkowski · May 19, 2008, 05:16:52 am

Quote from: xurizaemon on May 19, 2008, 02:45:49 am

I thought one of the features of 8bit encoding was that line lengths weren't an issue, because it broke the message into tidy octets anyway?

No, 8-bit only means that 8-bit characters (i.e., ‘high-ASCII’) can be used. 7-bit means that the email (encoded in any way, say – with quoted-printable) is using only ‘low-ASCII’ characters – i.e., there isn’t any byte in the message with value > 127.

Quote from: xurizaemon on May 19, 2008, 02:45:49 am

From what I understand, Mail_Mime ought to be handling things like SMTP compatibility for us anyway.

Yeah, Mail_Mime should wrap any message it sends (be it 8-bit or 7-bit) to less-than-1000 bytes, but it seems it doesn’t.

Chris Burgess · May 19, 2008, 02:08:25 pm

Right. I think then (based on your info about 7bit etc) that their mailserver is just calling it 7bit because there aren't any 8bit chars, and that the original issue - excess line length, regardless of encoding - is what was causing the problem.

Thanks for clarifying.

CiviCRM Community Forums (archive)

News:

Author Topic: Mail becomes 7bit in transit (Read 2953 times)

Chris Burgess

Mail becomes 7bit in transit

Chris Burgess

Re: Mail becomes 7bit in transit

Chris Burgess

Re: Mail becomes 7bit in transit

Piotr Szotkowski

Re: Mail becomes 7bit in transit

Chris Burgess

Re: Mail becomes 7bit in transit

twowheeler

Re: Mail becomes 7bit in transit

Piotr Szotkowski

Re: Mail becomes 7bit in transit

Chris Burgess

Re: Mail becomes 7bit in transit