CiviCRM Community Forums (archive)

I'm running the 386 initial bounces through again separately and while it is still running, out of the first 150, they all bounced.   I'm now going to send a regular email from my local email client to a few as a test.

Resending all initial bounces (before going out), resulted in all 386 again being rejected.  A couple of addresses I noticed were bad, but several looked fine from people I know.  I sent out a test message for 4 from my email client last night and 2 of the 4 have replied back that they received my test message.

My test message was not the same message as my newsletter, but they did go through to addresses that bounced (quick bounce) with our newsletter mailing.  I think, I'll now go back and send the exact same bulk email that quick bounced, with my email client. 

It does look like CiviMail may be a little quick in declaring a bounce instead of sending. 

I thought I'd also take the bulk email addresses and try sending it from Vertical Response and see if it goes through from there.

Looking back to when this issue started, it did not start gradually(as I indicated), but it is increasing from a burst start, perhaps because of new records being added:
August 6th - 0 initial bounces
August 13th - 343 initial bounces

My new theory is that perhaps I was trying to hookup IMAP2Soap and broke something and didn't notice it since 87% of the emails go out just fine.  Will retrace what I did with IMAP2Soap

Are any others having this issue with initial bounces.
= = = = = =

Well isn't this interesting:

I renamed the imap2soap.pl and .conf files so that they would not be found by Civimail and I'm rerunning the 384 bouncing contacts for the 3rd time.  While I've yet to confirm by phone the newsletter receipt, I'm fairly certain that many of them are now going through because of the tracked opens and click throughs that are now showing up already with the mailing in process.

While we can assume that I hooked up the imap2soap incorrectly, for me it was worse than non-functional - for about 13 % of our mailing, it stopped mailings from going to many good addresses.  This does not mean that some of the email addresses that "initially bounced" are not bad, but perhaps we can deal with them with the rest of our returned bounces.

Renewed interest in the bounce processing with PHP project.

Hope the above helps someone else.
Dave

In general, what you need to check is the reason behind these bounces (maybe the target servers are treating CiviMail as a spam source for some reason?), and this info is in the mailserver’s logs.

OK, new theory.  Previous examples and assumed resolution did happen as described above.  However, today while doing a mailing, back to premature bounces for one group of 48.

Looks like emails are sent out in order of internal contact ID.

We send out in batches of 50 every 10 minutes.

2 batches went fine + 2 apparently extras 102 went out fine.

Then 48 consecutive emails "bounced prematurely"

Then the next 4 batches of 50 have gone out fine and run of 2,427 is still running (50 every 10 minutes).

If an entire batch of 50 had bounced, I might suspect something like the next batch ran before the first completed.  However, the first 2 of the offending batch seemed to go out then the remaining 48 all failed. 

Wonder if there could be something wrong with a particular record and then when it is sent, if it kills off the entire batch.  Or, still might be a timing thing..... watching the run.

Dave

= = = = = = =

First record that "premature bounced" had a Bounce Type of Host and Bounce Reason of refused. In the current run with 48 bounces so far, no other bounce record has a known bounce type or a bounce reason.  First bounce record was #293

Looking back to run from a while back with 2399 including bounces of 385.  There were 6 records in the bounces with bounce type of host and Bounce Reason of Connection refused.  6 records x 50 per batch only gives 300 rather than 385, but in the ballpark.   

Last run had a connection refused at records 289, 770, 1114, 1548, 2325 and 3183  .... interesting to see if the current run bounces records at approximately the same locations.  If so, might be some bad records.

Another batch of 50 just bounced so now have 98 bounces.

Record numbers for connection refused - previous run vs current run which seems to bounce all remaining emails in the batch:

289  vs.  293
770  vs   770

3rd set of 48 did not have a Connection Refused so maybe this was not it.  I'm changing the 50 every 10 minutes to 15 every 10 minutes.  This might help or might just give more smaller batches to have issues.  We'll see.  Hope it is OK to change the number in the middle of a run.

Thinking back - the "premature bounces" might have started about the time I increased the batch size from 20 every 10 minutes to 50 every 10 minutes, so it will be interesting to see wwith the change to 25 every 10 minutes, if this effects the "premature bounces".

= = = = = =
Changing to smaller batches during the run, did not seem to solve the problem.  When an email bounces in a batch, all later emails in the batch seem to bounce.  Smaller batches mean smaller numbers of bounces from the batch, but still getting bounces.

= = = = =
My latest thinking is that my IMAP2Soap rename didn't do anything and it was just a coincidence that when I tried to rerun batch of bounces before the rename and they 100% bounced, but after the rename all went through.   I'll try this again with the latest set of bounces, still running, but should be about the same.
= = = = =
Results:
Original mailing: 2,427 attempted, 392 premature bounces
Reran group of 390 (not sure why I did not have 392) and they all processed accept for 14 premature bounces
Reran group of 14 that previously premature bounced, all went out.

I'm not sure how I explain to users why it takes 3 tries to get all the mail out the door.

Any suggestions for areas that we should investigate?

I'm also thinking it might be some environmental issue, especially if others are not running into the same situation. 

Interesting, my last run with a much smaller group of 384 had exactly 50 bounces.  We do batches of 50 and the very first bounce had the bounce type = host , reason = connection refused.     In past runs, over half of the bounced batches have the first bounced email (usually not the first email in the batch) as type=host, reason = connection refused, but not all.  Once any email in a batch bounces, all later emails in the same batch bounce, then we are fine with the next batch. 

Perhaps a later version of CiviCRM / CiviMail may solve this or moving to a different server.  We ran just fine with no "premature bounces" for 6 months and now we always get premature bounces with mailings over a few hundred.

Fortunately it is not hard to copy the bounce records to a spreadsheet, import them updating a bounce date custom field, create a new group, flag the group for mailings and have a go with the premature bounces and then repeat the process until there are no bounces left (so far 3 times for the largest run)

If the initial bounced email would always bounce, then some sort of greylist or verification would seem to be suspect.  What gets me is when we turn right around and resend, most of the first round bounces get through and when we did a third time, all the remaining got through.

When it says Bounce Type = Host and Reason = Connection refused, I wonder whose host is refusing the connection - mine?

= = = = =
see also similar reports at:
http://forum.civicrm.org/index.php/topic,5810.0.html

I thought I'd report back with new information.  We had a system problem with emails (unrelated) and I reduced the 50 emails per 10 minutes back to 1 email per 10 minutes and then increased to 10 emails per 10 minutes.

With 10 emails per 10 minutes we had 0 bounces.

I believe that I that I had tested with 20 emails per 10 minutes and still had bounces, so I concluded that the number of emails in the batch was not a factor, but this may be incorrect.  I'll repeat the 20 emails per 10 minute test.

We have had 15% bounces or so consistently for the last 8 months on mailings over 500 at 50 per 10 minutes.  Now on the last run of 3,000 at 10 per 10 minutes, no bounces.  Could be a coincidence and I'll repeat the tests at 20 and 50 per 10 minutes, but I thought I'd provide the information.

Dave