CiviCRM Community Forums (archive)

1. might want to switch to a better host

2. The only limit we have right now is mailings sent per cron run. You can make the cron go every hour and set the mailng limit to the hourly limit. However it might deliver all that email in the first few seconds / minutes (depending on how fast your machine is)

lobo

2. The only limit we have right now is mailings sent per cron run. You can make the cron go every hour and set the mailng limit to the hourly limit. However it might deliver all that email in the first few seconds / minutes (depending on how fast your machine is)

The mail limit applied this way is not very useful if the application tries to enqueue all e-mail messages at once. The mail per second limitation on our (Siteground) shared servers is applied in attempt to ensure that a single user will not fill-in the mail server queue and thus affect all other users on the same server. This is especially valid when dealing with mailing lists where the number of recipients is usually in the thousands range and naturally, having all the messages enqueued at once in such cases, is not a good idea.

The limit is also not something unique for SiteGround. I know (from my own personal experience) that many other hosting companies, such as APlus, IPowerWeb, Bluehost, LunarPages have the same or similar mail service limitations and it is there for a reason.

What you have recommended as first solution (i.e a change to a different hosting provider) might solve the problem for Koen but it is just a workaround in the long term and there will be other users of CiviCRM who will  be struggling with the same issue. I know that most mailing list scripts and applications (phpList, DaDa Mail and a whole bunch of commercial scripts) have a configurable option where you can instruct the script to send x number of messages every second or send the messages with X milliseconds delay in between. I was rather surprised to see CiviCRM has no such option, as opposed to the otherwise excellent functionality of the app.

Perhaps it would be suitable to implement a simple mail throttling feature? I do not imagine it to be  very difficult to code and it will surely fix the issue for good and for all CiviCRM users.

Well, I'm not a full time developer and I don't fancy myself one, but I looked at the files and gathered a small patch file, which I've attached to this message. The patch file edits 4 files of the application and I've listed the changes below (unfortunately the code tag is considered as link by the forum and I couldn't use it) :

File: CRM/Mailing/Config.php
Purpose of the edit: Add a configurable value for the throttling time to the config object.
The actual code:
-----------
     /**
      * How long (in micro seconds) should we wait in between each mail delivery? Setting this to 0 disables it.
      *
      * @var int
      */
     public $mailThrottleTime = 120000;
-----------
The above sets the mailThrottleTime to 120,000 which is about 120ms and having in mind  the limit of 10 messages per second rate, it will be just about enough to avoid any issues with it. The second file to edit is

File: CRM/Admin/Form/Setting/Mail.php
Purpose: Add a form element in the mailer configuration page
Code:

-----------
$this->addElement('text','mailThrottleTime', ts('Mailer Throttle Time'));
-----------

Next file is

File: templates/CRM/Admin/Form/Setting/Mail.tpl
Purpose: Render the above element in the config form.
Code:
-----------
  <tr class="crm-mail-form-block-mailThrottleTime">
      <td class="label">{$form.mailThrottleTime.label}</td><td>{$form.mailThrottleTime.html} <br />
      <span class="description">{ts}The time to (u)sleep in between each e-mail in micro seconds. Setting this above 0 allows you to control the rate at which e-mail messages are sent to the mail server, avoiding filling up the mail queue very quickly. Set to 0 to disable.{/ts}</span></td>
  </tr>
-----------

And the last file to edit

File: CRM/Mailing/BAO/Job.php
Purpose: Throttle each e-mail delivery for the configured micro seconds
Code (around 573 line after the hack to stop mailing job at run time)

-----------
  // If we have enabled the Throttle option, this is the time to enforce it.
  $config = CRM_Core_Config::singleton();
  if($config->mailThrottleTime > 0){
    usleep((int)$config->mailThrottleTime);
  }
-----------

It is in no way perfect, but as mentioned I'm not a full time developer and I don't really know CiviCRM code that well (as you did). Let me know if you have any comments/suggestions about it.
 

Hello,
I'm using CiviCRM 4.1.5 and I'm interested in the Val's patch.

Someone he tested? Does it work?

Thank you for your answers!

hello,

I have found : http://issues.civicrm.org/jira/browse/CRM-9698

In the file:CRM/Mailing/BAO/Job.php : where do I put these lines:

  // If we have enabled the Throttle option, this is the time to enforce it.
  $config = CRM_Core_Config::singleton();
  if($config->mailThrottleTime > 0){
    usleep((int)$config->mailThrottleTime);
  }

Val to put them here : Code (around 573 line after the hack to stop mailing job at run time)

I put it at the end of the function :
public function deliverGroup(&$fields, &$mailing, &$mailer, &$job_date, &$attachments) {

like this :
          // hack to stop mailing job at run time, CRM-4246.
          // to avoid making too many DB calls for this rare case
          // lets do it when we snapshot
          $status = CRM_Core_DAO::getFieldValue('CRM_Mailing_DAO_Job',
            $this->id,
            'status'
          );
          if ($status != 'Running') {
            return FALSE;
          }
        }
      }
  // If we have enabled the Throttle option, this is the time to enforce it.
  $config = CRM_Core_Config::singleton();
  if($config->mailThrottleTime > 0){
    usleep((int)$config->mailThrottleTime);
  }
      unset($result);

      // seems like a successful delivery or bounce, lets decrement error count
      // only if we have smtp connection errors
      if ($smtpConnectionErrors > 0) {
        $smtpConnectionErrors--;
      }
    }

    $result = $this->writeToDB($deliveredParams,
      $targetParams,
      $mailing,
      $job_date
    );

    return $result;
  }

Is that correct ?


Thank you for your help

euh ?

Is this parameter : Mailer Batch Limit - Throttle email delivery by setting the maximum number of emails sent during each CiviMail run (0 = unlimited) ??

But what means  this sentence : the maximum number of emails sent during each CiviMail run

I need more than one cron job

I would like a few details ...

Thanks