CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using CiviMail (Moderator: Piotr Szotkowski) »
  • throttling message send rate?
Pages: [1]

Author Topic: throttling message send rate?  (Read 5589 times)

Maarten

  • Guest
throttling message send rate?
October 04, 2007, 05:33:54 pm
Hi folks,

I have CiviCRM installed on a DreamHost account and just learned about the fact that DH has a message sending quota of 100-200 messages per hour to deter spammers. [1]

Can CiviMail throttle its message sending rate to cope with a limit like this? Is it something that would be relatively to hack in for a motivated PHP developer? (e.g. if messages are already sent by a cron job, adapting the script run to send N messages on each invocation only.)

Thanks much.

/Maarten.

[1] It appears that you can go through an approval process to get this limit lifted if you're willing to have every subscriber re-opt-in. DreamHost is awesome in providing free hosting to non-profits, so I honestly can't complain. :)

Piotr Szotkowski

  • Moderator
  • I live on this forum
  • *****
  • Posts: 1497
  • Karma: 57
Re: throttling message send rate?
October 05, 2007, 06:14:18 am
Quote from: Maarten on October 04, 2007, 05:33:54 pm
I have CiviCRM installed on a DreamHost account and just learned about the fact that DH has a message sending quota of 100-200 messages per hour to deter spammers. [1]

Can CiviMail throttle its message sending rate to cope with a limit like this?

Yeah, you can go to Administer CiviCRM → Mailer Settings and set the Mailer Spool Period and Mailer Batch Limit accordingly. You can also use just the Batch Limit and call the cronjob seldom enough (Dreamhost-limit / Batch-Limit times per hour).
If you found the above helpful, please consider helping us in return – you can even steer CiviCRM’s future and help us extend CiviCRM in ways useful to you.

Maarten

  • Guest
Re: throttling message send rate?
October 05, 2007, 09:01:25 am
Wow, how did I miss that?  Thanks!

Denver Dave

  • Ask me questions
  • ****
  • Posts: 471
  • Karma: 9
Re: throttling message send rate?
December 06, 2007, 10:13:43 pm
Does the Administer CiviCRM → Mailer Settings effect both the basic send mail to selected records and CiviMail ?

Kurund Jalmi

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4169
  • Karma: 128
    • CiviCRM
  • CiviCRM version: 4.x, future
  • CMS version: Drupal 7, Joomla 3.x
  • MySQL version: 5.5.x
  • PHP version: 5.4.x
Re: throttling message send rate?
December 06, 2007, 10:18:08 pm
Quote
Does the Administer CiviCRM → Mailer Settings effect both the basic send mail to selected records and CiviMail ?

No its only related to CiviMail.

kurund
Found this reply helpful? Support CiviCRM

Denver Dave

  • Ask me questions
  • ****
  • Posts: 471
  • Karma: 9
Re: throttling message send rate?
December 10, 2007, 09:29:19 pm
Could we review the Administer CiviCRM / CiviMail / Mailer Settings:

Mailer Spool Period - 180
Number of seconds between delivery attempts for new outbound mailings

Is this 180 seconds (3 minutes) for each email or for each batch of emails?  Seems like a long time for each individual email, but not very long for each batch.  What type of setting are others using here and what does it mean.

Mailer Batch Limit - 0
Throttle email delivery by setting the maximum number of emails sent during each CiviMail run (0 = unlimited).   Is this the number in each batch set above or the total emails requested to be sent?

Mail Spool Limit - blank
Set the limit of emails sent via smtp mailer, for more than limit send them in Spool table

Any examples.

I have 4 email addresses for myself - so sending - with above with
http://<your drupal site>/civicrm/mailing/queue?reset=1
sends all 4 emails immediately

Changing to
Mailer Batch Limit = 2
Mail Spool Limit = 3 or 4

Processes 2 first time and 2 with 2nd url request

Changing to
Mailer Batch Limit= 2
Mail Spool Limit = 1

Processes 2 first time, 1 with 2nd url request and stays running with no more emails even when requested.

Don't do new civicrm cycle, but just change mail Spool Limit to 2
Does send one remaining email.

Does not seem like the 180 second setting has any effect when I process by url (no cron job running).


Piotr Szotkowski

  • Moderator
  • I live on this forum
  • *****
  • Posts: 1497
  • Karma: 57
Re: throttling message send rate?
December 12, 2007, 01:23:28 am
Quote from: Denver Dave on December 10, 2007, 09:29:19 pm
Mailer Spool Period - 180
Number of seconds between delivery attempts for new outbound mailings

Is this 180 seconds (3 minutes) for each email or for each batch of emails?  Seems like a long time for each individual email, but not very long for each batch.  What type of setting are others using here and what does it mean.

It’s the time for which the spooler sleeps between calling the runJobs() – see tools/civimail-spooler/civimail-spooler.php.

Quote from: Denver Dave on December 10, 2007, 09:29:19 pm
Mailer Batch Limit - 0
Throttle email delivery by setting the maximum number of emails sent during each CiviMail run (0 = unlimited).   Is this the number in each batch set above or the total emails requested to be sent?

This is the maximum number of emails delivered during one runJobs() run.

Quote from: Denver Dave on December 10, 2007, 09:29:19 pm
Mail Spool Limit - blank
Set the limit of emails sent via smtp mailer, for more than limit send them in Spool table

Any examples.

I grepped the CiviCRM sources and it seems to be an unused setting, from back when we wanted to offer parallel spooler and SMTP options.

Quote from: Denver Dave on December 10, 2007, 09:29:19 pm
I have 4 email addresses for myself - so sending - with above with
http://<your drupal site>/civicrm/mailing/queue?reset=1
sends all 4 emails immediately

Changing to
Mailer Batch Limit = 2
Mail Spool Limit = 3 or 4

Processes 2 first time and 2 with 2nd url request

This makes sense and works as advertised.

Quote from: Denver Dave on December 10, 2007, 09:29:19 pm
Changing to
Mailer Batch Limit= 2
Mail Spool Limit = 1

Processes 2 first time, 1 with 2nd url request and stays running with no more emails even when requested.

Don't do new civicrm cycle, but just change mail Spool Limit to 2
Does send one remaining email.

This is rather surprising, interesting and – to my eyes – a bug. Please file an issue with the above finding at our issue tracker.

Quote from: Denver Dave on December 10, 2007, 09:29:19 pm
Does not seem like the 180 second setting has any effect when I process by url (no cron job running).

It’s a spooler limit, so it does not apply to the URL (nor the cronjob) trigger.
If you found the above helpful, please consider helping us in return – you can even steer CiviCRM’s future and help us extend CiviCRM in ways useful to you.

Denver Dave

  • Ask me questions
  • ****
  • Posts: 471
  • Karma: 9
Re: throttling message send rate?
December 20, 2007, 08:47:59 pm
Sorry, I have to ask.  If the cron job is set to every 15 minutes and the mailer spooler interval is set to 180 seconds (3 minutes), is the next spooled batch going to run only once every 15 minutes anyway, when the cron job calls the spooler?
- - - -
Let's say that we have 1,200 emails to send and some flexibility on how long.  Would we be better sending 100 once per hour or 25 every 15 minutes or 5 every 3 minutes?  What are other's doing?
- - - -
How resource intensive is just the act of running the cron job?  I know it checks settings and sends an email that it ran.  I guess I have to start someplace, so I picked running every 10 minutes and sending 20 emails in a batch.  So this gives 120 emails an hour and half a day to process our whole list - does this sound reasonable?
« Last Edit: December 22, 2007, 06:33:14 pm by Denver Dave »

Piotr Szotkowski

  • Moderator
  • I live on this forum
  • *****
  • Posts: 1497
  • Karma: 57
Re: throttling message send rate?
December 27, 2007, 01:35:04 am
Quote from: Denver Dave on December 20, 2007, 08:47:59 pm
Sorry, I have to ask.  If the cron job is set to every 15 minutes and the mailer spooler interval is set to 180 seconds (3 minutes), is the next spooled batch going to run only once every 15 minutes anyway, when the cron job calls the spooler?

I haven’t actually checked it, but I’d say – yes, this is how it would most probably work.

Quote from: Denver Dave on December 20, 2007, 08:47:59 pm
Let's say that we have 1,200 emails to send and some flexibility on how long.  Would we be better sending 100 once per hour or 25 every 15 minutes or 5 every 3 minutes?  What are other's doing?

It all depends on your setup. If you have a sanely configured mail system, it shouldn’t really matter (the mailing system should spool the mails in its queue and send them in a sane manner). These limits are mostly for shared hosting installs, which usually have a limit on mails sent per hour/day.
Quote from: Denver Dave on December 20, 2007, 08:47:59 pm
How resource intensive is just the act of running the cron job?  I know it checks settings and sends an email that it ran.  I guess I have to start someplace, so I picked running every 10 minutes and sending 20 emails in a batch.  So this gives 120 emails an hour and half a day to process our whole list - does this sound reasonable?

It’s not really intensive, and your settings look sane.
If you found the above helpful, please consider helping us in return – you can even steer CiviCRM’s future and help us extend CiviCRM in ways useful to you.

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using CiviMail (Moderator: Piotr Szotkowski) »
  • throttling message send rate?

This forum was archived on 2017-11-26.