CiviCRM Community Forums (archive)

Hi, I have been able so far to create a new civi Mailing object and populate it, but confusingly I can't see a parameter in that to specify the mail destination group.

For context, I am dealing with Civi using pure REST api from a remote server. I have a solution to getting a custom template onto the server; the new problem is setting a schedule and delivery group, and initiating the send. I am using the python-civicrm library from github as the intermediary on the client.

I presume send happens as a result of setting the schedule -- i.e. I don't need an API call to say 'send mailing'?  Is setting 'sheduled date' == 'now' safe or should I set a date of 'now + 1min' or similar?

So that leaves setting the delivery group. We already have groups defined, and I want to specify the group by name (and preferably be able to verify in advance that a group name is a valid destination, perhaps by doing a group name -> id lookup).

Can anyone offer pointers?

Ok I figured out my trouble. The API user needed the "CiviCRM: view all contacts" permission. Also, I had to manually turn off url_tracking parameter, as I had fatal DB errors with too-long URLs.

I find something very strange which must be a bug in Civi 4.5: If I set the mailing scheduled_date parameter, two jobs are created and the mailing is sent to each recipient twice!

It sounds like it could be related to the issue I had a while ago: when you create a new Mailing, there are two places where a schedule can be attached -- with in the Mailing code and in the caller of that code, where a chained mailingjob create is automatically added. I suspect that your double is because one job is created in the Mailing code, and then a chained create  causes a second job to be created.

Personally, I regard the automatic addition of a chained mailing job to be a bug in itself, given that you can do this yourself if that is what you wanted.

I ran into some similar issues when using the API for the new Angular-based CiviMail UI in v4.6 - it felt a little quirky. One can use Firebug to inspect the AJAX calls for a live example of using the API. For a full list of fields used by the new UI, check the client-side code for "crmMailingMgr":

https://github.com/civicrm/civicrm-core/blob/4.6.alpha4/js/angular-crmMailing/services.js#L153
https://github.com/civicrm/civicrm-core/blob/4.6.alpha4/js/angular-crmMailing/services.js#L274
https://github.com/civicrm/civicrm-core/blob/4.6.alpha4/js/angular-crmMailing/services.js#L305
  (This uses both Mailing.create for saving drafts and Mailing.submit for final submission. Mailing.submit is new in 4.6; it's just a thin wrapper around Mailing.create which copes with some permissioning edge-cases. If all the users are administrative or if all the flow-control is done server-side, then Mailing.create will be fine.)

I presume send happens as a result of setting the schedule -- i.e. I don't need an API call to say 'send mailing'?  Is setting 'sheduled date' == 'now' safe or should I set a date of 'now + 1min' or similar?

This is correct. FWIW, there should be no real difference between 'now' and 'now+1min'. In either case, the mailing is only scheduled. The agent which sends is driven via cron, so it won't actually go out until the next cron run, and the cron schedule is configured by the site-admin.

I passed in groups[include][]=866 and this seems to have added group 866 to the mailing. Yay! However, the mailing has zero recipients.

The mailing BAO won't automatically compute the full list of recipients until you set the "scheduled_date". Note lines 1646 and 1658 (which are largely unchanged in 4.6):

https://github.com/civicrm/civicrm-core/blob/4.5.5/CRM/Mailing/BAO/Mailing.php#L1646

It sounds like it could be related to the issue I had a while ago: when you create a new Mailing, there are two places where a schedule can be attached -- with in the Mailing code and in the caller of that code, where a chained mailingjob create is automatically added. I suspect that your double is because one job is created in the Mailing code, and then a chained create  causes a second job to be created.

Personally, I regard the automatic addition of a chained mailing job to be a bug in itself, given that you can do this yourself if that is what you wanted.

That definitely confused me, too. I found it easiest to opt-out of the auto-API-chained job by passing an extra parameter {'api.mailing_job.create': 0}; instead, I relied only on the "scheduled_date". This worked better because computing recipients is linked to 'scheduled_date".

This is a bit of a shame. At a high-level, I appreciate the idea of using API chaining to manage jobs and recipients -- it gives more flexibility to the API consumer. However, this particular implementation doesn't really get "there" and just invites bugs. AFAICS, the only way to get "there" is to either break backward compatibility or write messy code.

Are you using the REST api or something else... I'm not sure where groups[] is...

It took me a while to trace that one. There's actually a series of related parameters which are all undeclared and all managed in the BAO: groups[include], groups[exclude], mailings[include], mailings[exclude].

https://github.com/civicrm/civicrm-core/blob/4.5.5/CRM/Mailing/BAO/Mailing.php#L1608
https://github.com/civicrm/civicrm-core/blob/4.6.alpha4/CRM/Mailing/BAO/Mailing.php#L1682

FWIW, the behavior of 'groups' and 'mailings' seemed to be a bit quirky when performing multiple save/load cycle, so I changed the implementation a bit in 4.6. (That won't matter if you're doing an integration with send+forget behavior in 4.5; however, if you're doing an interactive UI like 4.6's, then it's a problem.)

Hi

I have identified why my attempt to use groups/include doesn't work. The URL encoding of the dict I set up results in 'groups=include' which isn't quite right. I'm using the Python requests module for this, though I don't think that's the problem as AFAIK HTTP POST params are single-level.

Is there a syntax like groups_include I could use.

I recall running into similar issues working on CiviCRM - NationBuilder sync from Python for a client, and the solution was to submit JSON as totten recommends.

Thanks for your help, guys.

I have been able to use json as suggested and it works fine.

I have found a problem within the python civicrm lib I'm using: it specifically excluded the 'json' key. I've submitted a bug report to maintainer tallus:
    https://github.com/tallus/python-civicrm/issues/16

As indicated in that report, there is a niggle it would be nice to clarify. In my testing, having fixed the 'json' key exclude, I tried to send all parameters using the json key, but the request was rejected because 'created_id', 'name' and 'subject' -- the required keys for Mailing -- weren't set. I infer that only http params are checked when looking for the required params, although all (http + json) get processed.

Would it be possible to modify the code so that all-json calls were possible? The alternative solutions are either to only json-encode params with an array value, or that callers need to know which params are required and which are optional, which is unfortunate.

In case it helps people, the call is made using this code:

    params = {
        u'name': template.name,
        u'subject': template.subject,
        u'created_id': template.creator_id,
        u'api.mailing_job.create': 0
    }

    # parameters that are json-encoded into the request.
    config = {
        u'from_email': template.from_email,
        u'from_name': template.from_name,
        u'body_html': template.html,
        u'body_text': template.plain,
        u'url_tracking': u'1',
    }

    if enable_mailingjob and len(groupids) > 0:
        # these cannot be in the http parameters.
        config[u'groups'] = {u'include': groupids,
                             u'exclude': []}
        config[u'scheduled_date'] = u'now'

    params[u'json'] = json.dumps(config)

    try:
        results = civicrm.create(u'Mailing', **params)

Just wondering... could this be because of the following sequence:

1. Request comes in with scheduled_date set
2. CiviMail prepares incoming template, notices scheduled_date, creates Mailing job for it. [Mailing.php:1608]
3. As the (same) request processing completes, the shim code adds a chained call (api/v3/Mailing.php:66) that requesting the creation of a second Mailing job, which is then created via API chaining.

Now we have 2 MJs, populated with the same template and data. Both get executed. I will say, though, that only 1 mailing job is reported to the REST caller. I'm not sure if this makes the above sequence impossible or just that Civi isn't set up to consider more than one MJ per call.

I have tried setting api.mailing_job.create = 0 in the request, but this seems to be ignored (in the sense that for a create-but-not-scheduled call sequence, I still get a mailingjob created. Does API chaining work through REST in call cases? Is the problem that api.mailing_job.create is beaten by api.mailing_job.create/api.default when called through REST?

IMO the line at api/v3/Mailing.php:66 should be deleted. It is not helpful at all.
  $params['api.mailing_job.create']['api.default'] = 1;

When the user creates a scheduled mailing you get the job anyway, and if you don't set a scheduled date, having a job attached makes the template unusable via the GUI because the jobs is considered complete.