Author Topic: Mass Mailing from search results confusion (Read 9142 times)

adharris · May 13, 2011, 08:31:49 am

I'm trying to wrap my head around how sending a CiviMail from search results works. I understand that you need a "Base Group" for such mailings, and that all members of the base group are included in the mailing regardless of whether or not they are in the search results. Therefore, I have seen people suggesting having an empty base group for the purpose of sending such mailings.

My understanding is that the purpose of the base group is that CiviMail needs a way to manage unsubscribes, but that does not mesh with my observations. Here's what I am doing and what i see as a result:

Created a Group called "Mailing Base Group." I added a single test contact to this group.
Searched for a contacts. My query returned 2 results. Select all results, "Schedule/Send Mass mailing"
Filled out all mail info, sent the email. Mailing when successfully to 3 contacts (the 2 search results and the 1 contact in the base group)
Unsubscibed one of the search result contacts via link in mailing. The unsubscribe page says "Unsubscribed from hidden smart group 10
Perform same search again, select same 2 contacts, schedule a new mailing, select same base group. Send another mailing.
Contact who is unsubscribed still receives new email

Is this the expected behavior? It seems to me that the unsubscription is actually stored in the hidden smart group. if this is the case, what is the purpose of the base group?

I'm kinda lost on how we would go about sending a mailing to a subset of one of our mailing lists. If we have an "advocacy" mailing list, and we want to send an advocacy email to all contacts in that list who reside in California, and we want the contacts unsubscription status in the advocacy group respected in this CA mailing, how would we go about doing that?

Donald Lobo · May 13, 2011, 09:48:00 am

IMO:

5 and 6 are bugs. 4 is also bad thing and should be fixed to say: "You have been unsubscribed from future mailings"

if you can investigate and provide a patch that would be great. we can help u get started on IRC. You should file an issue so we can track it.

Dont have any ideas right now on how to handle your specific use case

lobo

adharris · May 13, 2011, 10:11:41 am

lobo,

My gut says what should be happening for 4-6 is that an unsubscriber gets added to the base group with status removed, and then future mailings which use the same base group should respect the unsubscription status in the base group if the contact already is in the base group.

It's also important to note that the contact should only be added to the base group if they unsubscribe, not by virtue of being in the initial mailing. Otherwise the base group could not be used to send mails to disjoint sets of contacts.

If that seems right, I'll open a ticket saying that.

Also, can you explain to me the decision to make the mailing send to the entire base group, even for contacts who do not match the search criteria? It seems unintuitive. Also, if it sent to just the contacts that matched the search, then that would solve my use case.

Donald Lobo · May 13, 2011, 11:21:31 am

Quote from: adharris on May 13, 2011, 10:11:41 am

My gut says what should be happening for 4-6 is that an unsubscriber gets added to the base group with status removed, and then future mailings which use the same base group should respect the unsubscription status in the base group if the contact already is in the base group.

It's also important to note that the contact should only be added to the base group if they unsubscribe, not by virtue of being in the initial mailing. Otherwise the base group could not be used to send mails to disjoint sets of contacts.

If that seems right, I'll open a ticket saying that.

yes the above is right and is intended behavior

Quote from: adharris on May 13, 2011, 10:11:41 am

Also, can you explain to me the decision to make the mailing send to the entire base group, even for contacts who do not match the search criteria? It seems unintuitive. Also, if it sent to just the contacts that matched the search, then that would solve my use case.

dont remember why we decided this exactly. Might have been due to what the client wanted when we implemented this functionality

lobo

adharris · May 13, 2011, 04:03:37 pm

Ok, thanks. I created an issue http://issues.civicrm.org/jira/browse/CRM-8114

what are your thoughts on the work required to adjust this functionality so that there is an option to include/exclude the base group when sending a mailing from search? I would think having two options "Include the contacts who are in base group but not in the search results in this mailing?" and "Add all contacts from this search result to the base group" when sending this type of mailing would solve a bunch of our use cases.

This type of functionality is pretty key to how our organization manages our outreach. So depending on how much work this would be, we might be able to raise funds to see this happen.

ken · May 13, 2011, 06:36:49 pm

Interesting thread!

I have two thoughts.

Delete the hidden smart group once the mailing is sent
Clarify the purpose of the Base Group

Delete the hidden smart group once the mailing is sent: In 3.4, Mailing List groups can now be deleted without disturbing the mailing data. The hidden group seems to serve no purpose, once the mailing is sent. It seems that the only group which needs to be preserved if the "Base Group", to handle unsubscribes.

Clarify the purpose of the Base Group: the phrase "base group" is confusing and always bothers me when I use that screen as I remind myself what it means. Here are my thoughts ...

The behaviour when someone "Opts Out" is clear - no further CiviMails will be sent to this person
The behaviour when someone "Unsubscribes" is less clear - what are they unsubscribing from? What is the thread of communications of which this is a part?

My proposal to resolve this borrows ideas from Child Groups and the Petition Contact group (CiviCampaign) ...

Have a system group called "Unsubscribed from Search Mailings" (a better title is needed!)
Allow people to rename it (like the CiviCampaign 'Petition Contact' group) to suit their local usage
Allow people to create child groups (for instance, so people unsubscribing from fundraising letters can be tracked separately from volunteer requests)
Where "Base Group" is currently selected, have a dropdown showing "Unsubscribed from Search Mailings" and its child groups (if any)

This simplifies the usage as one doesn't need to select a "Base Group" unless one wants to change the default. One can see all the people who have unsubscribed from Search Mailings, or only those in one of the child groups (eg, volunteer requests).

Ken

Donald Lobo · May 13, 2011, 06:43:29 pm

so currently we do send the mail to all contacts w/status added in the base group. So option 1 is already covered

Not sure why u want option 2 (since the user should just add the contacts and then do a normal civimail to a group). Not sure we want to add functionality there. You can potentially do what you want via a hook in the current code base

The thing which is missing, which might be useful is:

Do not add any contacts from the base group to the mailing. This will allow you to use the Advocacy group as the base group and then send a mailing to only the members from CA. If they unsubscribe, they are then unsubscribed from the Advocacy group. I suspect this is a 10-20 hour project with the bug fixes and incorporating some of Ken's suggestions

lobo

adharris · May 16, 2011, 08:31:40 am

@ken
I'm not sure I follow much of what you are proposing.

Our goal is to be able to send a mailing to a subset of one of our mailing lists based on an arbitrary search criteria. The unsubscription should be handled through the parent group. It seems from what you are proposing, the contact would unsubscribe from the "Unsubscribed from Search Mailings" group, and would therefore not receive any future search-based mailings. This does not quite mesh with our use case, as we would have multiple mailing lists. For example, we might have an Advocacy and a Fundraising mailing list. We would like to send an email to all members of advocacy who are in a specific state. If someone unsubscribes from this mailing, they should still receive an email sent to contacts Fundraising group who have donated over X amount. Is something similar possible in the context of what you were saying?

@lobo
Sorry I probably could have been a bit clearer. My interest in option 1 is the ability to leave that option unchecked, so that the members of the base group are not included in the mailing. This concept is what you were suggesting in your post.

Option 2 is probably necessary, since the user could (and probably should) add all the contacts to the advocacy group before sending them a mailing.

Donald Lobo · May 16, 2011, 10:07:27 am

adharris:

yes, ok we both agree on option 1 in that format. And your use case makes a lot of sense

I assume u mean option 2 is probably unneccessary (since they are other ways of doing the same thing)

If you can work on this and contribute a patch that would be great

Ping us on IRC if u need help getting started

lobo

ken · May 16, 2011, 10:03:07 pm

@adharris

I'm considering the case where people use Advanced Search in an ad hoc manner, and then decide to use CiviMail to send a mailing to those people. In such a situation, there is no Base Group.

My primary request is that the concept of "Base Group" is clarified so that users like me understand what's going on.

Thanks,
Ken

adharris · May 17, 2011, 07:23:32 am

Ken,

Functionally, I think there always needs to be a base group, because there has to be a place to unsubscribe from. Creating a system group to manage this would limit the functionality; a contact would then unsubscribe from all possible search mailings. However, I do agree that in cases where there might not be a common trait between the search results the concept would be a bit confusing to the user. In these cases, it probably makes sense to have a "Default Base Group" (should probably be named to reflect your specific use case), which is used to handle unsubscribes from your ad-hoc bulk mails. I'm not sure that this group needs to be system created, as any empty group will get the job done.

I'm working on a few changes to clarify what a base group is to the end user, as well as changing the way they are referenced on the mail report page. suggestions?

ken · May 17, 2011, 03:23:51 pm

@adharris, @lobo,

Having thought about things further, may I clarify my earlier proposal?

Delete the hidden group once the mailing has been sent
Rename "Base Group" to "Unsubscribe Group"
Add pop-up help for the "Unsubscribe Group" field which says something like "It is a legal requirement that recipients of this email have the option to unsubscribe from future mailings like this one. Nominate which group that will be. For example, if you're email concerns fundraising, use one of you fundraising groups. Alternately, if there is no group that matches your purpose, you may create a special group that is used for this purpose. If you don't want people to unsubscribe, then that suggests the bulk mailing option is inappropriate and you need to use another mechanism." (This is a bit long but hopefully clear.)

Now that I understand better what a Base Group is, I withdraw my earlier suggestion of having a system default.

I think the pop-up help would go a long way to helping people use this feature.

Currently, when I go to schedule a mass mailing as an action after an Advanced Search, I'm required to supply a "Base Group". There's no pop-up help on that field (though there is for the Name and the Campaign). Renaming the field and adding the pop-up help goes a long way to explaining what the field is for and why it's needed.

I couldn't find any mention of base groups in the online documentation. Here's how the FLOSS manual describes it at http://en.flossmanuals.net/civicrm/ch044_everyday-tasks/ ...

Quote

There are two ways to select the recipients for your mailing.

* If you are sending mail to an existing group, go to Mailings > New Mailing. From this screen you can choose the groups you want to send the mailing to. You can also choose to exclude contacts who are members of another group or who have received previous mailings.
* Carry out a search (for example, using the advanced search) and then choose "schedule/send a mass mailing" from the actions drop down. You will be redirected to the New Mailing screen (step 1), where you can choose a "base group" for your mailing. You need to choose a base group for your mailing because you need to give people the option of unsubscribing from your mailing.

Ken

lcdweb · May 18, 2011, 06:11:09 am

good discussion here. and I too would like to see some of these fixes/improvements put in place.

but I think we can/should simplify things, because we can safely assume some things about the user's intentions.

* the confusion only impacts ad hoc mailings generated from search results. so i think whatever we do should be geared toward that specific use case. for example, I would vote against any functionality (optional or not) that includes people from the base group in the mailing. that is confusing, will quite often lead to unintended results, and such capability is better achieved through the normal group-based mailing workflow. the sole purpose of the unsubscribe/base group selection should to address the one gap -- determining what group the user is unsubscribed from.
* the key requirement that we need to accommodate is the ability for someone to unsubscribe to a mailing which was not generated from a group -- and thus does not have a clear place to unsubscribe from. so the current "base group" concept is fine, but I agree with Ken that we need to rename it and provide more help text for the user
* personally, I would like to see the option of setting a default value for the unsubscribe group through the civimail config. it's easy enough to do that via hook; but i think it would be a useful enough feature that we should just make it available via the interface.
* any reference to a hidden group should be completely hidden to the user. i understand the hidden group has been used to track who received a given mailing -- that makes perfect sense. but we should truly hide it from the user, as it can be/is very confusing. if we delete after use, we just need to make sure that the 3.4 changes truly removed any lasting need for that group (in past versions it was necessary for reports, etc.)

adharris · May 18, 2011, 08:12:46 am

@lcdweb:

I'm tentative to remove the ability to include the base group, as it was obviously the desired functionality at one point in time and I don't want to break something that someone relies on. However, I think that since the user could manually add the base group to the included contacts, this functionality is still in place. Therefore there wouldn't need to be an explicit option; you could just include the base group when necessary.

I wonder if makes sense to expose the "Unsubscribe Group" option to normal mailings as well? It would be possible to abuse, but the ability to "send an email to this group, using and storing unsubscription information from this other group" would be a pretty powerful feature.

I'd be on board with a default unsubscription group in the settings, but for our use case it would have to be pretty easily override-able on a per-mailing basis. For our use case, having a settable default would not be really necessary because the choice of which unsubscribe group to use will need to be made with mailing anyways.

Absolutely we should remove references to the hidden smart group. Right now i see it referenced on the unsubscribe page and on the mailing report page. Let me know if you see it elsewhere and I'll work that into the fix.

adharris · May 18, 2011, 01:38:37 pm

So i submitted a first version of a patch to do some of the stuff discussed here. Complete(ish) info at the issue: http://issues.civicrm.org/jira/browse/CRM-8114

It should make it so that users who unsubscribe from a search based mailing are unsubscribed from the base group, not the hidden smart group, which is the bug that i first noticed.

As per suggestion, i changed the form label for "Base Group" to unsubscription group and added some help text.
When using a search based mailing, the base group is no longer included in the mailing (you have to manually add them to the included groups list if you want them).
I also edited the mailing report so that it now properly lists the base group (before it was under excluded groups). The "Hidden Smart Group" text on the report was changed to "Search Results" to indicate that the mailing was sent from a search.

Let me know if you find something wrong, or if you have other suggestions.

CiviCRM Community Forums (archive)

News:

Author Topic: Mass Mailing from search results confusion (Read 9142 times)

adharris

Mass Mailing from search results confusion

Donald Lobo

Re: Mass Mailing from search results confusion

adharris

Re: Mass Mailing from search results confusion

Donald Lobo

Re: Mass Mailing from search results confusion

adharris

Re: Mass Mailing from search results confusion

ken

Re: Mass Mailing from search results confusion

Donald Lobo

Re: Mass Mailing from search results confusion

adharris

Re: Mass Mailing from search results confusion

Donald Lobo

Re: Mass Mailing from search results confusion

ken

Re: Mass Mailing from search results confusion

adharris

Re: Mass Mailing from search results confusion

ken

Re: Mass Mailing from search results confusion

lcdweb

Re: Mass Mailing from search results confusion

adharris

Re: Mass Mailing from search results confusion

adharris

Re: Mass Mailing from search results confusion