Mailing label address merging

[davesage]

Hi,

I'm new to civicrm and have just completed my first big data upload in order to output some mailing labels. I'm getting some behaviour which I'm a bit unclear about when using the merge address options.

So I can see two check boxes:

Merge labels for contacts with the same address
&
Merge labels for contacts belonging to the same household

Ok, so, I have set up households and contacts as member of the households (not all contacts are in households)

When I tick both the boxes I was expecting to get the contacts in the households to show the name of the household and for those not in households to merge if they have the same address by appending each name.

What I get it sort of this but the households are merging the household name AND the mambers of the household together which looks really funny as they just duplicate the names?

Looks like: 'x & y surname
                  Mr & mrs x surname
                  Street
                  City etc..'

where x and y are the first names of two individuals in the household with household name 'Mr & Mrs x surname'

Is there anything I can do to get the merge of contacts at an address but not when the address is at a household and it has a name? Hope that makes sense.

Cheers,

Dave

[Dave Greenberg]

I think most folks use one or the other option. You might try playing around with what happens if you modify the "Addressee" value for a particular household to be empty (edit household, communications preferences pane). If that fixes things, you'll probably need to do a manual sQL update to fix this for your existing households. You can fix behavior for new households by modifying the default household addressee format (Administer > Option Lists > Addressee Formats).

If this doesn't help, you might need to dig into the code a bit to fix - since it seems like a bit of a bug to be spitting out two contact.address lines.

[davesage]

Hi Dave,

Thanks for replying.

I've been digging around a bit more and have found that the behaviour for these 2 check boxes is very different on 'mailing labels' as apposed to 'export'.

On export it seems to be doing exactly as expected, the addressee gets merged properly without duplication.

On the mailing label output though there seems to be a few things that I can't explain.

1.) It drops the title from the first addressee even though the title is in the addressee for that contact.
2.) If you select the household check box all organisation records get dropped completely
3.) The addressees for households and individuals for households get duped as above.

I was expecting the behavious to be the same.

Should I raise an issue?

Can you point me at the files/queires that relate to the export function and the mailing output and I'll try to start to look at why there is different behaviour.

(The checkboxes do have slightly different text and there is a pop up explanation on the export screen - does this indicate that people are expecting different behaviour in these two actions?)

Cheers,

Dave

PS - it was interesting you said people use one or the other, have I misunderstood? Would it not be normal to want to put people into their households and also de-dupe any other people who are at the same address (maybe where users have forgotten to add them to the household)? Or else you'd still send out 2 items to one address? Sorry if I'm confused.

[Dave Greenberg]

Hmm - I might not be understanding how folks are using this myself. My point about "one or the other" was related to my sense that some folks use household contact records heavily, and others don't use them at all.

I think the functionality was spec'd by Brian Shaugnessy (lcdweb on this form and IRC). Might be better to try and grab him and get his thoughts on "expected behaviors".

Code for Export is in CRM/Export/BAO/Export.php - see mergeSameAddress() and mergeSameHousehold()
Code for Mailing Labels is in CRM/Contact/Form/Task/Label.php - same function names.

[LindseyM]

Hi
I'm new to Civi and am hitting a similar type of problem to Dave.  We deal with many different people - couples, families, single people living in large, shared houses.  As a result, when doing postal mailings, we cannot use 'merge by address' as we have some addresses with 20 people living there!  We therefore need to use households.
 
I need to be able to select groups of individuals for a whole range of purposes (eg by membership type, group, custom field) and then mailmerge labels and letters effectively for them, using the 'merge by households' option.  But... when you action labels, 'merge by address' selected alone works, but 'merge by household' doesn't.  And when you action an export, 'merge by household' works successfully but 'merge by address' doesn't.  Is this just a bug that needs fixing?

An aside is that we also want to mailmerge letters within Civi using pdf templates to write to individuals and couples selected (often within the same group).  The pdf letter action only seems to cope with individuals.  Can it recognise and deal with both couples and individuals in any given list? eg Dear Sam, Dear Tim & Bev.  If that's the case, is it possible to set up the addressee field in household records so that they draw automatically from a couple's records so that a mailmerge or merge by households exercise includes a couple's title eg Dear Tim & Bev.

In a nutshell, our needs seem very simple - select groups of people (including a mix of individuals and couples) and print off labels and mailmerged letters to them within Civi with appropriate addressing/titles.  This must be very easily possible but for some reason we just can't see it.  Any thoughts greatly appreciated.  Thanks.

Lindsey

[Dave Greenberg]

Hi Lindsey - Some responses inline...

I need to be able to select groups of individuals for a whole range of purposes (eg by membership type, group, custom field) and then mailmerge labels and letters effectively for them, using the 'merge by households' option.  But... when you action labels, 'merge by address' selected alone works, but 'merge by household' doesn't.  And when you action an export, 'merge by household' works successfully but 'merge by address' doesn't.  Is this just a bug that needs fixing?

There certainly might be a bug in these merge actions. Best bet is to create a simple set of a few contacts on the public demo and run the two merges, recording what the outputs ARE vs. what you're expecting. If you see a problem, post back here with details on what was in the records, what the output was and what it should have been. The demo DB gets reset periodically but I'll try to jump on it quickly after you post back. (I do think other folks are using these functions and I haven't seen a bug report - but ....).

An aside is that we also want to mailmerge letters within Civi using pdf templates to write to individuals and couples selected (often within the same group).  The pdf letter action only seems to cope with individuals.  Can it recognise and deal with both couples and individuals in any given list? eg Dear Sam, Dear Tim & Bev.  If that's the case, is it possible to set up the addressee field in household records so that they draw automatically from a couple's records so that a mailmerge or merge by households exercise includes a couple's title eg Dear Tim & Bev.

In a nutshell, our needs seem very simple - select groups of people (including a mix of individuals and couples) and print off labels and mailmerged letters to them within Civi with appropriate addressing/titles.  This must be very easily possible but for some reason we just can't see it.  Any thoughts greatly appreciated.  Thanks.

I'm not clear on what you mean by "The pdf letter action only seems to cope with individuals". You can definitely select the Print PDF Letter action for multiple contacts of various contact types and have it create the letters. If the issue is about the "greetings" used in the "merge" - the standard approach is to set the Postal Greeting field for contacts as you want them addressed in letters - and then using the {contact.postal_greeting} token. For individuals, this is set to a default of "Dear {contact.first_name}" when records are created. But for households, we haven't figured out an auto-magical way to populate the Postal Greeting that works for all the variations of "householding" - so you'll have to either set those by editing the household records (from Communications Prefrences pane in Edit Contact form). If that's too onerous, I'm pretty sure you can do an import / update to set "Postal Greeting Custom" field to desired values but a set of households (after manipulating an exported spreadsheet file to set values in a column as needed - e.g. 'Dear Tim and Bev' for a simple 2 person household, 'Dear Stumptown Gang' for a wacky group of student roommates, etc.).

We've gone through quite a few iterations trying to accommodate all the "very simple" combinations of people and household data that folks store in their DBs - so hopefully the existing options can work for you. Otherwise, there's always the option of using a hook to automatically set household greetings using a pattern that works for your data.

[LindseyM]

Dave - thanks very much for your reply.  I'll do the test you suggest re address/household merging and get back to you.  Do you mind if I throw a few quickfire questions at you about households:

- For Civi to recognise a household is it sufficient for people to be related to the household or do they also need to have the 'share address with' box ticked?  If this is the case, does the key address need to be the household address?
- Does Civi need to have a head of household identified?  ie Does head of household have any Civi significance or is it purely for users to record if necessary for their purposes? 
- As we intend use the 'merge by households' option frequently, do we need to ensure even individuals have a household for consistency? ie a household of one to which they're linked... or does this not matter?

Many thanks - I appreciate your input on this one as we're switching over to Civi this week!

[dowd]

Hi Dave,

I have been working with Lindsey on setting up Civi. Thanks for your reply, I have done a test household and done labels, export and PDF letter for each on the demo database as suggested

My Household is Fred, Joe and Sara Bloggs with Stephen Bloggs not in the household as a 'control'!
When selecting them all, including the household, and going to Mailing Labels and merging by Household  I get the expected results of one label for the bloggs household and one for Stephen Bloggs

But when selecting only the individuals and not the household and trying to create labels merged by household it creates labels for each individual (i.e. 4 labels instead of 2)

Whereas if you select "export contacts" and you do not select the household just the individuals then it will just export the two records, one for the Bloggs household and one for Stephen Bloggs as expected.

When trying to "Print PDF letter for contacts" there does not seem to be an option to merge by household to merge by address, the only way I found of doing it was to only select the household and not the individuals in the household.

There seems to be different results each of the different was of doing it, ideally we would like it to be like the "export contacts" way and when you select multiple individuals within a household it will merge them as otherwise we would have to include the Household record in every group where we want to merge from.

Hope that makes sense!

Dowd

[LindseyM]

Hi Dave
I've also just tested, like Dave.  An important point to note is that we are using households to group families.  Whenever I do a postal mailing I will always and only select individuals, but will want to 'merge by household' when it comes to mailing labels so that letters go correctly to couples and to students living in large shared accommodation.

I tested on the Demo with James Dowding, Dave Sharpe and Lindsey & Peter Mansfield.  Dave and James are individuals who happen to live at the same address.  Lindsey & Peter are in a household (Mansfields) and share that household's address.

- Mailing labels merged by address worked. 
- Mailing labels merged by household did not.  Dave who also posted mentioned that the latter works when the household itself is selected but we will never do this as we're always searching and selecting individuals for postal mailings.

- Exporting with merge by address gave me Dave and Lindsey as individuals (with no mention of James and Peter)
- Exporting with merge by household worked - giving me James and Dave as individuals and The Mansfield household

I do hope we can get mailing labels with 'merge by household' to work as this is our key need.  Many thanks

Lindsey

[Dave Greenberg]

Lindsay / Dowd - It does sound like there might be a bug in the Mailing Labels merge and a missing feature (merges) in the PDF Letter feature. I'm not all that familiar with the expected behavior for the merges :-( I'll ask a colleague who was involved in spec'ing these to take a look a chime in here.

Seems like you do have a workaround for now of using the export which does what you need - and then doing a mail merge in Word or Open Office to get the mailing labels printed. 

[Dave Greenberg]

Got a response from my colleague on the Mailing Labels piece of the puzzle:

just looking at the code for label and running a quick test -- the label household merge does require that the search include the household record in order to function, whereas  the export version does not -- which I believe is the preferred solution. so it does seem to be broken.

So go ahead and file an issue on the issue tracker for that "bug" with steps on how to recreate it. If possible, please investigate where thing are breaking down in the code and include a patch.

[dowd]

Issue logged - http://issues.civicrm.org/jira/browse/CRM-8338

[LindseyM]

Many thanks Dave.  You also mentioned that merges seems to be a missing feature in the pdf letter option.  I agree!  Having the ability to choose 'merge by address' or 'merge by household' when creating pdf letters for a group of people would be incredibly useful (invaluable in fact!).  Is that a possibility and something your colleague can put in place?

[Dave Greenberg]

Merge options for PDF letters hasn't come up a lot (or at all in my recollection) as a feature request. So although it seems useful, you'll probably need to come up with some resources (sponsorship $ or contributed code) to make it happen in the short term. If you're interested in pursuing either, let us know.

[LindseyM]

Dave - can I just check there's not currently a way to do what I'm after?

I frequently have to select a group of people which includes both individuals and couples.  I need to be able to send a letter to those people, with it saying Dear firstname for the individuals and Dear firstname and firstname for the couples.  A 'merge by household' or 'merge by address' option would make this possible. 

Given that this 'merge by' feature doesn't exist for pdf letters, do you know how other people are generating pdf letters to couples... or how Civi could potentially be manipulated to do this as we have some programmers on our team?

Thanks again
Lindsey