CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Import (Moderator: Yashodha Chaku) »
  • Import - Compound Names to Households, then Add Individuals?
Pages: [1]

Author Topic: Import - Compound Names to Households, then Add Individuals?  (Read 5499 times)

kenlyle

  • I post occasionally
  • **
  • Posts: 116
  • Karma: 7
Import - Compound Names to Households, then Add Individuals?
October 07, 2008, 01:48:55 pm
I am starting a new Civi install with 2.1.

I have worked through the tutorial at http://wiki.civicrm.org/confluence/display/CRMDOC/Simple+demo+--+Maintain+contact+database

I have a file of Donors, with addresses.  Many of the records are like "Smith, John & Jane"

I believe that John and Jane should be separate "individuals" in Civi-speak, but that they should be, together, a Household, because, apparently, from the file I have, the Household made a donation.

There are no External Identifiers in this file, so I am going to try to create one, because I figure it will be helpful in relating the individuals to the Households.  Is ZipCode&Street&Number a reasonable candidate?  It seems to the benefit of some possible sorting value for reporting.

So, I think that my action item is to create the External Identifier, import the Households, then create a file with the individuals split out, also with the External Identifier, but not as an Individual identifer, but a Household identifier....

Anyone been through this successfully, and willing to share any tips?  Or point to documentation resources?

Thanks,
Ken

petednz

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4899
  • Karma: 193
    • Fuzion
  • CiviCRM version: 3.x - 4.x
  • CMS version: Drupal 6 and 7
Re: Import - Compound Names to Households, then Add Individuals?
October 07, 2008, 02:51:20 pm
Import the Households (one row per Household) with whatever system of Household Name you want (eg Smith, John and Jane, or Smith J&J Household) - then when you import the Individuals (one row per individual) you use the Household Name as the column which will establish the relationship.

So you will need to create a spreadsheet with a row/name for each individual, and the Household Name will be repeated on each of the rows of the relevant individuals.

HTH
Sign up to StackExchange and get free expert advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

pete davis : www.fuzion.co.nz : connect + campaign + communicate

kenlyle

  • I post occasionally
  • **
  • Posts: 116
  • Karma: 7
Re: Import - Compound Names to Households, then Add Individuals?
October 07, 2008, 03:18:42 pm
Great, Peter, That does clarify.

kenlyle

  • I post occasionally
  • **
  • Posts: 116
  • Karma: 7
Re: Import - Compound Names to Households, then Add Individuals?
October 10, 2008, 08:07:31 am
It works like you said.

To clarify, for those following along at home:

Name,LastName,FirstName,FirstName2,HouseholdName,Address Line 1,City,St/Prov,Zip/Postal,Donation Date,Donation Total
"Smith Jr, John & Jane",Smith Jr, John , Jane,"Smith Jr, John & Jane Household",P.O. Box 728,Chilhowie,VA,24319,9/9/1999,0.01

I split the couples' names with the comma and &, then also made the field that appends " Household", in Excel.  The donation data is because these are people who donated an unknown amount on an unknown date.

You then select Household Member of  and Household Name for the dropdowns next to the Household Name field, and the relationship is created automatically. 

OK, I have found one quirky thing:

The import process will let you designate TWO columns to be imported to FirstName, as though it's really smart, and will let you import multiple contacts at a time.  I have seen, I think it was Dave, write about that, but it was something technical sorcery.  But not this way.  You only get one record.  There is no warning that you are causing a mismatch.  The system even reports that two records have been created, like this:

Total Rows     1     Total number of rows in the uploaded file.
Total Contacts    2    Total number of contact records created or modified during the import. (<-this is probably because there is a contact record and some kind of relationship record being created, but it's confusing.  A little more detail would be great.  "1 Individual Contact record was added, and 1 Individual-Household association was added.")

I can go into Civi and see "John's" record.  But "Jane's" record does not get created.

I CAN go back in and re-do the import, selecting Firstname2 to become FirstName....and the household now has 2 members, including "Jane", who has her own Individual contact record.

Thanks for the pointer.

Ken




Dave Greenberg

  • Administrator
  • I’m (like) Lobo ;)
  • *****
  • Posts: 5760
  • Karma: 226
    • My CiviCRM Blog
Re: Import - Compound Names to Households, then Add Individuals?
October 10, 2008, 09:05:46 am
Ken - If I'm following you correctly, the "Total Contacts: 2" report seems correct. The import should have created 1 Individual contact record (John Smith) and one Household contact record (Smith Jr, John and Jane Household). It also created the relationship between John and his household - but we currently don't "report" that (could be a good addition at some point, as well as clarifying counts for each type of contact as you've noted).

If you want to insert both individuals and the household in one pass, I think you need to use two rows - and "map" each individual to the household:
Name,LastName,FirstName,HouseholdName,Address Line 1,City,St/Prov,Zip/Postal,Donation Date,Donation Total
"Smith Jr, John & Jane",Smith Jr, John,"Smith Jr, John & Jane Household",P.O. Box 728,Chilhowie,VA,24319,9/9/1999,0.01
"Smith Jr, John & Jane",Smith Jr, Jane,"Smith Jr, John & Jane Household",P.O. Box 728,Chilhowie,VA,24319,9/9/1999,0.01
Protect your investment in CiviCRM by  becoming a Member!

kenlyle

  • I post occasionally
  • **
  • Posts: 116
  • Karma: 7
Re: Import - Compound Names to Households, then Add Individuals?
October 10, 2008, 09:36:56 am
I just did the import of the "spouses", the second named, traditionally female, members of the households.  I saw your reply about cloning records, but it was much easier to use the same file, sort the records with a FirstName2 to the top, and copy them to a new CSV, using Excel.  I don't know if the import would have skipped the records with no first name.....I think I read another thread that that was the case, but since it was an easy issue to sidestep, I sidestepped...

Here is what I got:

Total Rows     73     Total number of rows in the uploaded file.
Total Contacts    146    Total number of contact records created or modified during the import.

Since the spouse is being associated to an *existing* household, it *does* look like the Individual-Household relationship is being counted, and is being sorted into the wrong bin...

I doubt that this makes sense to the hypothetical "average" user, because there are only 73 "contacts" in the file.

There are many variants that would be more useful, but probably the most useful would be to report *by table name* (or friendly table name) the number of records "touched".

I suggest that to the average user, the 146 above should be 73, and that there should be another row added:
Total Relationship Linkage Whatever  73  Total number of Relationship Linkage Whatever records created or modified during the import.

(A particularly astute coder could probably come up with an even friendlier way to explain, but this seems like the biggest bang for the least effort, as the Civi code can obviously count the number of records impacted)

I'm glad that we agree that this could use some attention.  There's a JIRA issue about which I PM'd Lobo earlier...but can't find now.

Excuse me,I am a little cranky, having been through about 5 hours of fairly intense import work this morning...and running into almost all of the data format, dollar format, fieldname format (why do people put slashes in field names?) issues, but now master of my client's data domain.  :)

There seems to be no single, linear source of information for the import process of starting up a new Civi install, based on just a few hours of research.  Maybe I will be able to twist some of these findings into such a page or two on the Wiki...later.

Best,
K


kenlyle

  • I post occasionally
  • **
  • Posts: 116
  • Karma: 7
Re: Import - Compound Names to Households, then Add Individuals?
October 24, 2008, 04:12:55 am
I did another import today, and tried to take notes.  I added a page to the Wiki at

http://wiki.civicrm.org/confluence/display/CRMDOC/Importing+Households+and+Household+Members

Please feel free, if you have interest to read, use, and improve...

Thanks,
Ken

lgfish

  • Guest
Re: Import - Compound Names to Households, then Add Individuals?
November 04, 2008, 11:07:11 am
Hi Ken, thanks for posting your workflow above and on the wiki, it is really helpful. 
I am working on getting a lot of contacts and data out of raisers edge and into civi. 

Forgive me if this is mentioned elsewhere, I have a pretty basic question -

If i have already imported contributions made by individuals (many of whom give with their spouse, etc - the version of RE we are transitioning from didn't use households or household relationships), but I have not related those individuals to any households (i haven't created the households yet) - will it be easy to relate those individual contributions to the household retroactively?  Or, must the household creation step always come first? 

many thanks,
-lauren

kenlyle

  • I post occasionally
  • **
  • Posts: 116
  • Karma: 7
Re: Import - Compound Names to Households, then Add Individuals?
November 05, 2008, 07:04:12 am
I have really only operated on this from the perspective that I have outlined.  I found it easiest to go "top down", essentially having everything in one spreadsheet/CSV, and then letting Civi do the splits and relationships.

Obviously, behind the scenes, there is a database, and those relationships are defined and stored in the database.

I recall that in the 1.8 timeframe, the database schema was pretty complex, and I believe that some simplification was a goal, but I can see that there are likely some fairly complex queries to finding the entities that use a particular address record, and then checking whether there is a household record and assigning them.

It might actually be easier, since there are likely date/timestamps in the contribution data records you imported, to just delete the contributions, and re-do the whole thing "top down", just because it's so easy, and I can see doing the reverse being potentiall painful.  That's not advice at this time, I am just thinking that it might make best use of the facilitites that are available.

Then, down the road, if you have any issues, or need support, you will not be "out in the cold" with some custom developed matching code/process.

Kudos on moving from proprietary to Open!
Best,
Ken

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Import (Moderator: Yashodha Chaku) »
  • Import - Compound Names to Households, then Add Individuals?

This forum was archived on 2017-11-26.