CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Import (Moderator: Yashodha Chaku) »
  • Importing strategy
Pages: [1]

Author Topic: Importing strategy  (Read 1596 times)

robbrandt

  • I post occasionally
  • **
  • Posts: 89
  • Karma: 0
  • CiviCRM version: 4.0.7
  • CMS version: Drupal
  • MySQL version: 5.1.41-3ubuntu12.9
  • PHP version: 5.3.6
Importing strategy
November 17, 2011, 04:21:51 pm
I've been fooling around with this for a couple of days and not getting what I'm after; it's time to ask some strategic questions about how to do this.  I seem to have some technical issues resolved, but I'm still not getting what I am hoping for.  We have about 10,000 member records.

1) I have an existing custom relational database with our data in it, that I am wanting to migrate to CiviCRM.  Each member has a Member ID, and all transaction, membership, donation, task, etc table has data that relates to that Member ID.  How do I import the data so that these relationships are maintained?

2) I have imported a sample set of members, about 200 of them.  The import data has 2 sets of addresses: a billing address and a shipping address.  I have set up the mapping so that each of those addresses is imported; the billing address as a billing address, and the shipping address as the "main" address.  Both addresses imported correctly, but the billing address is marked as the primary address, is not marked as the billing address, and the main address isn't marked as anything.

Suggestions? Is there a detailed howto for importing data for a new installation somewhere?

robbrandt

  • I post occasionally
  • **
  • Posts: 89
  • Karma: 0
  • CiviCRM version: 4.0.7
  • CMS version: Drupal
  • MySQL version: 5.1.41-3ubuntu12.9
  • PHP version: 5.3.6
Re: Importing strategy
November 17, 2011, 04:39:24 pm
3) I'm also finding that I'm wrestling with addresses a bit.  You have a different approach to addresses than we do; I would call ours a structured data approach, while yours is a visual layout approach.  We have discrete fields for Organization, Department, Address 1, Address 2, City, State, Postal Code and Country, whereas yours simply follows the layout of how the address should appear, except for city state zip & country.  It isn't uncommon for our members addresses to use all those fields, like:

Dr. John Doe
Cornell University
Department of Botany
1234 Mystreet
Office #123
Ithica, NY, 12345, United States

At the moment I am mapping City state zip and country to their obvious mappings, 1234 Mystreet to Street Address, Cornell University to Additional 1, Department of Botany to Additional 2, and I created a custom field for Office #123.  It seems to be working, but I'm not at all certain that I'll be satisfied with this.  Any other suggestions?

robbrandt

  • I post occasionally
  • **
  • Posts: 89
  • Karma: 0
  • CiviCRM version: 4.0.7
  • CMS version: Drupal
  • MySQL version: 5.1.41-3ubuntu12.9
  • PHP version: 5.3.6
Re: Importing strategy
November 17, 2011, 04:47:54 pm
... and the hits just keep on coming :)

4) We have something called Family Memberships, where an entire family is made members for an extra few dollars.  I thought at first that your Household memberships would fit this perfectly, but I just created one as an example and I'm not sure.  We need to name each individual member and treat them as individuals.  For example one might be on our board of directors and the other not.  It's more like a 2-for-1 deal than a single membership.  I see that I can define household members, but will these society members via household function the same as society members via individual memberships?

robbrandt

  • I post occasionally
  • **
  • Posts: 89
  • Karma: 0
  • CiviCRM version: 4.0.7
  • CMS version: Drupal
  • MySQL version: 5.1.41-3ubuntu12.9
  • PHP version: 5.3.6
Re: Importing strategy
November 17, 2011, 05:11:16 pm
Quote from: robbrandt on November 17, 2011, 04:47:54 pm
... and the hits just keep on coming :)

4) We have something called Family Memberships, where an entire family is made members for an extra few dollars.  I thought at first that your Household memberships would fit this perfectly, but I just created one as an example and I'm not sure.  We need to name each individual member and treat them as individuals.  For example one might be on our board of directors and the other not.  It's more like a 2-for-1 deal than a single membership.  I see that I can define household members, but will these society members via household function the same as society members via individual memberships?

OK I answered this one myself.  Very cool.  I created a family contact, associated two professionals as household members, created a family membership type with cascading memberships, and it works! :D

robbrandt

  • I post occasionally
  • **
  • Posts: 89
  • Karma: 0
  • CiviCRM version: 4.0.7
  • CMS version: Drupal
  • MySQL version: 5.1.41-3ubuntu12.9
  • PHP version: 5.3.6
Re: Importing strategy
November 17, 2011, 05:46:09 pm
5) Is there a way to import relationships?

robbrandt

  • I post occasionally
  • **
  • Posts: 89
  • Karma: 0
  • CiviCRM version: 4.0.7
  • CMS version: Drupal
  • MySQL version: 5.1.41-3ubuntu12.9
  • PHP version: 5.3.6
Re: Importing strategy
November 17, 2011, 06:02:57 pm
6) OK, probably the most critical item, membership join & renewal pages.  I'm not really seeing how/where to set them up.  Contribution pages?

Our current join/renew process is a bit lengthy, because it is the one time a year that we know we have their attention.  Not only can they join/renew/change their membership, but they also answer demographic information, volunteer for different things, sign up for extra (paid) services or options, update their contact info, make donations, join sub-groups of the society, and even make gift membership contributions to specific named individuals (with contact data).  They whole process is about 8 pages long, and ends with a single transaction paying for their selections.

How to do that?  I have seen some commercial offerings that we are considering that only have simple pages for doing any of the above things, which isn't adequate.  But one alternative is to fake the multi-page approach with javascript, showing and hiding divs of different content along the way, ending with the only real Submit button on the page.

What do you think?

petednz

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4899
  • Karma: 193
    • Fuzion
  • CiviCRM version: 3.x - 4.x
  • CMS version: Drupal 6 and 7
Re: Importing strategy
November 18, 2011, 01:17:56 pm
Hi Rob - a rather intimidating list ;-)

1/ when we do complex imports we use the Migrate module - you can read about some of the outcomes here http://fuzion.co.nz/content/importing-memberships-related-contributions-using-civimigrate

2/ how are your location types set in civi admin? if Billing is set to Primary, then importing Billing to Billing should result in it being Primary. http://drupal.demo.civicrm.org/civicrm/admin/locationType?reset=1

3/ there are other parts of the address fields - if you look here you will see 'address parsing' - i don't fully understand the options so searching parsing might help you - also check the civicrm_address table and you will see fields inc 'street number', 'street suffix', 'name' etc which may correlate to your old system

4/ well done

5/ yes, if you are using the Import Wizard, if you have eg Person A and Person B in your import, then way down the bottom of the 'select field' options are all the types of Relationships, so you can set it so that Person B 'first name' is set as eg 'child of' 'first name' and then 'child of' 'last name' etc - note if the Person or Org that the person is related to doesn't exist, then a record will be created for them. Often better to import against the 'external id' of the contact if you have already imported all the contacts.

6/ Yep Contribution Page with the membership section enabled - system also now offers Price Sets for Memberships.
You can include Demographic options in a Profile that you expose on the COntribution Page which may take care also of Volunteer info etc. Paying for extra services may be covered by the Price Sets. Joining Groups also via the Profiles.
Gift Memberships - unsure, sorry.

Also check out (sound of trumpets) the Webform CiviCRM module which allows for very complex data to be entered including Person A also entering data about various 'related' people.

Webform CiviCRM also means you can do a bunch of stuff with Conditional fields thanks to the core Webform module (or a contrib one).

Webform CiviCRM doesn't (yet) handle memberships.

Note: I have no commercial interest in WebformCiviCRM other than it being the best thing since ... well since CiviCRM ;-)

Hope some of the above helps.

You may get more responses if you can break some of these items in to separate posts so it doesn't become too daunting a thread.

ps want to share a bit about what the project is for?
Sign up to StackExchange and get free expert advice: https://civicrm.org/blogs/colemanw/get-exclusive-access-free-expert-help

pete davis : www.fuzion.co.nz : connect + campaign + communicate

robbrandt

  • I post occasionally
  • **
  • Posts: 89
  • Karma: 0
  • CiviCRM version: 4.0.7
  • CMS version: Drupal
  • MySQL version: 5.1.41-3ubuntu12.9
  • PHP version: 5.3.6
Re: Importing strategy
November 18, 2011, 03:11:58 pm
Quote from: petednz on November 18, 2011, 01:17:56 pm
Hi Rob - a rather intimidating list ;-)

1/ when we do complex imports we use the Migrate module - you can read about some of the outcomes here http://fuzion.co.nz/content/importing-memberships-related-contributions-using-civimigrate

Excellent, thanks. I am not really familiar with Drupal so I never would have known about this without you saying so.

Quote from: petednz on November 18, 2011, 01:17:56 pm
2/ how are your location types set in civi admin? if Billing is set to Primary, then importing Billing to Billing should result in it being Primary. http://drupal.demo.civicrm.org/civicrm/admin/locationType?reset=1

I don't see "Primary", do you mean Default?  Mine was set to Home as Default, but I'm not using Home.  CiviCRM probably just picked the first one in the table :)  I've now set Main as Default.

Quote from: petednz on November 18, 2011, 01:17:56 pm
3/ there are other parts of the address fields - if you look here you will see 'address parsing' - i don't fully understand the options so searching parsing might help you - also check the civicrm_address table and you will see fields inc 'street number', 'street suffix', 'name' etc which may correlate to your old system

Thanks, I'll look at this.  The import options didn't display them, and I haven't gone so far as to look at the address database schema yet so I didn't know it does parsing like that.  Maybe the Migrate module will have access to them.

Quote from: petednz on November 18, 2011, 01:17:56 pm
5/ yes, if you are using the Import Wizard, if you have eg Person A and Person B in your import, then way down the bottom of the 'select field' options are all the types of Relationships, so you can set it so that Person B 'first name' is set as eg 'child of' 'first name' and then 'child of' 'last name' etc - note if the Person or Org that the person is related to doesn't exist, then a record will be created for them. Often better to import against the 'external id' of the contact if you have already imported all the contacts.

Cool, I see this.  Now I have to deal with unifying all our organizational names so that multiple versions of the same organization aren't created :(

Quote from: petednz on November 18, 2011, 01:17:56 pm
6/ Yep Contribution Page with the membership section enabled - system also now offers Price Sets for Memberships.
You can include Demographic options in a Profile that you expose on the COntribution Page which may take care also of Volunteer info etc. Paying for extra services may be covered by the Price Sets. Joining Groups also via the Profiles.
Gift Memberships - unsure, sorry.

Also check out (sound of trumpets) the Webform CiviCRM module which allows for very complex data to be entered including Person A also entering data about various 'related' people.

Webform CiviCRM also means you can do a bunch of stuff with Conditional fields thanks to the core Webform module (or a contrib one).

Webform CiviCRM doesn't (yet) handle memberships.

Note: I have no commercial interest in WebformCiviCRM other than it being the best thing since ... well since CiviCRM ;-)

I hadn't even thought about other CiviCRM modules, thanks.  I'll take a look.  WebForm sounds promising for other things I've been worrying about, at least.  It also occurred to me last night that since CiviCRM is open and I can access the schema, one option would be to keep our existing join/renew system but have it insert data directly into the CiviCRM database rather than our own.

Quote from: petednz on November 18, 2011, 01:17:56 pm
ps want to share a bit about what the project is for?

It is for 5 different scientific societies.

Thanks for the help, there is obviously a very steep learning curve and some things that are obvious to "old hands" are not to newbies.  Like the Default address option.

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using Import (Moderator: Yashodha Chaku) »
  • Importing strategy

This forum was archived on 2017-11-26.