GSoc2015 on project Social media integration with CiviCRM

[achia]

Hi folks, my proposal has been attached to this mail and i am hoping we could discuss on it.

[achia]

The project social media integration with CiviCRM is of practical importance to the goal of CiviCRM which is to better manage organization’s audience (Constituent Relationship Management).

This project involves ensuring that the platform of CiviCRM should be able to interact with social media platforms and ease or increase the effectiveness of the way an organization manages its constituent relationships.
Since CiviCRM aims at sustaining relationships with supporters over time, the social media integration project will help the organization to use CiviCRM to sustain supporters relationships by doing the following;

integrate with social media platform logins such as Facebook, twitter, and Instagram for quick and easy petition signing or event registration and to get authorized access to petition-signers' social graphs, pull in contacts from social media & match to CiviCRM contacts based on available information, import into CiviCRM data about who is re-tweeting, sharing, writing on your wall, establish ways to determine highly active users based on their social media activity, Add social media feed dashlet to see 'what's going on right now', Add ability to automatically publish/tweet about events, certain types of activities, possibly progress towards fundraising goals.

 Key deliverables of this project include the following; scope and specification document of the project, a UML model document of the project, Social media integration with CiviCRM to pull in contacts from social media and match to CiviCRM contacts base, Social media integration with CiviCRM to enable social media login, Social media integration with CiviCRM to import into CiviCRM data about who is re-tweeting, sharing, writing on your wall, Social media integration with CiviCRM to establish ways to determine highly active users based on their social media activity, Social media integration with CiviCRM to add social media dash-let to see ‘what’s going on right now’, Social media integration with CiviCRM module to add ability to automatically publish/tweet about events, certain types of activities, possibly progress towards fundraising goals,  development and user manual of the project.

[sonicthoughts]

This can be a powerful contribution to civicrm.  I've done some work on social integration but using the CMS rather than a native approach.  I have a few thoughts/suggestions to keep in mind:

1. Understand duplication/boundaries of CMS + social integration.  This is particularly important for user authentication as we do not want to ask users to authenticate twice (once for CMS, then again for civicrm.)  If you focus on twitter+FB and WP+Drupal CMS you may find that there is a middle ground in
 a) bridging with CMS like we do with contacts/groups and sync and
 b) leveraging all the existing code for the CMS.  For example, if a user logs in via twitter to the CMS then pass those credentials, to civicrm (or make them aware they exist.)

2. Think about BOTH input and output.  Getting user activity into civicrm is great, but sharing from civi is powerful.  For example, Pushing event info to a FB user's friends, noting a contribution or event attendance.

3. Pushing info to the organizations account/page is every powerful.  In drupal, we built a system to tag an event (publish to facebook) and automatically post to the org's wall (as a FB event with all the appropriate bells/whistles.) There is a drupal module called Facebook Autopost that handles this very well.  Likewise a push to twitter feed about a new event, grant app, etc. is useful.  It could even be integrated with dashboards/reports to publish stats/etc or the mail system to publish content.

In a sense the social networks are like an extension of the CMS as a publishing point, but civicrm has a lot of the native content.

Not trying to blow up the scope but please keep this in mind....

Best of luck!

Shawn

[sonicthoughts]

Something else to reiterate.  Most orgs who are interested about social integration are already using it (or should be.) so they have a twitter /FB account with followers, mentioners, etc.  Your proposal talks about capturing "public" info.  I assume you will have the org specify their accounts/pages and pull all follower, activities, etc. all of which are avail from the org's accounts(s)?

Also don't forget about rich media.  IMO it is often an afterthought but very important for user experience.  Assets like profile pics, event photos, and other media on social networks can really spice-up the civicrm content.

[achia]

This project has been divided into two sub-parts. One of the parts deals with social media login and the other deals with adding social media account information to contact records, finding ways of showing level of engagement for individual contacts and  to investigate generating automatic tweets/posts based on contact activity.We are trying to scope and i think getting all the possibilities from the community is a strong point rather than blowing the scope. Only when we have the possibilities then we can choose the ones which are of primary priority and start with then the rest of the items can be subsequently added as we maintain the project

Based on the second part of the project, we are proposing the scoping as follows;

1. we will be concentrating on making interactions with twitter and Facebook

2. What information can Civi get from Facebook and does this information differ by organizations? and if so should possibility for an organization to choose this information be provided or the extension should have a standard of information it gets from Facebook.

3.What information can Civi send to Facebook ? should the information that an organization send to Facebook be specific to that organization? if so, should possibility be added for an organization to choose the information that they will want to send to Facebook?

4. What information can Civi get from Twitter and does this information differ by organizations? and if so should possibility for an organization to choose this information be provided or the extension should have a standard or information it gets from Twitter.

5. What information can Civi send to Twitter ? should the information that an organization send to Twitter be specific to that organization? if so, should possibility be added for an organization to choose the information that they will want to send to Twitter?

if we have all these things as per the platforms, then we can prioritize and start the implementation accordingly.

[swaugh]

Two things come to mind for me:  there may be privacy concerns about "pulling' info from one platform into the database without notifying the person who has generated this content (I'm in BC, Canada, we have very stringent privacy laws) that it is being stored, and what it might be used for.
Secondly, this is a great idea - we are just now working on having participants in our closed facebook support groups identified in CiviCRM as contacts belonging to these groups.  In CiviCRM, the group transforms into a special kind of online 'event', which group members 'register' for.

[xavier]

Hi,

I'm not sure you can pull anything out of FB without the consent of the person, eg. without the user allowing our app to access her data.

[achia]

In terms of scoping this part of the project, this is how far we are gone. Feed back at this point will help greatly.

Social Media Integration with CiviCRM II Scoping

Information to be gotten from Facebook

        Match the Civi contact with the Facebook account
   Check if the user likes the organization on Facebook
   Check if the user is writing on the organization wall and what they are writing
   Check if a user is liking or commenting on the organization’s post and their comments
   Get the number of friends for that organization on Facebook


Information to send to Facebook

   Post to a user’s wall automatically.


Information to be gotten from twitter


   Matching a user’s profile to its CiviCRM contact
   Check if a user is following the organization on twitter
   Check if a user is tweeting at the organization and possibly get the tweets
   Check if a user is sending direct messages to an organization on twitter and possibly getting the messages
   Check if a user is retweeting at the organization
   Get the number of followers the organization has on twitter
Information to be sent to twitter
   Automatic tweets to user about an event or fund raising goal or status

Other information to display at the Civi end
   The total number of followers or friends of the organization
   Highly active users

[JoeMurray]

Could you provide more information on how you intend to do the matching with Facebook and Twitter accounts, and if there is any cost to using their API's for this?

[Chris Burgess]

I'm not sure you can pull anything out of FB without the consent of the person, eg. without the user allowing our app to access her data.

For sure, and each platform will have its own (changing) rules about permissions, about what data (if any) we are permitted to store locally, and how it should be treated. Eg Facebook Platform Policy.

Worth reading through that list; some of the requirements will have implications for us (eg Only use friend data (including friends list) in the person’s experience in your app. - I wonder how this has been interpreted in other CRM platforms which implement network graphs based on Twitter/FB data? - and Don’t proxy, request or collect Facebook usernames or passwords., the username part of which surprised me).

@JoeMurray, I expect that contact matching will be much like CiviCRM's existing approach to start with. NationBuilder seems to import all SM "friends" and then treat them as contacts, deduping as best it can. With SM profiles there is additional information available, eg if I have your email and want to match against a likely Twitter profile, I can perhaps compare your email's gravatar(s) and Twitter avatar to confirm additional match. (TASK NOT IN CURRENT SCOPE!)

[emilyf]

Now that we have entered the official coding section of the project, we are looking for daily updates from the student detailing the following:

- What you are working on today
- What issues / roadblocks you are trying to overcome
- Any other questions you have where we can help

Please keep us updated as to your progress!

[Andrew Cormick-Dockery]

Not sure if you're still on the planning part of the project or whether the scope is already fixed, but one of my clients indicated that linking Civi Events with FB events so that FB RSVPs of known contacts are recorded in Civi would be, quote, an "key feature".

[achia]

Well, as per this part of the project, the scoping is done. Any other feature my come later when we finish those listed in the scope.

As per the project progress, we had discussions about the user experience and how we will develop the extension to function. some key decisions already made are as follows;

-Each organisation will create a Facebook and twitter Application and provide the credentials or the key strings to the extension
-The extension will use those credentials to perform interaction with the social media platform
-We have created a base application for Facebook and Tweeter. we will use the credentials as the base for development. We may need more key credentials for facebook and twitter apps for some few organizations using Civi so we can have some real use cases while developing.

At this point, the beginning of the project is our first block: We need to be able to match a Civi contact with its Facebook account and its tweeter handler. Once we are over this step, the rest will be less challenging. This functionality is the base of the whole project and we need best methods of doing it right.

We are open here for options and methods.

[achia]

Project State

We started this project off by scoping the project from the initial GSoC proposal submitted and came out with the manageble scope above. According this scope, the whole of this part of the project depends on what information that we can get from Facebook and Twitter so we can use any as a matching criteria for the civi friends to the civi contact.

We started off with Facebook looking at the API and the Facebook SDK v4 for php, more practically the Graph API and a lot has change from version 1 to version two of the Graph API in terms of permissions allowed to get data about a user's friends from facebook

The idea we were trying to implement was to get the emails of a user's friend (CiviCRM user)via the graph API and match with the emails of the civi contact table because we believe the email is unique and we may have a match keeping in mind the case that a friend of Civi may not use the same email for facebook and in CiviCRM.

Now according to the new developments on the API and Privacy policies in the Facebook platform, For us to get information about any CiviCRM friend such as the primary email, our app must request the user's permission and it is granted the permission before it can get that information. Also based on the friend's privacy settings, Facebook will allow you to get available data.

The only information that can be gotten by our extension (the app) that does not require the friends' permission is the user_id, name, and picture via the /me/taggable_friends(https://developers.facebook.com/docs/graph-api/reference/v2.0/user/taggable_friends) endpoint, and only can be used after a review of your app by Facebook.

With user permissions, we can get the following; public_profile, email, and the user _friends

Question: what will be the experience or interaction flow of the extension if we have to request the permission of a user before get his information that we can use to match his contact to his profile to the civi contact?

What we are doing now

we have written the srcipt that uses the API and the SDK to get friends ID, name, picture as per /me/taggable_friends(https://developers.facebook.com/docs/graph-api/reference/v2.0/user/taggable_friends) endpoint, and will submit for Facebook review today, so we can use it to get those information though we do not know how helpfull it will be in matching while awaiting suggestions from the community.

Our next step which will start by the end of today will be to request the user's permission to get their emails so we hope to get feedback from the community today so we could move forward.

suggestions will be helpful

 

[achia]

So at this stage this is what is happening

Facebook:

We started with Facebook to get friends information that can help us match the friends of the CiviCRM organization with the Contact in the CiviCRM organization database. According to the Facebook New API, We can only get a list of friends that the organization tagged without the friends' permission, inevitable freinds via these two API calls;

/me/taggable_friends  and /me/inviteable_friends

but we want all the friends of the organization and their profile information including the name and their email address from Facebook but we cannot get these information without the user's permission. so at this stage we have resolved to the extension requesting the login from the friend and requiring permission to get their email, we we get their email, we match their email from Facebook with that in the Civi database.

I halted with Facebook at that point waiting for the login feature to be completed from the other part of this project then I can move forward to getting information from Facebook


Twitter:

Twitter does not release the email address of any user so we resolved in getting the information we can get. we have succeeded in getting the following information from twitter;

`user_id`, `screen_name`, `name`, `profile_image_url`, `location`, `url`, `description`, `created_at`, `followers_count`, `friends_count`, `statuses_count`, `listed_count`, `protected`, `suspended`, `lang`, `last_tweet_date.

I will be using the name to match it with the firstname, lastname fields of the CiviCRM contact table

But on this, we have the situation to do the matching manually or automatically. Automatically we cannot check on mismatching contacts to twitter profiles. Manually, an administration can check if a particular civi contact matches the respective Twitter profile. We are still not clear on how this will work thus we will need input from the community at this stage to figure out the user experience of the extension concerning this aspect.