CiviCRM Community Forums (archive)

I am extremely interested in contributing to CivicRM, and would love to do a mentored project with you guys for Google Summer of Code 2015. My specific project idea (combining the ideas from Xavier and Jaap) is to integrate IBM's Watson with CivicRM. Watson is the supercomputer developed by IBM to compete on Jeopardy! IBM has recently released a few interesting APIs for Watson. One in particular that is usable is a linguistic analysis API. Once integrated, CivicRM users can input text (emails, memos, books, etc) and Watson will generate a character analysis of the person who wrote the text.

My thought is CivicRM users can use this character analysis to better respond and interact with their clients by understand what type of person their client is.  More information can be found here: http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/user-modeling.html

I think using IBM Watson's APIs would be a great way to get a lot of data. However, I also believe that one of the end deliverables should be documentation explaining what kind of data we are getting and how it can be used for the future.

From my initial research, Watson's linguistic analysis seems to be limited to identification by rudimentary character types. My initial thought process is to create a guide that explains how to interact with clients that meet specific character types; however, I would need to conduct a fairly significant amount of research to create this guide. 

I would love to hear the community's thoughts on the matter. What avenue do you guys think is the best way to proceed, and how would you guys like to see this project develop?  Thank you, and I really really look forward to hearing back from everyone.

Hello again everyone!

I have done some additional research on IBM Watson's User Modeling API. Here are some key takeaways:
 
Watson gives us a lots of data back when conduct user analysis. Most of this data is not that usable; however, the top 2-3 key traits are usually pretty telling about a person's characteristics. An individual  inputted two years worth of "The Lord of the Rings" related twitter data into Watson. The top characteristics it spit out were "Authority Challenging", "Intellectual", & "Imaginative" with the top need being "Self-Expression". These seem pretty accurate with the general perception of the LOTR community.

Source : https://vinmisra.github.io/2014/12/19/gollum-is-a-pretty-vulnerable-guy/

While I don't yet know exactly how accurate Watson's User Modeling is I do know that it is quite consistent. Another individual used Watson to do an analysis of of several different sub-reddits.  Sub-reddits that scored highly for "Assertiveness" scored lowly for "Cautiousness", and the lists were not inverses of each other so we know that different personality facets have different metrics. 

Source: http://imgur.com/a/JNkse

My overall goal for this project is to create an extension to CivicRM that basically lets users craft messages/press releases/conversations that resonate better with their clients and/or consumers. I understand that this is a very lofty goal, so I would like help from the community in breaking down this project into more byte-sized (programming humor) chucks. What is it exactly that the community wants? What is a deliverable I can have done by the end of the summer that the community would be happy with?

Towards this end, I actually think that another one of the APIs that IBM released would be a better fit for us in this goal . The API I am talking about is the "Message Resonance" API. Basically you have a target feed (the two feeds for the demo are the "Big Data/Analytics" and "Cloud Computing" communities). When you craft a message, Watson will go through the message phrase by phrase. It will then assign a score to each phrase to evaluate it's impact to people in the target feed. I will have to do more research, but this seems like a much more usable API for the goal at hand.

I look forward to hearing everyone's thoughts!

Couldn't it be used for other content too (eg when writing an event description, a thank you email...)?

What I'm not use I understand is how watson would qualify the audience of a specific civicrm install: we have very little content written by our contacts, so how can we modelise that they are authority challenging vs. intelectual vs. ...?

FYI, I have an attentive.ly integration that pulls in recent posts from social media accounts linked to the user's email and/or twitter accounts.

My own interest in Watson was trying to get it to recommend appropriate actions to help contacts climb an organization's ladder of engagement. So for organizations with lots of data it would be able to determine that people who have volunteers 2x in last year and who are interested in a particular issue and who have donated more than $500 in total would be likely to respond to a solicitation on that issue. Not terribly crisp, but the main idea would be for Watson to use datamining techniques to recommend actions to the CiviCRM admin to optimize things like donations, online actions like petition signing, and so on.

I think you have a good approach. And I love the fact that you want to fix a few bugs to get familiair with the CiviCRM codebase. It shows to me you are doing the right things.
Probably you could ask in a separate thread which bugs you could solve.

If you have participated in Google Summer of Code in the past, please describe your participation:
I have never participated in Google Summer of Code before.

Have you applied to GSoC in the past and are you applying to any other organizations this year? If so, please explain.
I have never applied to Google Summer of code before. At this point, CivicRM is the only organization I am planning to apply to.

How many hours will you devote to your GSoC project each week? What are your other summer plans (vacation, this time of year isn't your summer and you are in class, etc)?
I plan to treat this like full-time job, and as such I plan to contribute around 40 hours a week towards CivicRM. I am currently on a Co-Op with the Clorox Company, and that will last until June.  Coding “starts” on May 25th, so for the first two weeks I’ll have to manage both. I am trying to mitigate this by starting early. I started collaborating with the community and doing lots of research about my project idea (IBM Watson Integration) in Late January/ Early February, almost two months before applications even started. I plan to continue working on this project in conjunction with my Co-Op until my Co-Op ends. There is a one month built in “community bonding period” in the Google Summer of Code experience. Since I started so early, I think by the time this period starts I will already have a lot of experience with the community.  This is not to say that I won’t want to get to know the community better, but I think I will be at a point in my project where I can both code and get to know the community better. That effectively gives me almost another month of “programming and milestone time”.  I think I have proven that I will be able to manage my Internship and GSOC at the same time for a few weeks because all of my preliminary research has been conducted while employed full time. I have put at least 15-20 hours a week into this research itself. 


Have you registered an account at CiviCRM.org?
I have. It is oshivum

Have you even built a site with Drupal, WordPress, or Joomla?  (Please provide details)
I have never built a Drupal site from scratch; however, I did work in my University’s College of Information Science Technology (IST). The specific department I worked in supported all online courses for my university’s IST department. My university is one of the largest in the United States, so the online degree platform is used by thousands of students and professors. In that department we primarily worked with Drupal sites, so I do have experience manipulating Drupal Sites.  Additionally, I worked for a company called Blue Mountain Quality Resources where I fully developed their client user portal using a CMS called Dot Net Nuke which is based of .Net. (this is the technology they wanted me to use, I’d much rather have used Joomla)

Have you ever built a CiviCRM site or helped on a CiviCRM project? 
Unfortunately, I have not.

Have you ever posted a questions to the CiviCRM Forums, JIRA, or GitHub?
Yes I have. The Forums has been my main source of communication at this moment.

Have you ever contributed code to CiviCRM?
I have not YET, but that will hopefully soon change. I am currently working on this bug: https://issues.civicrm.org/jira/browse/CRM-16029
 

Have you ever utilized IRC?
I have

Have you ever worked with Git? 
I have used git; however, my experience is limited. I have enrolled in online basic class to further my knowledge of git.


Which project idea sparks your interest and why? IBM Watson Integration/Extension:
This project really sparks my interest because it involves two separate disciplines I am interested in: Sales and programming. In Sales, the consumers are king, and you are always trying to find better ways to appeal to them. With this project we are essentially creating a way to understand the consumer better with the Personality Insights API and a way to communicate with the consumer better with the Message Resonance API.

Treating this project as a real proposal, provide your implementation plan with as much detail as possible such as weekly time breakdowns, methods of mentor communication, project management, and when to expect specific results/deliverables.
Weekly Breakdowns: Before coding actually starts I plan to do even more research about Watson. I will be getting deeply rooted into the IBM programming community so that when I inevitably need it, I’ll be able to get help quickly. Once programming starts, I will be working on a 9:00 AM - 5:00 PM schedule Mon - Fri.

Mentor Communication: I will be in constant communication with Jaap. I have already been communicating frequently with Jaap, but I feel that communication will only increase in frequency once coding begins. I plan to have a 30 min - 1 hr google hangout meeting with Jaap once per week. We have already had a meeting this way, and I think it went quite successfully. Additionally, at the beginning of each week, I will post a list of short-term deliverables that I want to get done this week. These short-term goals will be aligned with the overall timeline of the project. At the end of each week, I will post a weekly summary. This summary will detail which of my goals I accomplished, problems I did not anticipate, and general questions for Jaap/The Community. I plan to have the google hangout between Tues - Thurs, so that Jaap essentially knows what I am doing at all times of the week.

Project Management: Last year, GSOC students created a forum post to present their day to day activity. I actually believe that day to day posting isn’t very engaging. I plan to post, at a minimum, twice a week. At the beginning of the week I will post my plan for the week, and, at the end of the week, I will write a weekly summary detailing what I accomplished that week. This summary will include any milestones that were reached and any changes to the overall timeline and/or project plan. I will still be on the IRC channel daily, so daily communication will occur. I just believe that a few informational posts a week will inform the community on what I am doing better than a 1-2 line comment each day. Please let me know if you’d rather have day by day posts. This can be done on request.

When to expect specific deliverables: Please see below

Detailed Description (only if this is a new idea, otherwise if there is a description online, please provide the URL):
http://wiki.civicrm.org/confluence/display/CRM/Google+Summer+of+Code+-+2015

By discussing with various members of the CivicRM communit, we have been able to evolve this idea substantially since it appeared on the list above. The goal is to create two extensions to CivicRM that would allow CivicRM users to utilize IBM Watson’s services. 

One of the extensions would connect to the “Personality Insights” API. This API allows for user modeling and is in a general availability release version. The second extension will connect to the “Message Resonance” API. This API allows users to craft messages that resonate better with a target audience.

We’ve determined that the “Message Resonance” API is the one that the community would appreciate more; however, this API is still in it’s BETA version. The way we structured the project, we will have a completed extension with a general availability version API, and we will have detailed documentation on how to create an extension with the Watson Service. This way, we can implement the Message Resonance API as soon as it comes out of BETA.

Expected Deliverables: (list the main items that you will deliver be during the program)
There will be two (2) major deliverables. The first major deliverable is an extension to CivicRM that allows users to conduct user modeling of clients; this will use the “Personality Insights” API. The sub-deliverables for this part of my project include: A private instance of Watson, A Watson Application that runs off this private instance, and an extension that allows us to use this application directly from CivicRM.
 
Besides this extension, I plan to create extensive documentation detailing how to create an extension that uses the IBM Watson service. This documentation will include everything I do on both the IBM Bluemix side as well as the native coding I do for CivicRM. 

A stretch goal of mine is to also create another extension that utilizes the “Message Resonance” API, but I am likely to only get to this work after the GSOC timeperiod.


19 May: Begin work on creating a private instance of Watson
26 May: Create a private installation of Watson
02 June: Being work on a rudimentary Watson application
09 June:Create a rudimentary Watson application
16 June: Begin work on an extension that works with the Watson application
23 June: Mid term submission / Create an extension that works with the Watson application
30 June: Work on finalizing the Watson Application
07 July: Continue working on finalizing the Watson application
14 July: Finalize Watson application
21 July: Fix up extension to make sure it works with the new Watson application
28 July: User testing and debugging
04 Aug: User testing and debugging
11 Aug: Wrap up / Nothing Planned for the Sake of Scope Creep
18 Aug: Final submission / Nothing Planned for the Sake of Scope Creep

Potential Mentors (optional):
Jaap Jansma is the confirmed mentor. Xavier Dutoit is a potential mentor

Which aspect project idea do you see as the most difficult?
I think the most difficult part of this project will be working with the Watson API. Watson is relatively new, and while the community is bustling, there may be some bugs I’ll face that will take a fair amount of troubleshooting to overcome.

Which aspect project idea do you see as the easiest?
I think the easiest part of this project is actually going to be creating the CivicRM extension and linking it to the Watson API. While this may be the most technically difficult part of the project, my mentor Jaap has done this many times and should be able to provide a good amount of guidance.

Which portion of the project idea will you start with?
I will start with the Watson side. I will create a rudimentary Watson application. I will then create an extension that harnesses the power of this Watson Application. Next, I will finish out the Watson Application. Lastly, I will do user testing to make sure the extension works seamlessly.

Hey JoeMurray, thank you for the feedback! It is much appreciated.

The minimum amount of data to run this application is 1000 words of text. The documentation recommends 2000 words for meaningful results. The average word has about 4.5 characters and the average tweet has about 65-70 characters, so we would need to have ~70 tweets to run the application and about 140-150 tweets worth of data to get truly meaningful results. I think these figures are quite reasonable and achievable.

Furthermore, the one thing I picked up during my research was that Watson is quite consistent whether you analyzing 20k characters worth of data or 200k characters worth of data . While the level of certainty increases with more data, the results regarding the key personality facets Watson is measuring stay relatively similar. Based on this observation, I would say that after 2000 words of data, you reach point of diminishing returns quite quickly. Unfortunately, since Watson is such a new service, I cannot back this observation with proven research; however, I will try to document my testing in the future.

Additionally, IBM actually includes a fairly detailed guide on what the results of the Personality Insights API really mean and how to best use them. You can find this guide here at: https://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/doc/systemuapi/#outputBigFive

Lastly, I have three questions for you Mr. JoeMurray:

• Could you give me some more details about your attentive.ly integration. Knowing exactly how it works will allow me to better plan my project during this brainstorming stage.
• It seems that your attentive.ly integration will be a major tool that I will use to aggregate data for this project. I assume that, since I will be using this tool so much, I will be coming to you for help quite often. For this reason, would you consider signing up to be a co-mentor for this project?
• Is there anything else I can do to make you, and the rest of the developer community, feel more comfortable about my project proposal and GSOC candidacy?

Sources:
http://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/personality-insights.html
http://www.quora.com/What-is-the-average-length-of-a-tweet
https://www.flickr.com/photos/hepwori/6732189421/in/photostream
http://en.wikipedia.org/wiki/Wikipedia:Size_comparisons

As always, thank you so much for your feedback and support.

Your proposal is very good. One thought tough, you say about the project information you like the combining of sales and programming. I don't understand what you mean by that. How does sales come into the project?