CiviCRM Community Forums (archive)

Hi, I'm Aditya Nambiar a second year undergraduate student studying CSE at IIT Bombay . There was a seminar held at   IITB regarding GSOC. I came across CiviCRM and heard that they have a core team based in Bombay. I met Kurund Jalmi and Donald Lobo core team members of CiviCRM to discuss about Visualization for Mailing stats and A/B testing for CiviMail.

CiviMail is an extremely important component of CiviCRM. A/B testing is a relatively recent, highly effective, cost-efficient strategy for improving the conversion rate, and since CiviCRM is specifically designed for the needs of NGO and advocacy groups, and serves as an association management system it is also a must for it.
CiviCRM lacks this feature and it is essential for any mail blasting application. Also it is a commonly requested feature on the community.

Here is my plan:

Week 1-2: Research weeks:
1: Get up to speed on d3 ( and may be filterjs )
2: Get up to speed on civi mail functionality
3: Learn civicrm REST API.

Week 3: Design and Usability mock ups
1.Create wire frame for A/B testing and reporting.
2.Create specifications at a high level.

Week 4-6:
1. Blog post on civicrm.org summarizing findings of research and  the specification
2. Work on reporting.
3. Incorporate D3 (and FilterJS) into Civi reporting framework.
   
Week 7-9:
1. Bug fixes and improvements for reporting
2. Deploy reporting on demo instance
3. A blog post with details of the new version and requests of feedback from the community.
4. Implement an A/B testing screens and functionality.

Week 10:
1. Document work done.
2. Integrate with CiviMail-blast UI project.
3. Incorporate visualization in other CiviCRM reports.

Week 11:
1. Testing and bug fixes.
2. Merge into CiviCRM master.

Week 12:
1. Project wrap up and buffer week.
2. Final blog post

Any feedback and advice would be highly appreciated.

Thanks
Aditya

to clarify:

a. the focus of the project is A/B testing on mailings only (this will be combined with the new mail blast UI)

b. adding data visualization to make the mailing reports more visual and hence easily comparable across multiple axis is a smaller sub-project within the main project

if we are done ahead of schedule, we can always increase scope.Its nice to have a target that can be met

yes, would be great if aditya and sid can recruit a few more CS students from the school

lobo

How do you intend to link the data from the CiviMail to the contributions/ other actions? There is an extension here - https://civicrm.org/extensions/extended-mailing-stats that simply looks at contributions made within a certain time period by mail recipients after the mailing & you should probably take a look at that in your R&D phase.

A-B testing has been a heavily requested component. The guys at Wikimedia foundation have done extension A-B testing on their banners & the results are somewhere in the public domain - not sure if that would be a useful thing to link to in your intro?

Integration with Google Analytics might be an idea to explore.

Google Analytics has an API which could pull back into a mailing reports. Mailchimp has a nice auto append tracking on links feature.

Actions and goals that are in Google Analytics could be used for reporting and/or the measurement for an A/B test. In it's simplest form this could has number setting up a goal in GA like :

"Reached a transaction thankyou page" and has come from a link from the email which contains URL tracking [linktodonationpage]?utm_Source=Civimail&utm_medium=email&utm_campaign=Marchnewsletter

If an org has GA ecommerce tracking implemented this could go down to the sum total of contributions, not sure how many orgs have implemented this though?

N.B Many orgs using Civi will be using Google Analytics, some will be opposed to it in principle. If you consider it a good idea and if it's in scope having this switchable would be a good idea.

Instead of something switchable, get that as an extension and hook it in (or inject the dependence like the cool kinds seems to do these days) the rewrite url function when they are clicked (track url feature).

@Aditya don't panic, if it goes into every direction, it just means you have a lot in the community that are really liking your project. Stay focused, ignore most and you should be fine

Wow this really sounds awesome! And is probably something we also need for the donor journeys we are setting up with CiviCRM.

About the discussion of integrating with Google Analytics and mailings also take a look at an open source website analytic tool PiWik: http://piwik.org/

Hi, please find below my current proposal. Feedback regarding the proposal would be greatly appreciated.

Thanks
Aditya Nambiar


Application for Google Summer Of Code 2014: CIVICRM –MailBlasting

Personal Information

Name : Aditya Nambiar
Email : aditya.nambiar007@gmail.com
University : IIT Bombay
Graduation Date : 2016
Major : Computer Science & Engineering
Location : Mumbai
Timezone : +5:30 GMT
Drupal.org Profile : aditya.nambiar
IRC :
Resume : http://www.cse.iitb.ac.in/~adityan/Resume.pdf
Open Source Experience: I have made a few chrome-plugins.

I am an undergraduate student studying Computer  Science & Engineering at the Indian Institute Of Technology , Bombay. I have been programming since I was 10 years old and have a passion for programming. I am an expert C++ programmer and I am very active in competitive programming. I have participated in several Hackathon competitions held on campus such as those conducted by Yahoo , Microsoft and also  won an Hackathon! I have been involved in the development of several websites such as those of Techfest IIT Bombay (http://www.techfest.org/), which we created using Code Igniter, a PHP framework.
During my time at IIT I have learned several programming languages such as Scheme, Python, Java and Bash and have also done several course related projects the details of which are available in my resume.
Apart from programming, I have a very good academic standing. I have secured an All India Rank of 97 among half a million candidates (top .02%) in IIT-JEE entrance exam to get into the department of my choice. I have received several scholarships such as the KVPY and NTSE from the Govt. Of India. I have also secured several top ranks in several competitive examination  (All India rank 27 in National Cyber Olympiad, International Rank 5 in Maths Olympiad etc).






GSOC Information

If you have participated in Google Summer of Code in the past, please describe your participation:
None.

Have you applied to GSoC in the past and are you applying to any other organizations this year? If so, please explain.
None.

How many hours will you devote to your GSoC project each week? What are your other summer plans (vacation, this time of year isn't your summer and you are in class, etc)?
I am very enthusiastic about my GSOC project. I am ready to devote at least 40 hours a week. I do not have any other summer vacation plans and would like to invest all my time in GSOC.

CIVICRM INFORMATION

Have you registered an account at CiviCRM.org?
Yes, my user name is – aditya.nambiar

Have you even built a site with Drupal, WordPress, or Joomla?  (Please provide details)
No

Have you ever built a CiviCRM site or helped on a CiviCRM project?
No

Have you ever posted a questions to the CiviCRM Forums, JIRA, or GitHub?
Yes, http://forum.civicrm.org/index.php/topic,31984.0.html

Have you ever contributed code to CiviCRM?
No

If you have not previously contributed to any open source projects, can you provide example code from school?

Developed a windows app, which can seamlessly convert normal text into mathematical formulas and enable users to create graphics using Paint like app on the go. I wrote the code to render a canvas, which is a paint like application in the notepad enabling the user to create creative graphics in javascript . This Windows app secured the first place at the Microsoft Hackathon. Github link-  https://github.com/Manishearth/Kapi

Technical Information

Have you ever utilized IRC?
Yes

Have you ever worked with Git?

Yes, I have learnt SVN and Git for version control. I have used Git  for several projects some of them include the classroom notepad app and also a project on creating a complex machine using BOX2D a physics simulation engine.

Question based on knowledge of git:
You just committed some code (and not pushed yet), but you realized there is a typo in commit message. How would you change it (please explain each step of the solution)?

 git  commit –-append –m  “New commit Message”

Question based on given 'almighty_function':
The function will always return “FAIL” since for it to return “Success!” x must be equal to y and x must be equal to z by transitivity y would be equal to z . However in the “if condition” y also must not be equal to z!
One of the 3 conditions will have to be removed for any possibility of returning “Success!”

PROJECT INFORMATION

Which project sparks your interest and why?

A/B testing and visual reporting for CiviMail

About The Project

A/B testing is a form of testing that allows an organization to determine which mail will be more effective when sent to their mailing list. This involves sending the email to smaller subsets and using data and statistics to pick the winning email which will be sent to the rest of the group. A/B testing is standard and recommended practice to improve your results from any mass mailing and has been used by Wikimedia, Obama campaign and others very effectively.

This project aims at creating an A/B testing tool for CiviMail so all users of CiviMail can improve their response rate for all their mailings. To make this easier on the end user this project will also incorporate graphical reporting (such as using charts, graphs etc.) of the mailing statistics. This will allow the admin to interpret and monitor the results of their mailing campaign and hence make a decision based on data.

I plan to use AngularJS, D3JS and CrossFilter ( may be ) in implementing the project.

The admin will have the option of which part of the mail he/she wishes to test: Subject, Date/Time of sending ,or the mailing body. We will also set smart defaults for A/B testing.

Admin will analyze the results and making a decision based on his choice. The interface would be intuitive and easy to use. Also results shall include confidence intervals of how good is the result and how much it could be trusted. This can be computed by using sample size and response rate.


If time permits
•   The app will have the option of choosing what the admin would like to optimize. Options such as Open Rate, Number of clicks on a particular link , number of contributions generated through the mailing etc. could be provided.
•   I would also like to integrate it with Google Analytics so as to make the results more informative and helpful.
•   Two options will be provided for A/B testing, automated and manual. In the automated option the mail which proved to be better among the two sets (using an admin chosen criteria) shall be automatically sent to the remainder set.

Benefits

CiviMail is an extremely important component of CiviCRM.
A/B testing is a highly effective, cost-efficient strategy for improving the conversion rate, and since CiviCRM is specifically designed for the needs of NGO and advocacy groups, and serves as an association management system, A/B testing is a must for it.
CiviCRM lacks this feature and it is essential for any mail blasting application. Also it is a commonly requested feature from the community.

Expected Deliverables:

•   Blog Post on week 4 and 9
•   Setup public demo in week 8
•   Final code on github

Week 1-2: Research weeks
1.   Get up to speed on AngularJS, D3 and CrossFilterJS
2.   Get up to speed on CiviMail functionality
3.   Learn CiviCRM REST API.
4.   Research competitors design and usability

Week 3: Design and Usability mock-ups
1.   Create wire frame for A/B testing and reporting.
2.   Create specifications at a high level.

Week 4-6:
1.   Blog post on civicrm.org summarizing findings of research and also the specifications
2.   Work on reporting.
3.   Incorporate D3JS and CrossFilterJS into Civi reporting framework.
   
Week 7-9:
1.   Bug fixes and improvements for reporting
2.   Deploy reporting on demo instance
3.   A blog post with details of the new version and requests of feedback from the community.
4.   Implement an A/B testing screens and functionality.

Week 10:
1.   Document work done.
2.   Integrate with CiviMail-blast UI project.
3.   Incorporate visualization in other CiviCRM reports.

Week 11:
1.   Testing and bug fixes.
2.   Merge into CiviCRM master.

Week 12:
1.   Project wrap up and buffer week.
2.   Final blog post

Potential Mentors: Kurund Jalmi

Which aspect project idea do you see as the most difficult?

Getting up to speed with AngularJS for completion of a complex project. Getting up to speed with CiviCRM, CiviCRM API.

Which aspect project idea do you see as the easiest?

Weeks 1/2 on Research and UI mockups