Queries Regarding Integrating With Google Analytics And A/B Testing (GSOC 2015)

[vishalagarwal3010]

I am a 3rd year undergraduate student pursuing Computer Science And Engineering at IIT-Bombay.
The GSOC project topic dealing with integrating Google Analytics thrills me the most.
Below I have stated what my understanding of the project topic is and how I intend to go about it. I would love to receive comments, suggestions and corrections regarding the same.

First Part of the project:
My Interpretation- Using UTM tags to determine the source of the visitor (link shared on social media, newsletter ,etc.)
In order to track we only need the Google Analytics generated tracking ID (UA-xxxxxxxx-x)
For example, when we are creating a new event using CiviEvent we could ask the user to manually input the tracking ID. Now we can simple add the "tracking script" to the page CiviEvent currently generates. Appropriate UTM tags can be appended to the urls which CiviCRM currently generates.

Second Part of the project:
My Interpretation- Getting Data from Google Analytics and storing in CiviCRM database
Google provides an API called the "Embed API" this can be used to display Google Analytics Reports from CiviCRM itself.
To get the data from Google Analytics we can use the "Core Reporting API (v3)". We can then store and process this data locally in whatever way we want. We can make queries to this API multiple times to obtain information about how many users registered out of the total number of users who visited the page (conversion ratio).

Third Part of the Project:
My Interpretation: Allow for A/B testing (as is done for emails in CiviCRM) for web pages
Using the same example of CiviEvent, when creating an event we could ask the user to input separate tacking ids to different versions of the web page (say a fixed number 3). Now the user could manually activate a particular version for a fixed time (say 3 days) and use the Embed API to see the performance of each. Now he could simply destroy the other versions (space efficiency) and publish the one with the best response.

Note: Use of the Embed API was not a part of the original project idea

Issues:
The number of tracking IDs are limited (100). I am not sure of their re-usability.
This will involve the user to additionally well maintain his Google Analytics account.

Thank You,
Vishal Agarwal 

[Owen]

Hi Vishal,

Thanks for posting. Some pointers on the below :

My Interpretation- Using UTM tags to determine the source of the visitor (link shared on social media, newsletter ,etc.)
In order to track we only need the Google Analytics generated tracking ID (UA-xxxxxxxx-x)
For example, when we are creating a new event using CiviEvent we could ask the user to manually input the tracking ID. Now we can simple add the "tracking script" to the page CiviEvent currently generates. Appropriate UTM tags can be appended to the urls which CiviCRM currently generates.

UTM tags are for adding campaign parameters to visits, for use when you are posting links to your site from external sources. E.G. Social media, emails or other websites, or from paid adverts like Google Adwords. They aren't used internally on the wesbite for linking from one page to the next.

In almost all cases a website has a single UA number, you wouldn't generate one per event page.

Google provides an API called the "Embed API" this can be used to display Google Analytics Reports from CiviCRM itself.
To get the data from Google Analytics we can use the "Core Reporting API (v3)". We can then store and process this data locally in whatever way we want. We can make queries to this API multiple times to obtain information about how many users registered out of the total number of users who visited the page (conversion ratio).

This is a good idea, using the api to find out visits to an event page or donation page and then calculating conversion rates would be a good addition, using the API a bit more you could see which sites/sources are referring traffic to the pages.

My Interpretation: Allow for A/B testing (as is done for emails in CiviCRM) for web pages
Using the same example of CiviEvent, when creating an event we could ask the user to input separate tacking ids to different versions of the web page (say a fixed number 3). Now the user could manually activate a particular version for a fixed time (say 3 days) and use the Embed API to see the performance of each. Now he could simply destroy the other versions (space efficiency) and publish the one with the best response.

See above RE tracking IDs, Most A/B frameworks will split traffic rather than doing time based tests. I.E 50% of users get version A and 50% get version B. Google Analytics doesn't offer A/B testing out of the box, have a look at Google Experiments https://developers.google.com/analytics/devguides/platform/experiments-overview

Adding a/b testing to forms would be a really good project.

Other project ideas to consider

• Add google ecommerce tracking to contribution/event/membership forms.
  
• Pull topline website analytics into a report or into Civisualise.

There's a good couple of seed ideas in your project proposal but I'd suggest you do a bit more reading up on how different sites practically use Google Analytics to help them improve their marketing.

Happy to help with any questions.

Cheers

Owen.

[vishalagarwal3010]

Hi Owen,

1)
Regarding you comment on UTM tags. I understand that UTM tags are used for adding campaign parameters. Say for eg. when an user clicks on the share on Facebook icon on an Event page, CiviCRM generates a URL:
http://wp45.demo.civicrm.org/event-page/?page=CiviCRM&q=civicrm/event/info&id=1&reset=1
For us to trace the source of the visitor we only need to append the UTM tags to this generated URL. For the example for URL above:
&utm_source=facebook&utm_medium=share&utm_campaign=fundRaiserEvent
Yes now I realize that UTM tags themselves can be used to differentiate between event pages so a single UA id is sufficient (Thanks for correcting me).

2)
Thanks for pointing out about A/B frameworks splitting traffic rather than time based tests. After referring to the link on Google Experiments you suggested and reading up on A/B testing, I found that Google Analytics actually allows you to do A/B testing. Well not A/B testing exactly but it allows you to run the "multi-armmed bandit experiment" (almost equivalent).
This link is a step by step tutorial to set up one: http://www.newmediacampaigns.com/blog/google-analytics-ab-split-test-tutorial

I will read up on the project ideas you suggested and get back to you. Thanks a lot for your comments. I would appreciate if you could comment on points 1) and 2) of this post too.

Thank You,
Vishal Agarwal     

[Owen]

Regarding you comment on UTM tags. I understand that UTM tags are used for adding campaign parameters. Say for eg. when an user clicks on the share on Facebook icon on an Event page, CiviCRM generates a URL:
http://wp45.demo.civicrm.org/event-page/?page=CiviCRM&q=civicrm/event/info&id=1&reset=1
For us to trace the source of the visitor we only need to append the UTM tags to this generated URL. For the example for URL above:
&utm_source=facebook&utm_medium=share&utm_campaign=fundRaiserEvent

Correct, one thing to think about though is that we can only add campaign parameters onto links that we control. I.E. when the organisation is placing the link somewhere themselves. However if the fundraising event was featured in a local newspaper and they linked to our page they wouldn't put any campaign parameters on the link. We would still want to measure this traffic though. So instead of using the dimension Campaign which would be missing we'd use the dimensions Medium and/or Source and the metrics Visitors or PageViews. Using the API you'd filter using the page URL http://wp45.demo.civicrm.org/event-page/?page=CiviCRM&q=civicrm/event/info&id=1&reset=1.

Have just looked at the Embed API, it looks good. Another possible project would be to integrate campaign tracking into CiviMail so that emails go out automatically appended with campaign parameters. The embed API could then bring back web stats into CiviMail reports. Similar to what MailChimp does.

Thanks for pointing out about A/B frameworks splitting traffic rather than time based tests. After referring to the link on Google Experiments you suggested and reading up on A/B testing, I found that Google Analytics actually allows you to do A/B testing. Well not A/B testing exactly but it allows you to run the "multi-armmed bandit experiment" (almost equivalent).
This link is a step by step tutorial to set up one: http://www.newmediacampaigns.com/blog/google-analytics-ab-split-test-tutorial

Multi arm is essentially a/b with better optimisation. Looks like the Experiments tool is now better integrated with Analytics. However as a workflow it's not great. Say we'd want to A/B test two versions of a CiviContribute form, one has a suggested donation about and one doesn't. The workflow above would require a user to set up two forms in Civi, go to Analytics paste in the URLS, generate the tracking code and then insert the script into the code of the form. This last step would be impossible for most Civi users as it requires a developer. For it to work I suspect you might need to use the Google Experiments API to generate a new test from the Civi interface and for Civi to handle inserting the Experiments script on the form (probably a hook, a Civi developer would be able to point you better) and for the reporting to be pulled back into the Civi interface.

I'd recommend having a thorough play in a Google Analytics account too, you'll get a better feel for the terminology and the common dimensions and metrics.

[xavier]

Hi Vishal,

They are js libraries that allow A/B testing, would be great if you could check some and see how it could fit.

For what I remember, you put the two versions of whatever you want to test in the page into a hidden part, and the js library displays one of the other.

I don't recall how you measure the result, but I assume it's either by sending an event to GA or by altering the utm tags of the links you want to track.

Talking about events, it might be interesting to track more events, eg. if people click on the radio buttons to change the donation amount or keep the default...

Last suggestion for integration with google analytics: instead of relying on civi to track the click and open, use GA to track who open (google analytics can generate 1x1 transparent pixels, aka webbug) or who clicks (using the utm tags), with the api, you can get the result back into civimail.

So the con is that you only track links to your own site and you can't anymore have a per user tracking (might be a good thing)

On the pro side: it's faster (no redirect) and less resource intensive

A sneaky side effect marketing feature: if you add the utm to all the links, including those on other sides and they use GA, they will get a new source of traffic civicrm mailing, kind of like it

X+

[Owen]

They are js libraries that allow A/B testing, would be great if you could check some and see how it could fit.

Could then an a/b testing framework for contributions/participants be delivered without Google Analytics? Just using split URL and info from the DB on conversion?

Scanning the Expermients API there's a fair bit that can be down with their JS https://developers.google.com/analytics/devguides/collection/analyticsjs/experiments

Alternative is not to use Experiment but to use custom variables in GA. We use Visual Website Optimizer (paid tool) which does this. Glancing at some stand alone JS and PHP they seem to do this. GA wouldn't handle significance testing then though/

Last suggestion for integration with google analytics: instead of relying on civi to track the click and open, use GA to track who open (google analytics can generate 1x1 transparent pixels, aka webbug) or who clicks (using the utm tags), with the api, you can get the result back into civimail.

Might some orgs prefer this as an additional extra rather than a replacement? I'd certainly prefer both rather than either, others I'm sure would prefer none and some not to 3rd party. Would be good though as it would give some more insight as to devices and operating systems which are opening emails.

[vishalagarwal3010]

Hey Owen,
  The suggestion about integrating campaign tacking into CiviMail sounds great.
  Your suggestion about creating a new test using the Google Analytics API and then inserting the tacking code into the form, was the way I thought I would go about implementing it. The user will be transparent from all this so this sounds good.
     
Thanks,
Vishal Agarwal

[vishalagarwal3010]

Hi Xavier,
  I checked out other js based libraries for A/B testing. It turns out that most of them are paid and have issues like the webpage would be blank for a few seconds before actually being redirected to the A or B version of the page. Googles Analytics Experiment API seems to be the best option. Are there any specific issues related to GA that would make it an inferior choice?

Yes I plan to use GA to track the flow of visitors and then use the Google Core Reporting API to get the data back into Civi. Thanks a lot for your comments 

Thank You,
Vishal Agarwal
 

[vishalagarwal3010]

Hi Everyone,
This is my current proposal feedback on the same would be greatly appreciated.

PERSONAL INFORMATION

First/Last Name: Vishal Agarwal
Email: vishalagarwal3010@gmail.com
School/University: IIT-Bombay
Graduation Date: 2016
Major/Focus: Computer Science And Engineering
Location: Mumbai
Timezone: GMT + 5:30
Drupal.org Profile: VishalAgarwal
IRC: I use a temporary account
CV/Resume URL: www.cse.iitb.ac.in/~vishala/resume.pdf 
Preferred time of day for virtual/video interview: IST after 10:00 pm
Open Source Experience: None

GSOC INFORMATION

If you have participated in Google Summer of Code in the past, please describe your participation:
No this is the first time I am participating.

Have you applied to GSoC in the past and are you applying to any other organizations this year? If so, please explain.
No this is the first time I am applying. CiviCRM is the only organization to which I am applying.

How many hours will you devote to your GSoC project each week? What are your other summer plans (vacation, this time of year isn't your summer and you are in class, etc)?
I will devote at least 35 hours per week towards completing my project. I do not have any other commitments. This time of the year we have our summer break.

CIVICRM INFORMATION

Have you registered an account at CiviCRM.org?
Yes I have. Username - VishalAgarwal

Have you even built a site with Drupal, WordPress, or Joomla?  (Please provide details)
No I have never built a site with Drupal, Wordpress or Joomla

Have you ever built a CiviCRM site or helped on a CiviCRM project? 
No I have never built a CiviCRM site nor have I helped on a CiviCRM project


Have you ever posted a questions to the CiviCRM Forums, JIRA, or GitHub?
Yes I have posted questions to the CiviCRM Forum.
http://forum.civicrm.org/index.php/topic,36090.0.html

Have you ever contributed code to CiviCRM?
No I have not contributed code to CiviCRM yet.

If you have not previously contributed to any open source projects, can you provide example code from school?
Code of the projects I have contributed to in the past can be found on github.
https://github.com/VishalAgarwal

TECHNICAL INFORMATION

Have you ever utilized IRC?
Yes, I have used it to communicate with Kurund Jalmi.

Have you ever worked with Git?
Yes, I use git regularly for my course projects and lab assignments. 

Question based on knowledge of git:
You just committed some code (and not pushed yet), but you realized there is a typo in commit message. How would you change it (please explain each step of the solution)?
I would use the command "git commit --amend -m "New commit message""

Question based on given 'almighty_function':
Given this function:

function gsoc_function($x, $y, $z) {
  if ($y != $z && $x == $y && $x == $z) {
    return "Success!";
  }
return "FAIL!"
}

Please provide set of values for $x, $y and $z for function to return "Success!". Explain your solution.
What, if anything would you change or fix in the function?
There is no set of values for $x, $y and $z for which the function would return "Success!".
For the function to return success we need $y and $z to be equal to $x and not equal to each other. This cannot be satisfied for any values of $x,$y and $z.
For the function to work we can remove any one of the three boolean expressions from the if condition. 

PROJECT INFORMATION

Which project idea sparks your interest and why?
The project involving integrating Google Analytics with CiviCRM and enabling CiviCRM to do A/B testing excites me the most. With more and more web pages being posted day by day organizations want to know more about their visitor flow. Statistically measuring the success of a campaign is a very important thing. Google Analytics provides wonderful API’s for the same.
Getting hands on experience with integrating Google Analytics with CiviCRM would be a wonderful opportunity for me.

DETAILED IMPLEMENTATION

Common portion for the discussion below:
Under Administer->System Settings->Miscellaneous, the user will have a field to fill in the UA ID generated by Google Analytics
Under Event->New Event or Event->Manage Event->Configure Event, the user will have a checkbox either to enable or disable web tracking for that page 

The project can be divided into four parts-
      1.  Use Of UTM Parameters:
There is a tool by Google called URL Builder which enables a user to generate an URL appended with appropriate UTM parameters. This is not very user friendly for non-developers. The idea is to create a more user friendly interface within CiviCRM for the same where all the events/contribution pages (which have tracking enabled) can be handled together
This will be done on a new screen. There will be a dropdown from which a user can select a particular event/contribution page. There will be buttons of the type “For Facebook”, “For Mailing”, “For Twitter”, etc. which will generate the URL after appending them with the appropriate UTM tags. The user can then use this URL
Secondly, on an event page under “Help spread the word” section there are buttons to share the event link on social media. The idea is to append the URL’s generated by these buttons with appropriate UTM tags

      2.  Web Tracking:
If the user chooses to enable tracking and he has entered information about the organizations UA ID. Google Analytics tracking code will be appended to the event webpage which CiviCRM currently generates

      3.  Tracking Report:
Google Analytics provides an API called the Embed API. This API would allow the user to view reports generated by Google Analytics from within CiviCRM itself. The Embed API handles user authentication itself. This can be added under Reports->Google Analytics

      4.  A/B Testing:
The Idea is to use the feature of “Experiments” in Google Analytics to enable A/B testing
Under Event->Manage Event->Configure Event, if the user chooses to enable A/B testing he/she will be provided with space to paste the tracking code generated by Google Analytics, this code will get appended to the webpage generated by CiviCRM. (Note he/she will have to create two versions of the page using Event->New Event and then paste tracking code into the form for both). Now the user is ready to set up the Experiment using Google Analytics.

The initial project ideas can be found on this link:
http://forum.civicrm.org/index.php/topic,36090.0.html

EXPECTED DELIVERABLES

1.Detailed blogpost on the plan and mockups for the project ( Week 4 )
2.Detailed blogpost on the status of the project and getting feedback from the community ( Week 8 )
3.Public demo for showcasing the work done and getting feedback ( Week 9 )
4.
Finally the code would be made available on github and merged with the civicrm-core
Detailed documentation of the implementation along with the current functionalities
Notes on the scope for further development and application

PROJECT TIMELINE

19 May and 26 May: Research Weeks
Get up to speed on AngularJS, in order to design the new interface for URL builder
Get up to speed on the implementation of CiviEvent and CiviContribution
Gaining deeper insights into the API’s provided by Google Analytics
Gathering information about popular naming conventions for UTM parameters

02 June: Design And Mockup
Based on the research create a wireframe for the URL builder UI
Create high level specifications for the project
 
09 June, 16 June,23 June and 30 June:
Blog post on CiviCRM summarizing the findings of the research and the specifications
First prototype of the new URL builder UI ,the Web Tracking Feature, the Reporting feature and the A/B testing feature
Minimal set of tests to ensure working of the prototypes

07 July, 14 July and 21 July:
Blog post on civicrm.org with details of the new version and requests of feedback from the Civi community
Deploy a public demo instance
Incorporating changes based on the feedback from early alpha testers

28 July:
Detailed Documentation along with the current functionalities
Tutorials on how to use the New Features
 
04 Aug:
Buffer Week

11 Aug:
Testing and bug fixing
Merging new code with CiviCRM core

18 Aug:
Wrapping up the project and buffer week
Final blogpost on learning, what has been achieved and scope for future development and application


Which aspect project idea do you see as the most difficult?
Getting up to speed on AngularJS, in order to design the new interface for URL builder
Getting up to speed on the implementation of CiviEvent and CiviContribution

Which aspect project idea do you see as the easiest?
Appending UTM parameters to the URL generated by the buttons to share the event link on social media.

Which portion of the project idea will you start with?
I will start with building the UI for the URL builder followed by overloading the URL’s generated by the share on social media buttons


 

[xavier]

Hi,

The aim is to let "normal" users benefit from google analytics and a/b testing without having them building the url. Check civimail, it has an option to alter the urls to track the clicks, it would be nice to do it but add the proper utm_x params automatically, and use GA api to fetch back the results and put them in the mailing report

Could you try civi and see if you can imagine other places we could automatically add these utm_ params and add your findings in this thread?

[vishalagarwal3010]

Hi Xavier,
Here are a few places where utm tags can be automatically appended:

• The link in the email body which is sent under "Tell A Friend"
• Share on social media buttons. (Share on Linkedin, Reccomend on Google Plus, Tweet ,etc.)
• Embedding the link (appended with UTM parameters) in the various reports that are generated like Event Income Summary Report and Top Donors Report. This might help in capturing post event traffic.

I will continue adding to this list as I discover more.

Thank You,
Vishal Agarwal

[vishalagarwal3010]

Hi Xavier,
  I am from India. I follow IST (GMT+5.30). Currently I have my end semester submissions going on. I am free after 6pm (IST).
We can talk on Skype, IRC, Google Hangout, Facebook etc. whichever you prefer. Please tell me a time when you will be free so that I can contact you.


Thank You,
Vishal Agarwal
(my gmail id vishalagarwal3010@gmail.com)

[xavier]

Hi,

Please add my account (xavier.dutoit) so we can see

[vishalagarwal3010]

Hi,
   "xavier.dutoit" is your gmail ID/Skype Username?

Thank You,
Vishal Agarwal

[vishalagarwal3010]

This is an update on my progress

• Successfully setup a full local copy of civiCRM (I used this to setup and enable the extension gsoc_candidate)
• Familiarized myself with the front end of civiCRM
• Started going through the codebase trying to understand the program flow

TODO::

• Fixing a mutually aggreable communication channel and time with my mentor
• Update my proposal based on research/discussion so far

Thank You,
Vishal Agarwal