CiviCRM Community Forums (archive)

*

News:

Have a question about CiviCRM?
Get it answered quickly at the new
CiviCRM Stack Exchange Q+A site

This forum was archived on 25 November 2017. Learn more.
How to get involved.
What to do if you think you've found a bug.



  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using CiviContribute (Moderator: Donald Lobo) »
  • PHP A/B testing framework
Pages: [1]

Author Topic: PHP A/B testing framework  (Read 1082 times)

wtheaker

  • I’m new here
  • *
  • Posts: 12
  • Karma: 1
  • Black and Pink
  • CiviCRM version: 4.5.4
  • CMS version: Drupal 7
PHP A/B testing framework
November 17, 2014, 12:45:55 pm
Has anyone used a PHP A/B testing framework on a contribution page? I was hoping to start out with some basic things, like testing various images, and tracking the results with Piwik (a free replacement for Google Analytics).

I looked into using phpA/B http://phpabtest.com/quick-start, but had issues using PHP in a smarty template. Is it possible to embed PHP code in smarty templates?

I've also heard good things about http://sixpack.seatgeek.com/ since it's language agnostic, but haven't used it.
Support CiviCRM by becoming a sustaining member.

Michael McAndrew

  • Forum Godess / God
  • I live on this forum
  • *****
  • Posts: 1274
  • Karma: 55
    • Third Sector Design
  • CiviCRM version: various
  • CMS version: Nearly always Drupal
  • MySQL version: 5.5
  • PHP version: 5.3
Re: PHP A/B testing framework
November 18, 2014, 03:08:12 am
Quote
Is it possible to embed PHP code in smarty templates?
Yes. I can't remember if CiviCRM disables it by default, but it is possible.  It's not a brilliant idea though and smarty is designed to stop you from doing that for a few reasons (security, seperation of different layers etc.)

Looking at that phpA/B site, my first thought was is there a smarty integration with phpA/B.  I then thought that it would be cool to have a more generic way to integrate that with CiviCRM since it seems like quite a powerful tool.
i.e. I reckon there is a way to integrate it, but including it at the smarty level is probably not the best way to do it.  Maybe via a call to a hook when the page/form is loading.

I suspect there are other ways to do AB testing on contribution pages that people have experimented with. I'll mention it to people and see if they have ideas to share.
« Last Edit: November 18, 2014, 03:20:07 am by Michael McAndrew »
Service providers: Grow your business, build your reputation and support CiviCRM. Become a partner today

Owen

  • I post occasionally
  • **
  • Posts: 83
  • Karma: 2
    • Leukaemia & Lymphoma Research
  • CiviCRM version: 4.3.4
  • CMS version: Drupal 7
  • MySQL version: -
  • PHP version: -
Re: PHP A/B testing framework
November 18, 2014, 05:32:28 am
>Has anyone used a PHP A/B testing framework on a contribution page? I was hoping to start out with some basic things, like testing various images, and tracking the results with Piwik (a free replacement for Google Analytics).

Does it need to be into Piwik and not Google Analytics?

Otherwise we use this and I can recommend it, can drop a JS tag in the them and then make tests using wysywyg GUI and not get down into code level.

https://vwo.com/?v=3

Alternatively could you take a step back and randomise the URL to two separate Civi donation forms and track success in Civi?

JoeMurray

  • Administrator
  • Ask me questions
  • *****
  • Posts: 578
  • Karma: 24
    • JMA Consulting
  • CiviCRM version: 4.4 and 4.5 (as of Nov 2014)
  • CMS version: Drupal, WordPress, Joomla
  • MySQL version: MySQL 5.5, 5.6, MariaDB 10.0 (as of Nov 2014)
Re: PHP A/B testing framework
November 18, 2014, 06:53:30 am
Important topic. What we really need to support is the reporting, and the feedback into action. Ideally, you want to continue A/B testing until there is statistically significant difference in results as defined by a particular level of confidence, and then switch to the better alternative automatically.

Owen, is there a reasonable way to poll Google Analytics, or do a callback from it?
Co-author of Using CiviCRM https://www.packtpub.com/using-civicrm/book

Hershel

  • Forum Godess / God
  • I’m (like) Lobo ;)
  • *****
  • Posts: 4640
  • Karma: 176
    • CiviHosting
  • CiviCRM version: Latest
  • CMS version: Mostly WordPress and Drupal
Re: PHP A/B testing framework
November 18, 2014, 07:12:17 am
Yes, of course Google Analytics provides an API:

https://developers.google.com/analytics/devguides/reporting/

However I can also understand that not everyone wants to use GA (the original poster possibly being such). There are JS-based tools also such as

http://blog.alexmaccaw.com/abba
http://www.optimisationbeacon.com/testing/using-cohorts-js-an-open-source-javascript-ab-mvt-library/

I haven't examined these in enough detail to determine how easy they would be to integrate with CiviCRM, but at least they are pure JS solutions so they might be easier than a PHP-based tool.
CiviHosting and CiviOnline -- The CiviCRM hosting experts, since 2007

See here for the official: What to do if you think you've found a bug.

Owen

  • I post occasionally
  • **
  • Posts: 83
  • Karma: 2
    • Leukaemia & Lymphoma Research
  • CiviCRM version: 4.3.4
  • CMS version: Drupal 7
  • MySQL version: -
  • PHP version: -
Re: PHP A/B testing framework
November 18, 2014, 07:26:38 am
>Important topic. What we really need to support is the reporting, and the feedback into action. Ideally, you want to continue A/B testing until there is statistically significant difference in results as defined by a particular level of confidence, and then switch to the better alternative automatically.

Agreed (VWO the tool I posted below uses significance testing https://vwo.com/knowledge/calculating-ctbo)

>Owen, is there a reasonable way to poll Google Analytics, or do a callback from it?

It's got a good API, would just need to make sure the ecommerce tracking data went over in the right way in the first place. There's enough space for variables in the GA system for it but for example you'd likely want Campaign, Contribution form ID, a test ID and a variation ID.

>However I can also understand that not everyone wants to use GA (the original poster possibly being such). There are JS-based tools also such as

GA has understandably come up against some resistance from sectors of Civi community. JS/PHP and built into Civi would be the ultimate but that seems like more dev?


A split URL and a results page in civi compare #of donations or total amount donated seems to me to strike a balance between privacy and ease of implementation.

example.com/civicrm/donateredirect

Sends 50% to /civicrm/contribute/transact?reset=1&id=TEST1
the other to /civicrm/contribute/transact?reset=1&id=TEST2

Within Civi is a page that compares TEST1 to TEST2 and changes the redirect to one of them once significance on pre established criteria has been reached.

JoeMurray

  • Administrator
  • Ask me questions
  • *****
  • Posts: 578
  • Karma: 24
    • JMA Consulting
  • CiviCRM version: 4.4 and 4.5 (as of Nov 2014)
  • CMS version: Drupal, WordPress, Joomla
  • MySQL version: MySQL 5.5, 5.6, MariaDB 10.0 (as of Nov 2014)
Re: PHP A/B testing framework
November 18, 2014, 07:32:35 am
Of course Google provides an API. But looking at it a bit suggests there is no callback mechanism, so it would unfortunately have to be polled.

One thing I would expect they could do but haven't checked for lately is statistical significance on an A/B test. This page suggests they don't even have that: http://designmodo.com/ab-testing-google-analytics/

Optimize.ly is a paid service that seems to be a leader in this space.

Would be a great addition to CiviCRM to see this. A/B testing for CiviMail is expected in 4.6. It came out of a GSoC project, and Tim is reworking extensively I believe.
Co-author of Using CiviCRM https://www.packtpub.com/using-civicrm/book

JoeMurray

  • Administrator
  • Ask me questions
  • *****
  • Posts: 578
  • Karma: 24
    • JMA Consulting
  • CiviCRM version: 4.4 and 4.5 (as of Nov 2014)
  • CMS version: Drupal, WordPress, Joomla
  • MySQL version: MySQL 5.5, 5.6, MariaDB 10.0 (as of Nov 2014)
Re: PHP A/B testing framework
November 18, 2014, 07:36:11 am
@Owen: I like your thinking on how this could be implemented in terms of a URL.

Perhaps a UX element like "Make an A/B test for this Contribution/Event... Page" could create a copy of the existing page using existing code, and set up other stuff behind the scenes as required.
Co-author of Using CiviCRM https://www.packtpub.com/using-civicrm/book

Owen

  • I post occasionally
  • **
  • Posts: 83
  • Karma: 2
    • Leukaemia & Lymphoma Research
  • CiviCRM version: 4.3.4
  • CMS version: Drupal 7
  • MySQL version: -
  • PHP version: -
Re: PHP A/B testing framework
November 18, 2014, 07:41:27 am
>Perhaps a UX element like "Make an A/B test for this Contribution/Event... Page" could create a copy of the existing page using existing code, and set up other stuff behind the scenes as required.

I'd normally avoid doing split URL as it's not great for web analytics but in the case of Civi forms it makes a bit more sense and would ensure portability across CMS. Might be able to fit into GSoC code as well?

Pages: [1]
  • CiviCRM Community Forums (archive) »
  • Old sections (read-only, deprecated) »
  • Support »
  • Using CiviCRM »
  • Using CiviContribute (Moderator: Donald Lobo) »
  • PHP A/B testing framework

This forum was archived on 2017-11-26.