Building The Conversation

Easy peasy A/B testing using Google Analytics

We were looking around for an A/B solution for TC to answer various questions such as “Does average time on site increase if related articles are on the top of an article or the bottom?”

Of the solutions we looked at, all seemed pretty complicated so we rolled our own using Google Analytics and it came out with just a few lines of code.

Segment by custom variables to compare outputs of test. CC BY-ND

The Approach

Google Analytics allows you to set custom variables of which you can have 5 per visitor at various levels (page/session or visitor). By setting custom variables using controlled randomisation we could then compare behaviours with users that were presented with different options.

Firstly every user needs a uid as cookie. In application_controller

before do
  unless cookies[:uid]
    cookies.permanent[:uid] = Digest::SHA1.hexdigest(rand.to_s)

Next in a helper add the following helpers:

def ab_test(test_name, *choices)
  choices = choices.any? ? choices : [true, false]
  get_choice(test_name, *choices)

def get_choice(test_name, *choices)
  user = cookies[:uid]
  choice_index = Digest::MD5.hexdigest('NaCl' + test_name.to_s + user).to_i(16) % choices.length

Then in your layout

if ab_test(:related_placement, "top", "bottom") == "top"
  # Render related articles at the top
if ab_test(:related_placement, "top", "bottom") == "bottom"
  # Render related articles at the bottom of the page

Finally before you call trackPageview with your google analytics code call:

_gaq.push(['_setCustomVar', 1, 'related_article_placement', '#{ab_test(:related_placement, "top", "bottom")}', 1])

This solution allowed us to easily test a couple of assumptions without deploying much additional tech. Using Google Analytics segmentation we could then slice and dice against any other metric we wanted to check.


Setting the custom variables needs to be done before you call trackPageview - the variables ride on trackPageview’s request.