Easy peasy A/B testing using Google Analytics

We were looking around for an A/B solution for TC to answer various questions such as “Does average time on site increase if related articles are on the top of an article or the bottom?”

Of the solutions we looked at, all seemed pretty complicated so we rolled our own using Google Analytics and it came out with just a few lines of code.

The Approach

Google Analytics allows you to set custom variables of which you can have 5 per visitor at various levels (page/session or visitor). By setting custom variables using controlled randomisation we could then compare behaviours with users that were presented with different options.

Firstly every user needs a uid as cookie. In application_controller

before do
  unless cookies[:uid]
    cookies.permanent[:uid] = Digest::SHA1.hexdigest(rand.to_s)
  end
end

Next in a helper add the following helpers:

def ab_test(test_name, *choices)
  choices = choices.any? ? choices : [true, false]
  get_choice(test_name, *choices)
end

def get_choice(test_name, *choices)
  user = cookies[:uid]
  choice_index = Digest::MD5.hexdigest('NaCl' + test_name.to_s + user).to_i(16) % choices.length
  choices[choice_index]
end

Then in your layout

if ab_test(:related_placement, "top", "bottom") == "top"
  # Render related articles at the top
end
...
if ab_test(:related_placement, "top", "bottom") == "bottom"
  # Render related articles at the bottom of the page
end

Finally before you call trackPageview with your google analytics code call:

_gaq.push(['_setCustomVar', 1, 'related_article_placement', '#{ab_test(:related_placement, "top", "bottom")}', 1])

This solution allowed us to easily test a couple of assumptions without deploying much additional tech. Using Google Analytics segmentation we could then slice and dice against any other metric we wanted to check.

Gotchas

Setting the custom variables needs to be done before you call trackPageview - the variables ride on trackPageview’s request.