I just finished reading  the new Extreme Optimization report from Matt Gallant and it is packed with both excellent testing methodology and case studies.  If you are in doubt about the power of split testing, then this is a report you must read.  If you are a testing geek like me, you’ve probably already left this blog and started reading

In his report, Matt explains how even a newbie can beat the pants off a pro copywriter, just by using testing as a ’secret weapon.’

Here’s a shocking fact that most copywriters never talk about (even most of the “big names”): They are GUESSING 95% of the time. Now, in their defence they are making
educated guesses. They’ve studied other copywriters and they’ve put their chops to the test and created many winners. However, it doesn’t mean that a newbie can’t get into a market and clean their clocks. Especially, if the newbie learns the art and science of Radical Testing Results (Which you’ll be begin to learn starting with this report).

It’s not only copywriters who guess 95% of the time, if you expand the sample to include ‘marketers’ in general, the numbers are more like 99.9%  And here’s good news for you, MOST of those people will continue guessing and you can put yourself way ahead of the pack by implemeting a habit of continually measuring, testing, and improving.

One of the big “take aways” from Matts report for me is a simple formula I can use to rapidly accellerate my tests, even for offers that don’t have a lot of traffic to begin with.  Instead of waiting a long time to get statistically significant numbers, Matt suggests moving fast and looking for breakthrough results.  But, you still don’t want to guess, so it’s important to have some sort of rule of thumb to use when picking a winner in split tests.

Matt’s rule of thumb comes in the form of this formula that he calls his “big breakthrough formula:”

The Winner – Square root of the winner > the losing element

It works like this — In any split test, there will be a winner and a loser.  You know you have a reliable winner when it passes the big breakthrough formula test.

The big breakthrough formula is simple… You substract square root of the winner from the winner and it should be a bigger number the loser.

It sounds more complicated than it is…(most every simple calculator has a square root button)

Let’s say you’re doing an A/B test and element A has 20 sales and B has 12 sales. First, calculate the square root of 20 – which is 4.47.  Now, simply plug-in the rest of the numbers:

20 (winner) – 4.47 (square root of 20) = 15.53

The next question is: Is 15.53 a bigger number than 11 (the losing element)?  I hope you answered yes.

It may seem like a small thing, but I’ve struggled with this very issue many times. When doing a split test, how do you pick a winner?  What Matt’s formula does it it provides a simple way of ‘handicapping’ the winner of a split test to that you know it’s not just a winner, but a real winner — a breakthrough winner.

If you don’t get a winner that passes the test, then you have two choices 1) extend the test to see if the gap between the winner and loser expands or 2) pick a new test.  For me, if the test was close, I’d let it run for a while longer. If the difference is only slight, I’d create a new test — aiming for a quick breakthrough result.

Extreme Optimization Graph

Matt shows several informative graphs in his report and I just want to point out one thing, the vertical axis on all graphs is Visitor Value — one of the most important metrics you can measure in your sales systems. If you are a VisiOlo user, you already know that this metric is extremely valuable because you can see it, at-a-glance, right in your dashboard

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.