Don't Be Intimidated by Testing Terminology

So you're thinking about testing, but a bit overwhelmed by all the jargon and complexity. Should I run an A/B test or multivariate? Taguchi or Full Factorial? What about BT and Personalization? Wading through this sea of terms might make you concerned that testing is too complicated for normal marketers.

The good news is that testing actually is quite simple. Sure, there are different levels of testing complexity. However, the reality is that the simplest tests often yield you the largest portion of gains, and you don't need a Ph.D to run successful tests. Below are some layperson's descriptions of the different tests out there and which is right for you.

Test Type	Summary	What it means for you
Before and After	Track performance on a page, make a change, see if performance improved	This is uncontrolled testing since any number of factors can cause an improvement or reduction in performance in addition to the content change you made. Nevertheless, this is still better than doing nothing.
A/B	Splitting the traffic between 2 or more published pages and tracking which one gets the most conversions.	This is the easiest controlled test to run and usually takes minutes to set up. A/B tests also are great for testing big layout changes and often give you the biggest boost in conversion rate.
Multivariate	Carving up a page into a few sections, then trying different content in each section to create different versions of the page -- usually a handful of versions, occasionally a few dozen or a few hundred.	The most talked-about testing method, this requires more traffic, technical work, and time than A/B testing. It can deliver major conversion rate improvements after you've done a few A/B tests on the page and want to boost the conversion rate to its upper limits.
Segmented, personalized, and/or experiential testing.	Multivariate plus cross-tabbing results based on visitor characteristics and modification of the entire user experience across pages.	This is the most sophisticated level of testing, which can push conversion rates even higher but requires significant time, technology, and resources. Nevertheless, if you've maximized your performance with A/B and multivariate tests, this is the next possible step.

Go back to article list