I was recently describing multivariable testing to a friend I hadn't spoken to in awhile. I explained to him that the process involved starting with millions of possibilities, and then very efficiently and systematically winnowing down the possibilities until the best performer is revealed. “Oh I get it,” he said. “It's just like American Idol.”

In some ways this is a nice analogy that you can use when describing multivariable testing to your less-than-tech-savvy friends (the ones who still think of AJAX as something you use for cleaning your kitchen). And there are similarities. In addition to the Darwinian process inherent in both, they also both represent democracy in its purest form. On American Idol, who stays and who goes is purely based on the votes of millions of viewers. With multivariable testing, which creatives stay and which ones go are also based on the collective decision making of millions of visitors to a website.

But there is one crucial difference between American Idol and multivariable testing. And it's the same critical difference that separates multivariable testing from other “black box” optimization techniques you might encounter.

One of the great virtues of a multivariable testing process is its transparency. When using multivariable testing, you not only end up with a version that performs better than all the other millions of options, but you can also determine with pinpoint precision why it performed better.

So, as in the case of Time, this month's Customer Success Story, it's not just that the subscription page below performed the best out of the near 1 billion permutations analyzed. We know that it increased subscriptions 16.2% better than the control. And more importantly, we know why. Because of the systematic, iterative multivariable testing process, we're able to learn that it was emphasizing the ease of signup, changes to the right banner, and other messaging changes that drove the improvement.

Now contrast this with American Idol. On Idol, we learn each week who is in the dreaded “bottom three.” But we get no insight at all into what is really working and not working for the other performers. Is one singer winning in a landslide, or are a group of performers separated by the smallest of margins? Pity the poor American Idol finalist. They never know for sure if they were one vote away from the bottom three and in need of a complete makeover, or possibly coasting along in first place and would be better off by simply giving the audience “more of the same.”

A better analogy to multivariable testing can be found in the other annual spring survival of the fittest rite — college basketball's March Madness tournaments. Once again we start with a long list of possible candidates, and winnow it down to a champion. But here we have great transparency. We can clearly see why Florida marched to a championship. We have all the scores, we have all the statistics, and we can re-watch the games to find the turning points. We can understand if it was great defense, great foul shooting, or a great bench that led to victory. And if you're a competing coach, you have great data that you can use for evaluating what it is going to take to be successful in the future.

In marketing optimization, a “black box” may spit out a winner for you, but it gives you little help in plotting your future strategy. Sometimes it will work, and you'll end up with a Kelly Clarkson. But you'll also probably end up missing key chestnuts nestled among the “losers,” the Jennifer Hudsons and Chris Daughterys, the also-rans who, with a bit of polish proved to be far more successful than the actual winners.

In running your online business, don't settle for a “black box.” While a “black box” approach like American Idol makes for great television, it doesn't make for an optimization system that is of great value to a business. A transparent system, like multivariable testing, not only gives you a winner, but also gives you the insights that can drive additional winners for seasons to come.

Time Magazine is the nation's oldest and most popular newsweekly. The Time.com website serves as the magazine's Internet presence, offering online-specific features such as blogs, in addition to content found in the magazine.

Time wanted to leverage a recently launched archive section on their website in order to attract new magazine subscribers. After attempting some internal A/B tests that were time consuming and very limited in scope, Time decided to bring in Optimost to conduct multivariable tests on their subscription page and help them to maximize paid subscribers to Time Magazine.

The dedicated Optimost Client Team developed a test plan for the subscription page, which included 11 variables and 73 different values (versions of variables) to be tested. Based on this plan, the Optimost solution engine quickly identified and generated almost one billion possible permutations of the page. With its advance testing methodology, Optimost was able to conduct multivariable tests on a subset of these permutations and identify a winning creative that increased subscriptions by 16.2%. In addition, Time was able to learn exactly what factors did and did not affect conversion rates, including:

1. Changing the top left banner vs. the top right banner
2. Changing “assurance” text to a VeriSign logo
3. Better emphasis of the key promotion
4. Making specific copy changes

You can access all of our customer success stories at: http://www.optimost.com/case_studies.phtml.

In previous newsletters, we have explained the importance of web analytics solutions and how they provide marketers with complete visibility into the behaviors of their web visitors. One of the key features of a web analytics solution is its ability to help marketers quickly identify bottlenecks and trouble spots that are hindering marketing and sales efforts. This is why Optimost partners with all of the leading web analytics vendors — the analytics data tells you where your problems are, and web site optimization solves these problems. One of Optimost’s web analytics partners is Coremetrics.

Coremetrics offers a suite of on-demand solutions that enables companies to optimize their marketing investments and maximize customer value online. The company’s solutions are powered by LIVE (Lifetime Individual Visitor Experience) Profiles, a comprehensive source of online customer data, which empowers businesses to perform detailed profile mining, build a multichannel view of customers and target consumers based on their online behavior.

The Coremetrics Suite includes:

• Online Analytics: The core of the suite which provides the ability to understand and analyze customer behavior
• Precision Marketing: Complements Online Analytics by empowering companies to target customers based on their behavior, attract high value customers and increase customer lifetime value
• LiveMark: A benchmarking solution that provides key performance indicators specific to your industry

Click here for more information about how Coremetrics and Optimost can work together to transform your online marketing programs.

For more information about Coremetrics, please go to www.coremetrics.com.

Click here for a complete listing of Optimost Partners.

If you are an Optimost partner and you are interested in either having your company featured in our Partner Spotlight or your upcoming events listed in our newsletter, please send an email to .

Seth Rosenblatt, the VP of Marketing and Business Development at Optimost, will be interviewed on the “Online Marketing with RSS Ray” Radio Show. The interview topic will be “Measurable Results with Multivariable Testing” and will take place on Wednesday, April 4, 2007 at 1pm ET/10am PT.

To learn more about the show and find out how you can listen to the interview, please visit the RSS Ray website at www.rssray.com.