This past Friday morning, as I walked out of the somewhat dark confines of Grand Central Station, I was greeted by one of the first glorious sunny days in New York City this season. After walking just two blocks, I was greeted by a man in a yellow shirt with a card table hawking a wide array of the most stylish, name-brand (and probably hottest) sunglasses available. I noted that the man was doing a pretty brisk business this unseasonably warm morning, clearly outselling some other guy selling DVDs on a card table across the street.

Here on the streets of Manhattan, was a great, low-budget example of the benefits of dayparting. And it got me wondering: Who invented the media concept of dayparting? We all know who invented radio (Marconi) and who invented the Internet (Al Gore), but who invented dayparting?

After some quick research on Questia, the world's largest online library of books (which is featured in this month's Customer Success Story below), it appears to me that the concept of dayparting was developed by Archibald Crossley. Crossley had a research company doing daypart analyses on behalf of the Cooperative Analysis of Broadcasting, an association of 30 big radio advertisers, in 1930. Crossley's research clearly showed that audience levels varied markedly throughout the day, and these 30 savvy advertisers could make sure to sponsor programs when audience levels were highest.

Seventy-seven years later, daypart analysis continues to be a critical part of traditonal media buying. But the Internet allows you to take dayparting one step further. Because with the Internet, not only can you easily measure when the size of your audience peaks during the day, but you can also measure when the conversion rate of your audience peaks as well. This is critical information, because your marketing dollars will be used most efficiently when your conversion rate is highest.

For many sites, peak audience times and peak conversion times are very different. Take Questia's website, for example. Questia's traffic peaks between 9am and 5pm, but their conversion rate peaks between 11pm and 7am. In fact, their conversion rate is 18.5% higher during these late night hours. So even though the traffic is lower during the middle of the night, Questia can actually afford to pay 18.5% more for keywords during these hours because those visitors convert at a higher rate.

Not only do audiences convert at different rates during different parts of the day, but also the messaging that maximizes conversions varies over the course of the day as well. For example, on the landing page of one leading dating site, we found that while an image of a happy couple led to the highest conversion rates during the workday, an image of three attractive blonde women significantly increased conversion rates during the evening.

So when analyzing your website's performance, it's important to not just look at when traffic peaks, but it's equally critical to understand when conversion peaks as well. And further, it's important to optimize the site's messaging based on the different dayparts.

Which brings me back to the man selling sunglasses near Grand Central Station. As I returned to the train station late that Friday night, I once again came across the man in the yellow shirt. Same man, same yellow shirt, albeit a lot sweatier and smellier. But now, instead of sunglasses, he was selling roses, the perfect salve for a guy like me, feeling guilty about yet another late night at the office away from my wife. As I bought my dozen roses, I thought to myself, while the man in the yellow shirt clearly doesn't understand the value of deodorant, he without a doubt clearly understands the value of dayparting.

Questia is the first online library that provides 24/7 access to the world's largest online collection of books and journal articles in the humanities and social sciences, plus magazine and newspaper articles. To complement the library, Questia offers a range of search, note-taking, and writing tools that help students locate the most relevant information on their topics quickly, quote and cite correctly, and create properly formatted footnotes and bibliographies automatically.

Questia believes in continuous improvement, and they recognized that the page designs their web team put together, no matter how good they looked, couldn't possibly be optimal. So they decided to bring in Optimost to help them find the best solution to increase the number of subscribers who signed up for their newsletter.

The dedicated Optimost Client Team developed a test plan for the newsletter signup page, which included 8 variables and 50 different values (versions of variables) to be tested. The Optimost solution engine quickly identified and generated almost two million possible permutations of the page. In less than a month, Optimost conducted multivariable tests on the Questia email intercept page to identify a winning creative that increased email address subscriptions by 112.9%.

By making several simple changes to their website, Questia was able to achieve success by utilizing multivariable optimization with Optimost:

• Changing the main photo on the page
• Altering the wording on the submit button
• Adding “please” to the email field instructions
• Reducing the transparency on the background image

Based on the impact on this effort, Questia has expanded their optimization program to cover more revenue-driving applications throughout their site.

Q: How does the number of pages being tested affect sample size with respect to multivariable and A/B testing?

In short, the number of pages has very little impact on sample size when testing pages through a multivariable approach. However, in A/B testing, the sample size solely depends on the number of visitors that reach each individual page. Within the optimal design framework, an experiment conducted on a page will undergo iterations of multivariable tests called “waves.” Typically, three to four waves of multivariable tests occur, with a final wave that verifies results using a much smaller subset of pages in an A/B fashion. Figure 1 shows a typical sequence of wave testing for a homepage and the number of page permutations per wave. There are four areas (A, B, C and D) being tested on the homepage and two variations being tested for each area. The homepage currently receives about 4,000 Visitors per day.

Each multivariable wave (Waves 1, 2 and 3) contains eight uniquely generated pages, whereas the final verification wave (Wave 4) of the “control” (the original page) and two specifically constructed pages, called “supers.” The supers are made up of the best values improving the success metric. Waves 1 though 3 are essentially exploring all the test values for each area and evaluating which ones have outperformed the control or benchmark value. Once these values have been thoroughly tested, the supers are designed and tested in an A/B/C fashion. As you can see, the total sample needed for each multivariable wave (48,000 Visitors) was less than that of the verification wave (72,000 Visitors) even though the number of pages (8 versus 3) is larger in each of those multivariable waves. How can that be?

The reason for this is that in the multivariable waves, we are sampling at the variable level and not the page level. The goal of the multivariable wave is not to accrue as much sample to each page, but rather accrue enough sample per variation of each variable (or area) being tested. Figure 2 better illustrates how visitor sample is distributed to the variations of each variable within the experimental design. Figure 2(a) shows variable A having two variations or values: value c (control) and value 1. Essentially, to calculate the effect of A1 versus Ac, the sample needed for each value is the total number of visitors associated to that value's occurrence amongst the pages tested. For instance, if the goal of the experiment was to increase conversion rates, one would like to know what the average conversion rate is when Ac is only present (shaded in grey) and likewise what the average conversion rate is when A1 is present (shaded in blue). The sample needed for each of the averages is both 24,000 Visitors (4 x 6,000) since there are four instances of Ac and four instances of A1 amongst the eight pages. Figure 2(b) illustrates the same even (but uniquely different) distribution of sample amongst values Bc and B1 with each value garnering 24,000 Visitors.

Within a multivariable wave, the sample will always be greater on a variable level. With 48,000 Visitors distributed evenly to two values (24,000 Visitors per value), this significantly outweighs the 6,000 Visitors that each page receives. For that reason, reaching significance on a variable level occurs much faster than on a page level. Since the homepage observes 4,000 Visitors per day, each multivariable wave lasts approximately 6 days to reach significance on a variable level. The total duration of the three multivariable waves is 18 days.

When moving into a verification wave, the perspective on sampling reverts back to that of an A/B/C approach. Wave 4 emphasizes sample gathering at the page level since the goal of this wave is to determine the best page between the two supers as well as have the control page remain as our benchmark. In order to reach significance, 24,000 Visitors per page is required. Wave 4 consists of three pages requiring a total of 72,000 Visitors. The verification wave will last 18 days, which is three times the duration of a single multivariable wave. This example demonstrates why verification waves require more sample (and hence take longer to reach significance) even though the number of page permutations are much smaller when compared to the multivariable waves.

Web analytics solutions can provide marketers with insight into the behaviors of their web visitors. However, simply implementing a web analytics tool does not guarantee success. In order to maximize the value derived from web analytics, it is important to integrate data from other online channels to gain a comprehensive view of visitors and develop a strategic plan to take action on this wealth of data.

Optimost has partnered with Stratigent, a web analytics consulting firm, to help clients optimize their web analytics investments, and ultimately improve their online business performance. Stratigent's clients can leverage the collective expertise of Stratigent's team of consultants with diverse experience in the realm of web analytics. Through their thorough analysis of web analytics data, Stratigent is able to effectively identify web pages that are proper candidates for multivariable testing and optimization with Optimost.

Stratigent offers solutions such as Web Analytics Strategy, Proactive Data Analysis, Tool Optimization, Analytics Vendor Selection, Analytics Training, and Multivariable Testing (in conjunction with Optimost).

For more information about Stratigent, please go to www.stratigent.com or contact them at info@stratigent.com or 1-877-427-2900.

Click here for a complete listing of Optimost Partners.

If you are an Optimost partner and you are interested in either having your company featured in our Partner Spotlight or your upcoming events listed in our newsletter, please send an email to .

Optimost has scheduled its inaugural Optimization Summit for Wednesday, October 3, 2007 at Moscone Center West Hall in San Francisco. Click here for more information on the summit. Stay tuned for more details!

For more information on the Optimization Summit, please contact us at .