Slightly Tweaking Controls Helped In Lifting Delta.com
Investor's Business Daily
by Jennie Phipps
August 15, 2007
In online marketing, little things can mean a lot.
Delta Air Lines (DAL) made a few small changes to Delta.com as a result of user research, and the company says those small fixes have added up to about $15 million in additional revenue so far this year.
That's not huge money, but Delta calls it significant, especially considering it just came out of bankruptcy in April.
Delta.com's overhaul last year was much needed, says Brian DAmato, general manager of strategy and measurement for the Web site.
“If you look at where Delta was, compared to some of the other airlines from an online ticket volume and ticket share standpoint, there were plenty of opportunities to improve,” DAmato said. “And being a company that was in bankruptcy at the time, it was important that we improve.”
To begin its improvement, Delta appointed Abby Stephenson to a newly created post as manager of usability for Delta.com. With a master's in human factors, a degree that combines engineering and psychology, Stephenson's job is to make the site work better for its users.
At first, she relied mostly on focus groups, recruiting a few passengers at a time at an airport and asking them questions about their Delta.com experience. Stephenson says that while that kind of research is useful — “You'd be surprised at the trends you see” — it's also slow.
Delta needed faster results.
DAmato was approached at a trade show by salespeople from privately held Optimost. The company specializes in so-called multivariable analytics. That is, it uses various ways to test how Web sites are perceived by users. Among other things, it measures the differing reactions of various types of customers.
Optimost's Math Approach
“I didn't believe we'd be able to show the improvement they promised,” he said. “But (their approach) is all math-based. So I figured why not, let them fall on their face.”
Optimost is among the small companies that offer Web site testing, including basic A/B testing, in which two versions of a Web page, or a portion of a Web page, are tested. But Delta chose to use its multivariable analysis.
“Multivariable testing is just orders of magnitude more efficient with the same amount of traffic than A/B testing,” said Mark Wachen, co-founder and CEO of Optimost. “For example, 10 variables and five values gives us 10 million possible permutations. Now, we're not going to run all 10 million versions, but we might run between 100 and 200. In the offline world, testing 100 or 200 versions of almost anything would be nearly impossible.”
Other companies that offer multivariable analysis include Kefta Dynamic Targeting, Memetrics xOs Enterprise, Offermatica and SiteSpect.
Optimost focused initially on the path Delta.com's customers take to book and buy a ticket. The results of their tests were deceptively simple.
“Some things are mind-boggling,” DAmato said. “We found that at the point where users select a credit card, if we eliminated icons for the credit cards, the success of the page rose 3.5%.”
In general, Delta.com found that less is more. “The more we got rid of instructions, the more performance improved,” DAmato said.
Ambiguity also discouraged customers from continuing an online purchase. Originally, all of the buttons that moved customers through the ticket-buying process said “Click here to continue.” When Delta.com changed the final button before the customer's credit card was charged to “Click here to finish,” sales rose 2.5%.
“Clarity in action improves performance,” DAmato concluded.
Delta.com also learned that it's not too helpful to look at what other companies are doing with their Web sites. Customers differ. Plus, those other sites might be making mistakes.
‘Accountable Culture’
“Testing reinforces an accountable culture,” DAmato said, emphasizing that it holds true at all levels.
It used to be when a high-level executive had an idea for the Web site, the Web team felt obligated to follow through with the change. Now, testing holds sway.
When a top dog wants a change, DAmato can reply with confidence, “Thanks for your input, but our tests tell us it doesn't work.”
Delta.com is still testing variables. Among other things, it's using podcasts.
It's moved from the booking process to other aspects of the site. Initial tests suggest that green buttons encourage customers to proceed through the process better than red ones — but only at certain times of the day. What times? Delta.com is still testing. It plans to add another full-time employee devoted to the process.
“Optimost has changed the conversations that we have around here from opinion-based to data-based,” DAmato said. “It's really making us a numbers-driven organization.”
Find this article here
