Optimization FAQs
How long does it take to get an initial test up and running?
What do you need from your clients in order to set up and complete an initial test?
How much time does it take to complete an initial test?
How often can I run tests on my web pages?
Do I need to develop all of the creative pages that will be tested?
How many variables can be tested at the same time?
What types of content can you test on a page?
How many pages can I test at one time?
How quickly will I see results?
How long will you continue to optimize my website?
Answers
How long does it take to get an initial test up and running?
There are many different types of experiments, including A/B Tests, Simple Multivariable Tests, Full-Page Multivariable Tests, and Content Targeting. Most clients will implement a strategic optimization program that includes a combination of these testing types. Simple experiments can be launched within a few days or a week, while more complex experiments may take a month to set-up. Generally speaking, clients are up and running with at least their first experiment within the first 30 days of working with Optimost.
Top
What do you need from your clients in order to set up and complete an initial test?
In Optimost’s Managed Solution, Optimost reduces the burden on a client’s team by developing a strategic optimization program, developing individual test plans, implementing and monitoring experiments, and doing analysis of results. However, at a minimum, clients need to be involved in the process by reviewing and approving test plans developed by Optimost as well as placing the Optimost JavaScript code on the pages being tested and tracked. Clients also have complete access to all test functionality and reports through the Optimost Campaign Management Console (CMC).
Top
How much time does it take to complete an initial test?
Just like set-up time, the time to reach "statistical significance" in a test can vary depending on the scope and complexity of an experiment. However, Optimost generally recommends experiments be conducted in a series of "waves" (or mini-tests), which allow marketers to get learnings very quickly (often within a week) as well as validate results from one wave to the next. Optimization is a continual process that ideally never ends, but generally speaking, statistical significance for a full-page multivariable optimization experiment will be reached within one to two months. Simpler experiments, or sites with very high traffic, can reach significance earlier.
Top
How often can I run tests on my web pages?
There is no limit to the number of experiments that can be run using the Optimost platform, and therefore tests can be launched continually. Normally, Optimost helps clients develop a strategic optimization program that outlines different types of overlapping experiments. In addition, any content targeting undertaken on the site would be a continual process.
Top
Do I need to develop all of the creative pages that will be tested?
No. The proprietary Optimost authoring language creates templates that drive the creation and presentation of all possible web page permutations. Even layout changes are done automatically by the system—no additional HTML needs to be coded by your team. The only creative work undertaken by the client is the supply of any original artwork or photography that the clients want to test.
Top
How many variables can be tested at the same time?
There is no limit to the number of variables, or sections of a page, that can be tested simultaneously. In fact, Optimost generally recommends that clients test everything and anything on a page, as it is often found that increases in conversion rates can come from the "non-obvious" factors on a page. However, Optimost clients typically test 7-12 elements, or variables, in a full-page multivariable optimization experiment, which can generate millions, or even billions of page permutations.
Top
What types of content can you test on a page?
Optimost is unique in that it does not restrict you to test only certain types of content. Any type of content or application on a site can be tested, including HTML, XML, CSS, Flash, JavaScript, AJAX, Video, Audio, Forms, and other applications.
Top
How many pages can I test at one time?
Multiple pages can be tested simultaneously, either together within a single designed experiment (e.g. multiple pages in sales funnel) or as separate experiments in unrelated portions of a website. Visitor information or page variables or presentation can be passed from one page to the next to develop integrated cross-page experiments.
Top
How quickly will I see results?
You will start seeing results soon after an experiment is launched. Optimost generally recommends conducting experiments in a series of "waves" (or mini-tests). This allows marketers to get learnings very quickly, often within the first week of an experiment going live.
Top
How long will you continue to optimize my website?
The savviest marketers understand that optimization is not a "destination," but rather a "journey." This journey consists of a methodical program of continual improvement, because unless you’ve reached 100% conversion, you can always do better. In addition, your message, products, and audience will evolve over time (and may even be seasonal), so continual optimization is essential. Lastly, every experiment gives you great learnings to further hone your optimization program, including doing more sophisticated audience segmentation and content targeting.
Top
