First, let me tell you that this is a very abridged version of the Conversion Rate Optimization Plan we use, but if you are currently just doing split testing without the rest of the steps that should be involved, then this is a great place to start to build your own methodology.

Step 1: Identify Goals

Start building your Conversion Rate Optimization Plan with your company’s Key Performance Indicators. These are the numbers that, when they improve, should improve the bottom line. Determine which of the KPIs you can influence and work to change those with the following steps.

Step 2: Gather Data

It is important to gather both quantitative AND qualitative data. The quantitative data like Google Analytics will tell you WHAT is happening. Qualitative data like user surveys, heatmaps, scroll tracking and usability testing will tell you WHY something is happening.

Step 3: Analyze Data

I start by looking for anomalies in the analytics patterns. If you are just starting, the data will often paint a picture. Look at the conversion rates at each step in the funnel and find the areas in the bottom or mid-funnel that show the greatest drop-off. Focus on the bottom of funnel first as those pages typically are more directly tied to the bottom line. Remember, to look at demographic segments in the analytics data, especially those that match your most profitable customers. Many tests are not segmented, but it is always a good place to make sure you are creating the best experience for your most profitable audiences.

Step 4: Create Hypothesis

Taking what you’ve done so far and formulating it into a workable hypothesis can be daunting at first.

Use this simple formula to get the hang of it: Because of “A” issue (Use quantitative data to describe the issue and qualitative data to support). I believe doing “B” for “C” [describe your change “B” and what segment will be tested “C”] will improve “D” (describe how a specific KPI “D” measurement will change). Lastly, you need to describe how much of a change “E” you need to see in what period of time “F”.

Step 5: Design Variants

If you are doing a complete page redesign, you’ll likely need to get some help from your design/web team. If you are just testing headlines or some other non-structural change, then you can typically use the testing platform to make the changes. You may need to do some quick Photoshop mock-ups to get buy-in from the powers that be. I recommend to keep this process as simple as possible. To do this, find the one person that can approve your tests. Avoid committees at all costs… unless you get paid by the hour. <– I’m kidding.

Step 6: Implement Tech

Your testing platform should make this pretty easy. We use Google Tag Manager to simplify code insertion. BUT, we never use it for inserting split testing code. It is almost guaranteed to have issues. Have your developers inject the code once into the site’s theme files for all pages and you’ll not need to it again. Be sure to test everything once the tools are integrated. Test it from multiple IP addresses and on each type of device that you include in your segmenting.

Step 7: Run tests

Run any test for the full length of time rounded up to the nearest week. Do not trust the results until the full time has passed. Don’t even look at them. I also like to run tests for at least two extra days. No matter what testing tool I’ve used, I get wonky data at the very beginning. Let’s say the calculator says you need 17 days to get statistically significant data. Then run the test for at least 21 days. I would run it for 23 days and remove the first two day’s data from the results as the data always seems to be a bit wild at the onset of a test.

Step 8: Analyze Results

Be ready for surprises. While some Conversion Rate Optimization tests will be game changers (“unicorns”), many will show low results or no statistically significant results. This is OK. We learn from all tests and move forward. Also, a handful of the low percentage changes can easily add up to the same end result of one of the “unicorns”.

How is your Conversion Rate Optimization Plan working for you?

So, that is our framework. It isn’t set in stone as it is designed to be flexible and to grow as necessary. I’d love to hear how your Conversion Rate Optimization Plan differs. Or, if you use this one, what did you change to suite your specific case. Let me know in the comments.

Additional tools

Additional Tools to help with your Conversion Rate Optimization Plan:

• VWO Split Test Duration Calculator (Statistical confidence on this one is set to 80%)
• Convert.com A/B Test Duration Calculator (We like this one a little better since you can dial in the level of statistical confidence you require)
• VWO A/B Split Test Significance Calculator