CRO 105: Construct Data-Based Hypotheses

The last two articles discussed tools for conducting conversion research for your website. And we hope by now you have analyzed your website conversion performances, qualitatively and quantitatively. Because in this article, you will be forming hypotheses from the data collected during your conversion research.

What is a hypothesis?

Hypothesis is a scientific term for an assumption or idea proposed to be tested in an experiment. These ideas are usually formed based on some observations. Hypotheses formation is the third stage of the scientific method and has been the foundation for most profound laws in the natural and physical sciences.

This article will apply the same method to formulate theories for improving your website conversion rate. These theories may not always work for your website. They do not become a solution for improving your conversion rate until you’ve confirmed them using the A/B testing.

Parts of hypotheses

In website performance optimization, a strong hypothesis consists of three parts: the change, the effect, and the reason.

• The change

The first part is the change you want to make to improve your conversion rate. It’s not a guess but an idea based on insights from your qualitative and quantitative analysis. And to come up with a meaningful change, you need to understand the problem first.

For example, your website has a high bounce rate, and we know that’s a problem based on our metric definition in CR0103. The proposed change could be to add internal links and form chains of related content on your blog that will encourage web visitors to go from one page to the other.

For an ecommerce store, it could be to have a “people who bought this also buy” hyperlink on all your product pages that will take them to another product of the same category or catalog. Also, the change would be to put the hyperlink in the most clicked area gleaned from the Scrollmap so that it’s visible to all visitors.

Therefore, your change for this hypothesis would be “I believe adding internal backlinks to my pages,” and the next part will follow.

• The effect

What effect do you think your proposed change would bring to your site? This effect should point to a certain conversion metric. For the example above, your effect could be to “reduce the website bounce rate by 10%.”

Another way to understand the effect is to consider it the goal you aim to achieve with the change. And that’s the reason for the “10% decrease” in the example. It will serve as a target by which you grade the effectiveness of the new change.

How do you come up with the target? By referring to the market standard that you measured in the second article of the series. In that exercise, you performed a conversion rate audit, revealing the gap between you and your competitors.

Remember, the goal you set then was to meet up and surpass the market standard. So, your target is to cover the gap. If it’s 10%, let that be the target.

• The reason

Since we are talking about science, there’s a law that states that for an object to change its direction, an external force must be acting on it. It means nothing happens without reason.

Your reason for the bounce rate example could be, “It will cause people to stay longer on your website and make them check other website pages.”

When it finally works, you can say this is the reason for the increase in your conversion rate and record it as a working strategy.

Combining the three parts will result in the hypothesis below

I believe adding “people who bought this also buy” hyperlinks to all my product landing pages will result in a 10% decrease in bounce rate because it will make people browse other product pages.

More examples:

I believe adding a testimonial slideshow on my landing page will result in a 20% increase in the clicks of the “Start a free trial” button because it boosts people’s confidence in the product.

I believe reducing the signup form to five fields will result in a 25% increase in the free trial registration because it’s direct and facilitates the registration time.

Note: The hypotheses take the form, “I believe…………. will result in……………..because……………” Make sure you use the same format when formulating your hypotheses.

Prioritizing your hypotheses

After forming hypotheses, the next thing is prioritizing them in the order of their importance. Many skip this step and go ahead into testing, which usually results in disaster and confusion. You can have so many hypotheses and testing straightway may be aimless. For this reason, it’s good to prioritize before testing.

There are prioritizing frameworks and tools that you can use to achieve the task at hand. The simplest and most used is the P.I.E framework developed by Chris Goward at WiderFunnel. P.I.E stands for Potential, Importance, and Ease. The VWO uses the same framework. Let’s see how the framework works.

Potential: represent your confidence in achieving the goal with the proposed solution. You can assign a score between 1 and 5 (1 is the lowest score and 5 is the highest score).

Importance: It’s a score that shows the essentiality of the page you want to optimize in your visitor’s journey. It’s also ranked between 1 and 5 (1 being the lowest, and 5 being the highest). This score should increase with proximity to the checkout page.

Ease: Though some pages are essential, they are not all easy to optimize. The home page may be crucial in your conversion journey but usually contains many complex elements to optimize. The form page is one of the pages to assign a high score for this category. The same ranking system used in the last two parameters applies here too.

The scores for each hypothesis are arranged in a table as shown above. The average value is calculated by adding all the three scores (potential, importance, and ease) and dividing it by 3. Then, you test using the descending order of the hypotheses averages. This implies that hypotheses with the biggest average scores are considered to have high priorities.

Conclusion

Hypotheses are the basis for groundbreaking laws and solutions. They help to perform meaningful A/B testing that will lead to the improvement of your website conversion rate. Every hypothesis consists of three parts: change, effect and reason. And don’t forget to prioritize before you test. Make sure you perform all the exercises in this article. Every one of them counts a lot.