Unless as an alternative to A/B testing

Learn how you can use Unless to test page personalizations and run simple experiments.

First of all, you should know that Unless is not an A/B Testing tool per se.

Nonetheless, many of our customers use Unless to experiment with content, CTAs, and design. Although it doesn't provide an A/B Testing Dashboard, you can quickly see how your ideas perform using the Insights page. If you have little traffic or if your website isn't compatible with A/B Testing Tools (which is often the case for Single Page Applications), Unless can be a great alternative.

How to set up an experiment:

With Unless, you can create simple experiments that strongly resemble an A/B test. Here's how to create an experiment step-by-step.

1. Create a personalization

Choose a page (we chose the homepage), create one personalization (we named ours Version B) and choose "Everyone" as the target audience.

versionb unless-ab-testing

2. Set up a goal

Create a goal (we created the goal "signup") and set the control group to 50%.

unless-goals-insights-4

3. Result

50% of the traffic you direct to the page will be sent to the personalization and 50% will see the original, unpersonalized version of your page.

Why we only support simple experimenting

Generally, testing your personalizations is a good idea. However, running proper experiments requires a lot of traffic and the more personalizations you test, the harder it gets to reach statistical significance. Also, the longer you run a test, the higher the risk of results polluted by external factors. Lastly, personalizations influence each other, so with every additional personalization it gets harder and harder to pinpoint what caused a dip or uplift in your goals.

To prevent this, we put an emphasize on continuous monitoring and offer a control group. Visitors assigned to the control group will see the original version of your page. This way you can always check how your personalizations perform in comparison to the unpersonalized experience.

insights-goals-unless On the Insights page, you can analyze how often each goal was reached for the original, unpersonalized experience (aka your control group) vs. personalized sessions (aka personalization participants)