Compare Content Versions

In this module, you will learn how to compare different versions of content in Optimizely CMS and use A/B testing to evaluate which version performs better. This feature helps you test content variations, analyze user engagement, and publish the most effective version based on real performance data.​

​By the end of this module, you will be able to:​

• Understand the purpose of version comparison and A/B testing in content management​

• Create and configure an A/B test for a content item​

• Define test parameters such as hypothesis, conversion goals, participation, duration, and confidence level​

• Monitor test results and determine a winning version​

• Publish the winning variation directly from the test results

The Compare Versions and A/B Testing features in Optimizely CMS allow you to test and refine your content strategy. You can create variations of a page, run controlled experiments, and analyze user behavior to make data-driven decisions before final publication.​

​When creating or editing a page, use the Publish dropdown menu to access additional options beyond immediate publishing. From here, you can choose to:​

• Mark as Ready to Publish​

• Schedule for Publish​

• A/B Test Changes (when the page has already been published)​

​Setting Up an A/B Test​

Once the A/B Testing feature is available in your CMS, you can initiate a test by selecting A/B Test Changes from the Publish dropdown. In the A/B testing interface, you can:​

• Add a hypothesis describing what you want to test​

• Set a conversion goal (e.g., increased site stickiness, engagement time, or clicks)​

• Define participation criteria (such as 100% of visitors to the page)​

• Specify the duration (e.g., 30 days)​

• Choose whether to start immediately or schedule for later​

• Optionally set a confidence level (typically 90–99%)​

​Once configured, start the test to begin collecting data. Visitors will be randomly assigned to different versions (A or B),and their interactions will be tracked against your defined goal.

Monitoring and Managing Results​

You can view test performance in real time from the View Test option. The results will show metrics such as:​

• Total views​

• Conversions based on your defined goal​

• Conversion rate per variation​

If a clear winner emerges, a fire icon will indicate the best-performing version. You can manually select the winner by clicking Pick Winner, which will automatically publish that version. Alternatively, you can abort a test at any time if necessary.

The following demo video will walk you through setting up and running an A/B test, analyzing the results, and publishing
the winning version:​