How does Optimizely compare to Omniture?

AB testing


A / B testing offers webmasters the opportunity to optimize their website bit by bit by comparing two website variants (A and B). Small changes are made in the process. This is followed by an evaluation by means of web analysis, whereby the most promising version is determined.

Application examples for A / B testing [edit]

A / B testing is used to measure the success of two different variants of the same website or just individual elements of the website. The test scenario can be implemented for different areas.

Testing new web designs [edit]

For example, you can find out whether a new web design is better or worse received by users than the old version. When creating a new template, two different designs are often switched online and tested to determine which one the users like better. The focus is not only on improving usability, but also on increasing conversions, for example by inserting buttons with a call-to-action.

Testing of revised features [edit]

Even if the user guidance and navigation are revised or new features (e.g. filter, search function, purchase advisor, product comparison function) are introduced, A / B split tests are useful to check the correctness of the underlying decisions. In this way, operating errors can also be eliminated, for example. Features can also be carried out in the context of usability tests with A / B scenarios.

Testing the conversion of landing pages [edit]

If a site operator is not sure whether a landing page is already exhausting the maximum conversion potential, he can run a second, differently structured landing page in parallel and find out which one brings a higher conversion rate. The effect of images and videos, the structure of contact forms, the amount of text or special discount campaigns can also be checked using A / B testing.

Testing of SEO measures [edit]

It is not only possible to test the impact on conversion rates caused by changes to websites. It is also possible to do A / B testing for SEO measures. The principle is the same as the test of CR, but an SEO would like to know what effects his optimizations - such as changes to texts - have on the KPIs of the search performance, i.e. on the metrics of clicks, impressions, CTR or position.

How it works [edit]

Changes made to a website can have an immediate impact on traffic. Both the dwell time and the bounce rate adapt and can make big leaps. Since the key figures do not always change for the better, the A / B testing usually enables a gradual improvement, whereby the success is always kept in mind.

When performing A / B testing, the incoming traffic is split up, with part of the visitor flow being diverted from the original page and directed to a slightly modified page. This splitting is done arbitrarily and without the knowledge of the user. Ultimately, it is evaluated whether variant A or variant B is more likely to contribute to the desired success. An increase in the conversion rate, e.g. by clicking a banner or ordering a newsletter, counts as success, depending on the objectives of a website operator.[1], [2]

Temporal factors [edit]

It is important that both variants are tested at the same time during A / B testing, so that, for example, a visitor can see variant A and variant B alternately. If you change the variants z. B. every hour, after a day or according to the number x of visitors, there is a risk of falsification. How many visitors visit a website and how willing they are to buy depends not only on the website itself, but also, in particular, on external disruptive factors. With regard to the exact time and day, it matters, for example, whether it is a working day or a weekend day, whether it is morning, afternoon or evening and whether the users have already finished work or are still working.

Advantages [edit]

  • The change can be subjectively compared through A / B testing. The personal view often differs greatly from the target group's approach. When optimizing a website, only the interests of the target group should be taken into account.
  • The test can be carried out easily with the help of suitable tools. The effort is low and does not require any prior technical knowledge.
  • The analysis of the test results is clear. The website can be converted directly after the A / B testing.
  • This tool is also suitable for websites with only a few visitors per day.

Disadvantages [edit]

  • A / B testing only makes sense if the changes made are manageable. If tests are carried out in several places, it is impossible to determine the direct trigger of the success in the evaluation. Several tests are necessary to make several improvements to a website.
  • Since there is only one winner in A / B testing and this variant is then pursued, half of the information is automatically lost. If you test the next two variants, the consideration of how the conversion would have changed in the deleted version together with the changes in the latest test is left out.
  • Excessive changes that are reversed a little later can confuse existing customers. Therefore, A / B testing should only be carried out with new visitors.
  • In order for the data to be really meaningful, it should reach statistical significance. This can take a long time, especially for smaller pages with only a few visitors or conversions.[3]

Providers of A / B testing [edit]

There are a number of tool providers through which A / B testing can be carried out:

  • Optimizely: easy-to-use system, suitable for SMEs as well as large companies, with WYSIWYG editor and results in real time, test 30 days, then 14 euros for up to 2,000 visitors (bronze), other packages for higher demands
  • Ryte: With Ryte, A / B tests for SEO can be set up quickly and intuitively. This feature is available in the paid version. So SEO no longer have to play the typical "trial & error" game, but can easily read the possible results on other pages in the tool and continue to implement tangible measures.
  • Google Optimize: Free offer from Google (formerly Google Website Optimizer), natively integrated into Google Analytics, but less intuitive setup and fewer options compared to other services. For extended and more extensive tests with the tool, there is now also a paid version of it.[4]
  • Kameleoon: easy installation within a few minutes, intuitive operation thanks to the WYSIWYG editor with simple or extended reporting, minimal loading times, freemium account up to 2,500 visitors per month, paid packages Quickstart (up to 15,000 visitors), Learner (up to 65,000 visitors) , Standard (for 200,000 visitors), Premium (without visitor limit).
  • Visual Website Optimizer: easy-to-use tool with WYSIWYG editor, numerous additional functions such as heat and click maps, easy installation, 30 days free test for up to 1,000 visitors, then three packages between 10,000 and 300,000 visitors ($ 49 to $ 999 per month).
  • AB Tasty: Europe's leading all-in-one conversion optimization. Complete solution that offers everything a strong website optimization needs: A / B testing & multivariate testing, personalization, a marketplace with plugin and widget templates to increase user engagement, strong insights with session recording and heat maps, smart reporting and APP optimization .


In addition, there are more expensive solutions, such as Adobe Omniture, HP Optimost or Sitespect, which can be useful for large companies.

Benefits for online marketing

User behavior on websites is not only becoming more and more important for successful conversions. User signals such as bounce rates or the length of stay are important criteria for search engines today in order to determine the quality of a landing page. For this reason, A / B testing is becoming even more important. By specifically testing different website variants and content, important visitor key figures can be optimized in the long term. For e-commerce websites, such tests are especially important for increasing sales.

References Edit]

  1. ↑ Definition of A / B test Onlinemarketing.de. Retrieved on May 9, 2021
  2. ↑ Test types available in Optimize Google Support. Retrieved on May 9, 2021
  3. ↑ Why an A / B split test makes Karl Kratz blind. Retrieved on May 9, 2021
  4. ↑ Google Optimize and Google Optimize 360 ​​in comparison with Google. Retrieved on May 9, 2021.

Web links [edit]