{"id":1670,"date":"2018-04-01T21:20:00","date_gmt":"2018-04-02T01:20:00","guid":{"rendered":"https:\/\/www.dialoginsight.com\/temps-rompre-avec-strategie-tests-ab\/"},"modified":"2023-06-21T16:08:19","modified_gmt":"2023-06-21T20:08:19","slug":"time-break-up-with-a-b-testing-strategy","status":"publish","type":"post","link":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/","title":{"rendered":"Is it time to break up with your A\/B testing strategy?"},"content":{"rendered":"

It happens to the best of couples marketers. You set goals to boost subscribers and sales. You decide to set up an omni-channel action plan to achieve those goals. You go as far as devising an A\/B testing strategy for your website, landing pages, PPC ads, and the entire gamut to measure, optimize and score.<\/p>\n

After all, isn\u2019t that what\u2019s in the books when digital marketing pundits dish out conversion love advice?<\/p>\n

Alas, after a while\u2014a few months, maybe even a year\u2014you look at the stats and notice that, perhaps after a few quick wins, your conversion rates lack their initial spark or have not evolved into something more meaningful for your business (read: more sales).<\/p>\n

And then you\u2019re hit with the stark realization that you\u2019ve lost that loving feeling with your A\/B testing strategy.<\/p>\n

That light breeze you just felt is the result of marketers around the world leaning back in their chairs and nodding in agreement.<\/p>\n

It is something we often see with results-driven marketers who long to embark in a full-time relationship with their A\/B testing strategy\u2014but without understanding how much commitment it takes.<\/p>\n

A\/B testing cannot be approached as a fling or Tinder date; you can\u2019t simply set up a control, put together a few tests over the course of a few months, make itty bitty tweaks to colours, images, copy and design layout, and call it a day.<\/p>\n

As top experts in digital marketing purport, you\u2019ve got to always be testing<\/em>. And you\u2019ve got to always<\/em> delve deeper than grazing a few stats to glean true insight into what works and what doesn\u2019t.<\/p>\n

Unless you have a full-time data scientist or a harem of analysts (and kudos for you if you do!), you\u2019re probably strapped for time and resources to fully understand what\u2019s wrong with your A\/B testing strategy.<\/p>\n

But there are some common mistakes to A\/B testing you should consider before heading to divorce court.<\/p>\n

You’re calling A\/B test results too early<\/strong><\/h2>\n

How many times have you conducted A\/B tests using a control and a variant, where the variant literally surges to become king of the mountain for a few weeks or months? Did you declare a winner and roll out? What happened in the long term? Did you consistently increase your conversion rates?<\/p>\n

If not, the reason is simple: you called the test results too early. Remember, website traffic and visit behaviour can vary from day to day, week to week, sales period to sales period. Stopping tests too soon, without the right sample size, will likely skew results.<\/p>\n

Also, make sure you run tests until you have at least a 95% confidence level for the winning variation<\/a>, which means there is only a 5% chance your results are a fluke. Your results need to be statistically significant over time to ensure you are implementing the right changes. If not, you become the victim of web site traffic\u2019s peaks and troughs. Try a A\/B split and multi-variant test duration calculator<\/a> to determine the best test length for you\u2019re A\/B tests.<\/p>\n

Your sample size is way too small<\/strong><\/h2>\n

An offshoot issue of conducting A\/B tests over too short a timeframe is that your sample size, or number of conversions, is probably too small, which may incite you to jump to conclusions too quickly.<\/p>\n

For example, just because your control page generated a few more conversions over three months does not necessarily mean you should pop open the champagne. It could be that you ran your test during a major promotion or other marketing campaign.<\/p>\n

According to ConversionXL<\/a>, there is no magic number of conversions you need to hit before finalizing your A\/B tests. However, aiming for at least 100 clicks\/conversion per variant or even more is a standard rule of thumb.<\/p>\n

Not sure what time of A\/B test sample size you need? Use a sample size calculator like this one<\/a> or this one<\/a>. While the science may not be exact, you can at least get a ballpark number as a gauge.<\/p>\n

You’re testing too much at the same time<\/h2>\n

Another huge mistake is not conducting each A\/B test from scratch\u2014and using only one variant.<\/p>\n

Some marketers start off with one test and with one variant, and then add over time contrasting variants to the same test. For example, you use a different headline in original test versus your control. Then, you add a completely new form field to that same test. After a while, you switch up the test\u2019s design.<\/p>\n

When you go on such a testing spree, how on Earth can you determine what change actually<\/em> made the difference between your test and control? Keep focused on variant for one test.<\/p>\n

Your’re forgetting your funnels<\/h2>\n

Your sales funnels are one of the most important factors that can influence the validity of A\/B tests.<\/p>\n

Here\u2019s why. Imagine you are trying to A\/B test a landing page and a product page. For each page, you concurrently run one or two variants against the control pages. You congratulate yourself for running multiple tests at the same time.<\/p>\n

Too bad they are both in the same funnel. There could be a huge overlap of traffic between the tests, making it nigh on impossible to figure out the ideal path to conversion.<\/p>\n

In order to mitigate the chaos, fully map out your paths to conversion by determining each page that falls into each path. Work from end to start, as the bottom of the funnel doesn\u2019t affect the top of the funnel as much. By doing so, you are creating a controlled test environment in which you are more likely to isolate the good from the bad.<\/p>\n

You don’t create competitive variants<\/h2>\n

So, you have identified the right A\/B testing period, got the right sample size, mapped out your paths to conversion, and pinpointed one variant to test, whether it be copy, call-to-action (CTA) or a design element. And blast off! The testing commences.<\/p>\n

While you should only test one variant at any given time, make the change between it and your control more significant and sexy. If you are testing copy, use completely different language. If you are testing overall layout, switch things up more than just placing your checkout button a tad higher up on the page. Conduct research with your competitors or other successful companies to see what they do with the specific variant you are testing and test their ideas.<\/p>\n

A relationship with you\u2019re A\/B testing strategy takes a lot of work as there are many things you can test, including messaging, CTAs, forms, layout, gated vs non-gated content, and so much more. However, by keeping in mind these best practices, you\u2019re sure to make progress and reignite you\u2019re A\/B testing flame.<\/p>\n","protected":false},"excerpt":{"rendered":"

You\u2019re probably strapped for time and resources to understand what\u2019s wrong with your A\/B testing strategy. There are some common mistakes you should consider before heading to divorce court.<\/p>\n","protected":false},"author":3,"featured_media":473,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34],"tags":[],"class_list":["post-1670","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-omni-channel-marketing"],"yoast_head":"\nIs it time to break up with your A\/B testing strategy? - Dialog Insight<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Is it time to break up with your A\/B testing strategy? - Dialog Insight\" \/>\n<meta property=\"og:description\" content=\"You\u2019re probably strapped for time and resources to understand what\u2019s wrong with your A\/B testing strategy. There are some common mistakes you should consider before heading to divorce court.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/\" \/>\n<meta property=\"og:site_name\" content=\"Dialog Insight\" \/>\n<meta property=\"article:published_time\" content=\"2018-04-02T01:20:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-21T20:08:19+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.dialoginsight.com\/wp-content\/uploads\/2023\/05\/est-il-temps-de-rompre-avec-votre-strategie-de-tests-a_b-_.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Sophie Lamarche\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Sophie Lamarche\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/\"},\"author\":{\"name\":\"Sophie Lamarche\",\"@id\":\"https:\/\/www.dialoginsight.com\/en\/#\/schema\/person\/86b00483c1475c0aee1c3cf21cf7292e\"},\"headline\":\"Is it time to break up with your A\/B testing strategy?\",\"datePublished\":\"2018-04-02T01:20:00+00:00\",\"dateModified\":\"2023-06-21T20:08:19+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/\"},\"wordCount\":1173,\"commentCount\":0,\"image\":{\"@id\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.dialoginsight.com\/wp-content\/uploads\/2023\/05\/est-il-temps-de-rompre-avec-votre-strategie-de-tests-a_b-_.jpeg\",\"articleSection\":[\"Omni-Channel Marketing Campaign\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/\",\"url\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/\",\"name\":\"Is it time to break up with your A\/B testing strategy? - Dialog Insight\",\"isPartOf\":{\"@id\":\"https:\/\/www.dialoginsight.com\/en\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.dialoginsight.com\/wp-content\/uploads\/2023\/05\/est-il-temps-de-rompre-avec-votre-strategie-de-tests-a_b-_.jpeg\",\"datePublished\":\"2018-04-02T01:20:00+00:00\",\"dateModified\":\"2023-06-21T20:08:19+00:00\",\"author\":{\"@id\":\"https:\/\/www.dialoginsight.com\/en\/#\/schema\/person\/86b00483c1475c0aee1c3cf21cf7292e\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#primaryimage\",\"url\":\"https:\/\/www.dialoginsight.com\/wp-content\/uploads\/2023\/05\/est-il-temps-de-rompre-avec-votre-strategie-de-tests-a_b-_.jpeg\",\"contentUrl\":\"https:\/\/www.dialoginsight.com\/wp-content\/uploads\/2023\/05\/est-il-temps-de-rompre-avec-votre-strategie-de-tests-a_b-_.jpeg\",\"width\":1200,\"height\":628,\"caption\":\"est il temps de rompre avec votre strategie de tests a b\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.dialoginsight.com\/en\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Is it time to break up with your A\/B testing strategy?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.dialoginsight.com\/en\/#website\",\"url\":\"https:\/\/www.dialoginsight.com\/en\/\",\"name\":\"Dialog Insight\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.dialoginsight.com\/en\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.dialoginsight.com\/en\/#\/schema\/person\/86b00483c1475c0aee1c3cf21cf7292e\",\"name\":\"Sophie Lamarche\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.dialoginsight.com\/en\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c33b62b92dd96c951d01d5941062e7183d27ab8ae3a0dea67cfdeaeed7aedeed?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c33b62b92dd96c951d01d5941062e7183d27ab8ae3a0dea67cfdeaeed7aedeed?s=96&d=mm&r=g\",\"caption\":\"Sophie Lamarche\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Is it time to break up with your A\/B testing strategy? - Dialog Insight","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/","og_locale":"en_US","og_type":"article","og_title":"Is it time to break up with your A\/B testing strategy? - Dialog Insight","og_description":"You\u2019re probably strapped for time and resources to understand what\u2019s wrong with your A\/B testing strategy. There are some common mistakes you should consider before heading to divorce court.","og_url":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/","og_site_name":"Dialog Insight","article_published_time":"2018-04-02T01:20:00+00:00","article_modified_time":"2023-06-21T20:08:19+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/www.dialoginsight.com\/wp-content\/uploads\/2023\/05\/est-il-temps-de-rompre-avec-votre-strategie-de-tests-a_b-_.jpeg","type":"image\/jpeg"}],"author":"Sophie Lamarche","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Sophie Lamarche","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#article","isPartOf":{"@id":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/"},"author":{"name":"Sophie Lamarche","@id":"https:\/\/www.dialoginsight.com\/en\/#\/schema\/person\/86b00483c1475c0aee1c3cf21cf7292e"},"headline":"Is it time to break up with your A\/B testing strategy?","datePublished":"2018-04-02T01:20:00+00:00","dateModified":"2023-06-21T20:08:19+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/"},"wordCount":1173,"commentCount":0,"image":{"@id":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dialoginsight.com\/wp-content\/uploads\/2023\/05\/est-il-temps-de-rompre-avec-votre-strategie-de-tests-a_b-_.jpeg","articleSection":["Omni-Channel Marketing Campaign"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/","url":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/","name":"Is it time to break up with your A\/B testing strategy? - Dialog Insight","isPartOf":{"@id":"https:\/\/www.dialoginsight.com\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#primaryimage"},"image":{"@id":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dialoginsight.com\/wp-content\/uploads\/2023\/05\/est-il-temps-de-rompre-avec-votre-strategie-de-tests-a_b-_.jpeg","datePublished":"2018-04-02T01:20:00+00:00","dateModified":"2023-06-21T20:08:19+00:00","author":{"@id":"https:\/\/www.dialoginsight.com\/en\/#\/schema\/person\/86b00483c1475c0aee1c3cf21cf7292e"},"breadcrumb":{"@id":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#primaryimage","url":"https:\/\/www.dialoginsight.com\/wp-content\/uploads\/2023\/05\/est-il-temps-de-rompre-avec-votre-strategie-de-tests-a_b-_.jpeg","contentUrl":"https:\/\/www.dialoginsight.com\/wp-content\/uploads\/2023\/05\/est-il-temps-de-rompre-avec-votre-strategie-de-tests-a_b-_.jpeg","width":1200,"height":628,"caption":"est il temps de rompre avec votre strategie de tests a b"},{"@type":"BreadcrumbList","@id":"https:\/\/www.dialoginsight.com\/en\/blog\/omni-channel-marketing\/time-break-up-with-a-b-testing-strategy\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dialoginsight.com\/en\/"},{"@type":"ListItem","position":2,"name":"Is it time to break up with your A\/B testing strategy?"}]},{"@type":"WebSite","@id":"https:\/\/www.dialoginsight.com\/en\/#website","url":"https:\/\/www.dialoginsight.com\/en\/","name":"Dialog Insight","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dialoginsight.com\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dialoginsight.com\/en\/#\/schema\/person\/86b00483c1475c0aee1c3cf21cf7292e","name":"Sophie Lamarche","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dialoginsight.com\/en\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/c33b62b92dd96c951d01d5941062e7183d27ab8ae3a0dea67cfdeaeed7aedeed?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c33b62b92dd96c951d01d5941062e7183d27ab8ae3a0dea67cfdeaeed7aedeed?s=96&d=mm&r=g","caption":"Sophie Lamarche"}}]}},"_links":{"self":[{"href":"https:\/\/www.dialoginsight.com\/en\/wp-json\/wp\/v2\/posts\/1670","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dialoginsight.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dialoginsight.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dialoginsight.com\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dialoginsight.com\/en\/wp-json\/wp\/v2\/comments?post=1670"}],"version-history":[{"count":0,"href":"https:\/\/www.dialoginsight.com\/en\/wp-json\/wp\/v2\/posts\/1670\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.dialoginsight.com\/en\/wp-json\/wp\/v2\/media\/473"}],"wp:attachment":[{"href":"https:\/\/www.dialoginsight.com\/en\/wp-json\/wp\/v2\/media?parent=1670"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dialoginsight.com\/en\/wp-json\/wp\/v2\/categories?post=1670"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dialoginsight.com\/en\/wp-json\/wp\/v2\/tags?post=1670"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}