You are then steering on behavior, which is much more reliable than assumptions or stated intentions. The condition is of course that you perform sound experiments and know how to interpret the results. Because there is only one thing worse than not testing - and that is testing incorrectly.
Significance and marketing
Anyone who has taken statistics classes will remember the term 'significance'. Simply put, significance occurs when the difference between two values found is so large that it cannot be due to chance. A significant result tells you that the difference found can be attributed to a specific cause. And that the difference will occur again if you repeat the experiment. To prove this, the sample size must be large enough.
Often you settle for 95-99% certainty. In that case you can be almost certain that it will succeed. Only then you still have a small risk that this will be the case. In some cases you may also settle for a 90% chance. And in extreme cases sometimes 80%. That is very opportunistic, by the way, and then you have to be aware that you are taking a gamble.
But perhaps you have good reasons for it. Perhaps the differences between the outcomes are so small or there are other interests at play. For example, that the experiment should not only lead to more sales, but that it also has other positive side effects. For example, brand building or user-friendliness.
Also read: What marketers can learn from iran telegram data young parents
Why is significance so important?
You might wonder what the importance of significance is in marketing. In a marketing experiment, you just determine a number or a percentage that you want to improve, that you are going to test for, right?
Not quite. Let me illustrate with an example why this is a dangerous way of thinking. Suppose you want to test whether using emojis in the subject line of your newsletter results in a higher open rate. You read in a study that this can result in a difference of no less than 15%. You want to test that. However, after conducting your experiment, the difference turns out to be only 3.3%. A disappointing result. You don't understand how that other company came up with that 15% and decide to try other ideas.
If you stop there, you're going to miss the mark. If you had done some more calculations, you might have found that that small difference of 3.3% is indeed a significant result. Then you would have found out that your target group is indeed more likely to open a newsletter with an emoji in the subject line. However, you will have to improve your subject lines even further to achieve the desired result.
