Strength metric overview

Learn how the strength metric helps you determine if a variation is a high or low performer.

Strength is a metric in AI-optimized optimizations that shows the probability that a given variation will perform better than the Base (no change) variation based on historical data. Optimize compares the site’s conversion rates and traffic between Base (no change) and a given variation over time.

When you initially launch a variation, the default strength value is 50%. The strength value increases or decreases as more data is collected. High values mean the variation will most likely perform better, while low values mean that Base (no change) will likely perform similarly or better.

How strength is determined

There are 3 aspects to determining the strength of a variation. First, the conversion rate difference between variations needs to be better by a non-trivial amount. As this metric fluctuates based on visitor engagement, a very small difference (e.g., 0.5%) could indicate that there's no clear high performer or that high performer could flip back and forth from day to day.

Next, there needs to be a non-trivial amount of traffic that routes to both variations. If only a few people experience the variations, they may not represent the whole of your visitor base.

Finally, the system makes sure that the difference between the variations is sustained and not just a coincidence. If the variation maintains a consistent lead over time, then the chances of consistently outperforming Base (no change) are high.

What actions should I take based on strength values?

• For low values (e.g., 30-0%) — you might want to pause the variation and try something different
• For high values (e.g., 70-100%) — you might want to try leaning into these ideas. Consider creating new variations that line up with the concepts of your high-performing variations.