Website personalization performance
Once you’ve added a campaign goal (or goals) to a website personalization campaign, RightMessage will automatically A/B test personalized content against the control, or default, content.
If you haven’t yet set up campaign goals, head over to our guide on it and get those set up. Without goals in place, there’s nothing to report on.
Understanding how we A/B test personalization campaigns
By default, new personalization campaigns with an attached campaign goal are tested as follows:
90% of traffic will have the campaign attempt to personalize content.
10% of traffic will be “held back” and treated as the control group.
Regardless of whether a campaign is personalizing content, RightMessage will capture segmentation and event conversion information. This allows us to compare and contrast not only how the control group converts relative to the personalized group, but also how each segment within these groups convert relative to each other.
In short, you’ll be able to see how each segment converts compared to other segments, and overall stats on how effective personalization is for your business.
Split testing is session-wide
When a visitor is placed into either the control (unpersonalized) or personalized cohort for a particular personalization campaign, they’ll remain in that cohort throughout the rest of their session.
This is done because some customers change radically different things – like offers and pricing – as part of their personalization, and we never want a visitor to be confused about seeing one thing promoted on one page view, and then upon returning a few minutes later seeing something entirely different.
Selecting a campaign and, optionally, a goal
On the Dashboard , click the top right card – Conversion Lift .
Ensure the time window for this campaign is set to reflect the window you want to report on. (By default, it’s the last 7 days.)
Your personalization vs. control conversion rate
This will view give you an overall understanding of how personalization is performing across all of your campaigns, along with which campaign is currently the front-runner.
A/B Testing & Conversion Tracking
RightMessage's A/B testing feature helps you measure the impact of personalization on your conversion rates. Here's how it works:
Control vs Personalized Groups
Control Group (~x% of visitors)
Sees default, non-personalized content
Establishes baseline performance
Randomly selected visitors
Personalized Group (~y% of visitors)
Sees personalized content
Tests your personalization strategy
Majority of your traffic
Key Metrics Tracked
Impressions
Number of times content is shown
Tracked separately for each group
Conversions
Successful form submissions/clicks
Measured per group
Conversion Rate
Percentage of impressions that convert
Calculated automatically
Conversion Lift
Percentage improvement over control
Shows personalization impact
Includes statistical confidence
Understanding Your Results
Observed Lift: Shows percentage improvement
Statistical Confidence: Indicates reliability of results
Confidence Interval: Range of likely true improvement
Pro Tips
Let tests run until statistical confidence is high
Monitor both groups for unusual patterns
Consider seasonal or timing effects
Test different personalization strategies
Remember: A/B testing is automatically enabled - no setup required. Just implement your personalization and RightMessage handles the testing.