Skip to help content
RightMessage

How lift is measured on the Ads page

Find out what the Ads page's lift and extra ad value compare, why they can differ from the Campaigns page, and what Settled, Not settled, and Too early mean.

Available on the Scale and Pro plans.

Updated

View as Markdown

The Ads page's lift answers one question: for people who arrived from your ads, did the personalized page convert better than the original? That's a narrower question than the Campaigns page asks, so the two numbers often differ, sometimes by a lot. Here's how it's measured and how to read it.

What does the Ads page compare?

It compares the two groups your landing page's campaign already splits visitors into: those assigned to personalization and the holdback, who see the standard page. It counts only people who arrived from an ad, and counts each of them from their first ad visit. A conversion before someone clicked your ad isn't the ad's, so it's left out.

The comparison covers the period and goal selected at the top of the page, like every other number there. It keeps the whole period even when the campaign's test restarts partway through, for example after you edit a segment group the campaign uses. Until you pick a goal, the page counts the goal of the campaign testing your busiest landing page, so the comparison counts what that campaign's test counts.

Every lift on the page is measured the same way: all your ads at the top of the page and in the Lift report, each landing page and each audience on it, each segment and combination, and each ad in its details. When the campaigns testing your landing pages cover different pages and hold back the same share, their visitors are added together. Otherwise, the comparison uses the campaign that saw the most of those visitors.

How is extra ad value worked out?

What personalization earned you turns the lift into money. It takes the ad spend that landed on pages in a test and splits it between the two sides by their share of visitors. Then:

  • Extra ad value is the lift times the spend that reached personalized visitors: what the extra conversions would have cost you on the standard page.
  • Each side's cost per conversion is its share of the spend divided by its conversions.
  • The standard-page equivalent is what the personalized side's conversions would have cost at the standard page's cost per conversion.

The figure is an estimate, which the asterisk beside it says. It appears only when the lift is above zero, the comparison isn't too early, and your spend is in one currency. Otherwise the panel shows the lift as a percentage of the standard page's conversion rate. How this is worked out gives every number behind it, including the likely range, in dollars when it doesn't dip below no difference.

What personalization earned you on the Ads page: the extra ad value with its Settled or Not settled tag, the sentence behind it, and the personalized and standard-page conversion rates and cost per conversionOriginal
What personalization earned you on the Ads page: the extra ad value with its Settled or Not settled tag, the sentence behind it, and the personalized and standard-page conversion rates and cost per conversion

Why is it different from the Campaigns page?

The Campaigns page counts everyone the campaign reached: visitors from search, email, social, returning customers, and ads. The Ads page counts only the ones who came from an ad, from their first ad visit on. Same campaign, same split, a smaller and different group of people.

The two pages can also cover different time. The Campaigns page judges the campaign's current test from the day it started, on the campaign's own goal. The Ads page uses the 7, 14, or 30 days and the goal selected at the top of the page.

With few conversions, a handful of people can move either number a long way. In one real account, the Ads page showed +181% while the Campaigns page showed +71% for the same campaign. Two of the holdback's bookings came from people who had never arrived from an ad. The Campaigns page counted them; the Ads page rightly didn't, which left its holdback with fewer conversions and the lift much higher.

Neither number is wrong. Use the Campaigns page to judge the campaign as a whole, and the Ads page to judge it for your ad traffic. Expect them to agree more as conversions grow.

What do Settled, Not settled, and Too early mean?

Each lift comes with a 95% likely range: where the true difference most likely sits.

TagIt means
SettledThe whole likely range is above no difference, or the whole range is below it.
Not settledThe range still includes no difference. The extra ad value reads so far. It needs more conversions on both sides.
Too earlyEither side has fewer than 10 conversions. The number, when there is one, is shown grey as an early read. Treat it as a hint, not a result.

Hover over a too-early lift to see each side's conversions so far, such as Too early: 5 vs 2 conversions. With no estimate yet, it says No conversions yet, No holdback conversions yet when only the personalized side has converted, or No estimate yet.

Collecting means a side has fewer than 30 visitors, or nobody in the holdback has converted yet, so there's nothing to compare against. At the top of the page, it reads "The test is running. Results appear once both sides have visitors."

How do I read the Lift report?

The Lift report tab has one row per test: each personalized landing page that keeps a share on the standard page. Each row shows the page's ads, how many see the standard page, and its campaign; each side's conversion rate and conversions; the lift with a small range bar; and the extra ad value with the spend it's based on.

On the range bar, the dashed line is no difference. A settled row reads "Likely between" two values; a row that isn't settled reads "Could be anywhere from" one to the other. With more than one test, All tests shows the comparison from the top of the page: the tests' visitors added together when their campaigns hold back the same share, otherwise the busiest campaign's.

Under the rows, Not measured lists landing pages with no live campaign and personalized pages with A/B testing off, each with See pages.

The Lift report tab: a row per test with the personalized and standard-page conversion rates, the lift with its likely range, and the extra ad value, then what isn’t measured and how to read itOriginal
The Lift report tab: a row per test with the personalized and standard-page conversion rates, the lift with its likely range, and the extra ad value, then what isn’t measured and how to read it

Why is there no lift at all?

The lift says what's missing:

You seeIt meansFix it with
No standard page (A/B off)The campaign shows everyone the personalized page, so there's no holdback.Start measuring at the top of the page or in Do next. It saves a holdback to your draft for you to publish.
Can't measure · no goalNo goal counts as a conversion yet.Choose or create one in the Goal menu.
Can't measure · goal not attachedThe landing page's campaign doesn't record the goal you're counting.Attach goal in Do next or the Lift report, or count a goal the campaign already measures.
Goal not recorded yetNobody has reached the goal since it was set up.Check that it fires where you expect.
A dashNo live campaign runs on the page, the comparison couldn't be loaded, or the ad has no tracking link, so its visitors can't be recognized.Personalize with Righty, or fix the ad's tracking link.

Can I use it to decide a test?

Not on its own. A test's result belongs on the campaign's Testing tab, which judges everyone in the test. The Ads page tells you how personalization is doing for your ad traffic over the period you picked, and suggests a test when an ad audience's version converts often enough to compare two versions within about four weeks. See Read the Ads page.

How fresh are the numbers?

Visitors, conversions, and lift are read in the background and read again when they're more than ten minutes old, or after a new day, a new ad, or a publish. While a newer read runs, the line under the title says Updating and the page keeps showing the earlier numbers. The first time a large ad account's numbers are read, it says Measuring visitors and conversions. Large ad accounts take a few minutes. The summary at the top shows placeholder bars until the numbers are ready. A new goal or ad shows a dash until it's read, not zero.

Still need a hand?

Tell us what you’re trying to do and where you’re getting stuck.

support@rightmessage.com