Read and manage your split tests
You'll find what each test metric means, what you can edit, and how to start, finish, or remove a test without confusing it with campaign testing.
Use A/B Testing to compare your questions, offers, forms, and whole flows. This page explains the numbers, the settings you can change, and when you need to publish. Each test compares the original, called control, with named alternatives.
For setup steps, see Test questions, forms, and offers or Test whole flows.
Resource tests, campaign experiments, and random routing
| If you'd like to compare… | What changes | What you can measure |
|---|---|---|
| Question wording | Wording and, for choice questions, answer wording or images | Answer rates and resource-test statistical calculations |
| Offer or form content | Headline, description, button text, image, and link settings for link offers | Conversion rates and resource-test statistical calculations |
| Whole flows | Flow structure, such as question order or number of steps | Flow completion and conversion metrics, without a statistics-engine verdict |
| Personalized and original pages | Personalized website content versus default content | Published experiment versions and scheduled decisions after visitors have had time to convert |
| Random paths in a flow | Which path a visitor takes inside one flow | Ongoing routing, without a test lifecycle or statistical winner |
Campaign tests group newly enrolled visitors by week and allow seven days for each visitor to convert. Evidence checks start at the second check, with a limit of 26 checks and minimum event counts in both groups. Their Winner, Loser, and other decision states are separate from a resource card's Statistical Confidence display.
Use the campaign's decision state to judge a campaign. Its audience segments aren't automatically separate randomized test groups. See testing website personalization.
The A/B Testing list
The list searches resource names and test types. Its controls are:
| Control | Options and behavior |
|---|---|
| Display | All tests, Offer tests, Question tests, or Flow tests. Forms use the offer-test type. |
| Status | All statuses, Running, Draft, or Completed. |
| Show all tests | Clears a focused single-test view opened through a link containing a test ID. |
The default is all types and statuses. Tests are grouped by lifecycle, with running tests first. A focused test link shows that exact test instead of applying the ordinary search and dropdown filters.
For tested questions and offers, Used in expands the list of flows that reference the resource. Each row's View action opens its flow. The test follows the shared resource, not only the flow used to open the editor.
Starting, ending, and publishing
| State or action | Meaning |
|---|---|
| Draft | Variants can be edited. A draft test isn't included as an active test in the published snippet. |
| Start test or Start Test | Saves pending test edits and explicitly activates a draft test. The test receives a start time and its variants become locked. |
| Running | The test is active. Published configuration can assign eligible visitors to its versions. |
| Publish after starting | Includes the active test in the site's configuration. Publication alone never starts a draft test. |
| End test or End Test | Requires control or an existing variant as the selected winner. Captures completion details and ends the active test. |
| Publish after ending | Applies the selected version to the site and removes the active comparison from the published configuration. |
| Completed | The test can't restart. Its variants and completion details can't be changed. |
Starting isn't a private preview mode. Once a test is active, another collaborator's publication can include it.
Ending with control keeps the original resource. Ending with an alternative promotes that alternative into the draft. For a flow, promotion replaces the original flow structure. Publication is still required before visitors receive the selected version everywhere.
The central End Test dialog includes version performance. It can warn when the selected version isn't the current best performer. That warning doesn't choose the winner or prevent an intentional choice of control. A test can end before reaching the displayed significance threshold.
Delete a draft test
Question and offer editors provide Delete draft. The flow toolbar calls the action Delete Test while the test is a draft. Confirmation permanently removes that draft test and its alternatives, but keeps the original resource. Separate changes made to control aren't rolled back.
An active test can't be deleted. It must end with a selected winner, including control if the original is to be retained. There is no return from Running to Draft, and a completed test can't be restarted.
What each version can change
| Resource | Alternatives beyond control | Alternative-specific changes |
|---|---|---|
| Question | One to three | Question wording, description, and choice-answer wording or images |
| Offer or form | One to three | Headline, description, button text, and image |
| Link offer | One to three | The offer content above, plus destination URL and new-tab behavior |
| Flow | Up to four | The copied flow structure |
Question alternatives use the next available Variant B, Variant C, or Variant D name. + Add variant adds one. Remove discards an alternative, but at least one must remain. Only one draft or running test can exist for a question.
Both choice and input questions support tests. Question tests divide traffic evenly between versions and remember each visitor's assignment in their browser. Ending unlocks the question. Ending with an alternative applies its question, description, and answer wording to the shared question. Keep current wording keeps control.
Offer and form tests use a version selector and Results or Edit view. Edit shows the selected version's content and preview. Results shows performance and stays empty until you start and publish. Add version creates an alternative. Remove version removes the selected alternative while retaining at least one. You can't remove control. The editor is read-only outside the draft state.
Changes to offer and form tests save automatically. Editing Control changes the shared original, so check the selected version before editing. Use the offer's link restores an inherited link destination. Non-link forms don't have link overrides. Visitors receive a randomly assigned version wherever the shared resource appears.
The content preview shows the selected version in a widget. Where available, you can choose the widget, desktop or phone view, light or dark colors, or full-size view. If no widget uses the form, the preview uses a default widget. Content and action previews use the website snippet's rendering code. Fields has a separate layout and selection preview. These previews don't publish the form or supply live test measurements.
Question and offer alternatives inherit content they don't override. For questions, answer membership, answer order, segments, branches, display, behavior, validation, Sync, and Auto-Segment settings are shared. Form field labels, placeholders, membership, order, and post-submission actions aren't independent test overrides.
Whole-flow creation copies control into Variant A. Control and variant tabs switch the structure shown in the builder. Double-clicking a tab opens its name field. Enter or clicking outside the field saves the name, and Escape cancels. Add variant copies control again and rebalances stored traffic weights approximately evenly. It doesn't duplicate all shared resources referenced by the flow.
The flow-test button isn't available in template previews or system flows. An existing draft or running test shows its toolbar instead. Five versions are possible in total. With three versions, stored traffic weights are 34% for control and 33% for each alternative. Those are assignment probabilities, not a promise of equal visitor counts.
Flow assignments stay in the visitor's browser. A later test on the same flow has a separate assignment. The widget's targeting, trigger, and enabled state still determine whether visitors can see the flow. In End Flow A/B Test, Control (original flow) is selected by default.
What locks when you start
A running question test locks wording, answers, display, behavior, and Auto-Segment rules. Sync, the internal question name, and which flows ask it remain editable.
A running offer or form test locks its content, fields, and actions everywhere that resource appears. The backend rejects changes to any started test's variants, including whole-flow variants. Flow controls that remain visible while running don't remove that restriction.
Read a running test
The main card shows Unique Visitors, Answered for questions or Conversions for other resources, the overall rate, and Best Performer. Each version appears beside control, with a content preview where supported. The offer and form Results table shows Version, Shown, Conversion rate, and Lift vs control, including conversion counts.
Best Performer is the version with the highest observed rate. It isn't necessarily statistically significant, and it doesn't automatically end the test.
| Metric | Meaning |
|---|---|
| Visitors or shown visitors | Unique visitors recorded for that test version |
| Answer rate | Visitors who answered divided by visitors shown the question, expressed as a percentage |
| Conversion rate | Converting visitors divided by exposed visitors, expressed as a percentage |
| Overall rate | Total recorded conversions divided by total recorded exposures across versions |
| Lift vs control | Relative change in rate compared with control |
| Answer shares | Share of respondents selecting each choice, when variant-level answer capture is available |
Lift is relative, not a percentage-point difference. A change from 20% to 25% is 25% lift and a 5-percentage-point increase. When control has a zero rate, relative lift isn't meaningful; resource tables can show Lift unavailable.
Question answer-share differences are percentage points. Multi-select respondents can choose several answers, so their answer shares don't have to total 100%. Unavailable coverage means the required variant-linked answers weren't recorded. It doesn't mean that zero people chose an answer.
A loading state or Results are unavailable right now means you don't have a measurement yet, not that nobody converted. The retry control requests the report again.
Read the statistical details
The resource card's Statistical Confidence compares the leading alternative with control. If control leads, the displayed comparison instead uses control against the best alternative. It isn't a single statistical comparison across every possible pair of versions.
Show details expands these values:
| Value | Meaning |
|---|---|
| Observed Lift | Relative rate improvement for the displayed comparison |
| P-value | A two-sided calculation for the observed difference in rates |
| 95% CI | The calculated interval for relative lift, not an interval for the raw conversion rate |
| Est. days to significance | An estimate based on observed rates, accumulated exposures, and elapsed test days |
| Recommendation | The card's current interpretation, such as Ready to conclude or a need for more evidence |
The card labels its 100 × (1 − p-value) score Statistical Confidence. That score isn't the probability that a version is better, and 1 − p isn't statistical confidence. The card marks a comparison significant when its p-value is below 0.05. Its interval and time estimate use the available data, so neither guarantees which version will perform better or when the test will finish.
An interval spanning zero includes no difference in the displayed comparison. With missing exposures or a zero reference rate, calculated zero bounds aren't evidence of a precisely measured zero lift.
Can the displayed percentage fall after 95%?
Yes. RightMessage recalculates the displayed score as new data arrives. It can rise or fall, and a different version can take the lead. Crossing the threshold doesn't freeze the comparison, stop traffic, or publish a winner.
This behavior is separate from campaign decisions based on stored, scheduled evidence checks. The resource card isn't the campaign experiment policy.
Whole-flow measurements
Flow version cards include Visitors, Completed, Completion, Drop-off, and Avg Time. Completion counts visitors who reach the flow end. Drop-off counts visitors with unfinished runs that didn't reach the end. Average time is based on recorded run durations, not a claim about every visitor's attention.
If any version has an offer or form step, the test's primary conversion metric is offer or form conversion. Otherwise, it is completion. Completion and conversion therefore need not be the same count.
Whole-flow tests have no statistics-engine verdict. The generic test card can still display Statistical Confidence based on its counts. That isn't a scheduled campaign-engine Winner or Loser, and RightMessage doesn't automatically choose or publish a flow.
Read a completed test
A completed card expands to show the chosen version, completion date, duration, final counts, rates, winner lift, significance, and available content previews. Its Impressions label summarizes recorded test exposures; it doesn't change those counts into all page views.
Ending saves the original content and the metrics available at completion before it promotes the selected variant. That saved comparison remains available if you change the shared resource later. Older tests without a saved snapshot can fall back to available resource or metric data. A failed metrics snapshot can leave the saved metrics empty without preventing the test from ending.
The completed card reports Significant, Not significant, or N/A when significance isn't available. Significance includes the same calculated percentage when available. There's no separate statistical-power field. The selected winner is the version you chose to keep, even if the comparison wasn't significant.
Route visitors randomly without a test
A flow's Split node starts with two groups and routes each group to its own steps. It uses a random value stored in the visitor's browser, mapped to the current number of groups. It doesn't reroll on every page view or rotate visitors through paths in order. Changing the number of groups can move an existing visitor to another group.
Random routing has no control, statistical panel, completed snapshot, or action to promote a winner. It stays in the flow until you change the structure and publish.