Price testing: what to test, how to split visitors, how long to run it and how to pick the winner
A lower price that converts better can still leave you with less money: at the margins in this post, $129 has to sell to 20% more visitors just to match $147. Here’s how to plan, size, split and read a price test on a VSL offer, keep it honest, and roll out the winner.
On this page
- Plan: what to test in a price test
- Size: how many visitors a price test needs
- Split: why a price test needs a split, not a before-and-after
- The honesty rules for price testing
- Read: judge prices by contribution per visitor
- Roll out the winner, then test again
- Before you launch a price test
- Frequently asked questions
Price testing means showing two or more prices to comparable visitors at the same time, keeping each visitor on one price, and choosing the price that leaves the most contribution per visitor after refunds, not the one with the best conversion rate. Change one thing, size the test before it starts, run full weeks, and never show a "was" price you didn't really charge.
A price test runs in five stages: plan it, size it, split the traffic, read the result and roll out the winner. Open each stage to see what happens in it, why it matters and what usually goes wrong:
A price test in five stages
1Pick one lever and write the plan
What happens: Choose one price lever, the prices to compare, the metric (contribution per visitor after refunds) and what you will do with each possible result, before any traffic arrives.
Why it matters: Once numbers arrive, every result can be told as a story. Decisions made beforehand are the ones you can trust.
What goes wrong:
- A new price and a new headline tested at once
- No end date, so the test stops whenever someone likes the result
2Size it on the break-even lift
What happens: Work out how much more a lower price must sell, or how much less a higher one may sell, just to tie on contribution per order. Then find the visitors each price needs to detect that difference, and the days your traffic takes to deliver them.
Why it matters: A test too small to see the difference that matters can’t tell a real winner from luck.
What goes wrong:
- Sizing on a rule of thumb instead of your own margins
- Prices so close together that the test would need more than a million visitors per price
3Split visitors, not weeks
What happens: Assign each new visitor to one price at random and keep them on it when they return. Run every price at the same time, on the same traffic, with the video, the page and the checkout agreeing.
Why it matters: Ads, audiences and seasons change from week to week. Only prices running at the same moment compare prices instead of weeks.
What goes wrong:
- A video that says one price over a page that shows another
- A checkout link that still charges the old price
- New ads launched into the test halfway through
4Read contribution per visitor at the planned end
What happens: When each price reaches its planned sample, compare contribution per visitor, then again once the guarantee window has closed on the last orders.
Why it matters: A price changes both how many people buy and how much each order leaves. Contribution per visitor holds both.
What goes wrong:
- Stopping the day one price pulls ahead
- Calling the test on conversion rate
- Forgetting refunds, which arrive weeks later
Read more: Revenue Per Visitor (RPV): Formula, EPC, RPM and Tests
5Roll out the winner everywhere
What happens: Move all traffic to the winning price and update every place it appears: the video, the offer block, the checkout, the ads and the emails.
Why it matters: A price updated in some places and not others confuses buyers and turns into refund requests and support tickets.
What goes wrong:
- An old price left in an email sequence or an ad
- Never testing again as costs and traffic change
Plan: what to test in a price test
Price is more than the number on the buy button. The steps between bundle tiers, the shipping fee, the guarantee and a payment plan all change what a buyer pays or risks, and each answers a different question. If your VSL's main offer is a three-bottle pack at $147 and you wonder whether $129 would make more money, that's a price-point test. If you want more buyers on the six-pack, it's a ladder test. Here are the five levers, each with an example and what to watch:
Five price levers to test
Illustrative examples, one lever per test
The price of the main offer
The most direct test, and the one where conversion rate misleads most.
$147 against $129 for the three-bottle pack
Watch contribution per visitor, the share of buyers on each tier and refunds by price.
The steps between tiers
Moves buyers between one, three and six bottles.
$49, $43 and $39 a bottle against $49, $45 and $41
Watch the mix of tiers, not only the average order.
Who pays for delivery
A fee on the single nudges buyers toward bundles, and can lose some cold visitors.
$5.95 shipping on one bottle against free shipping on every order
Watch single-bottle orders and step-ups to the three-pack.
How long buyers can ask for a refund
A longer window can win buyers, and its refunds arrive later.
30 days against 90 days
Read it only after the longer window has closed on every order in the test.
One payment or several
Lowers the first payment; the later ones still have to be collected.
$147 today against three monthly payments of $49
Count only payments actually collected, and state every amount and date before the buyer pays.
Test one lever at a time. A new price with a new guarantee is two changes, and you won't know which one moved the result. Start with the price of the offer most of your buyers choose, because it touches the most orders.
Write the plan down before any traffic arrives: the lever, the prices, the metric, the sample size, the start and end dates, and what you'll do with each possible result. A plan written in advance is what stops a test from turning into a search for the story you hoped for.
Size: how many visitors a price test needs
Start from your economics, not a rule of thumb. Your margins tell you how much more a lower price has to sell just to tie, and that's the difference the test must be able to see.
Take the three-bottle pack. It costs $26.10 in product and $9 to ship, and payment fees and refunds together take 10% of the price:
- At $147, each order leaves $147 − $14.70 − $26.10 − $9 = $97.20.
- At $129, each order leaves $129 − $12.90 − $26.10 − $9 = $81.00.
$97.20 ÷ $81.00 = 1.2, so $129 has to sell to 20% more visitors just to tie. If $147 converts 2.5% of visitors, $129 needs 3.0% to earn the same per visitor. That 20% is the lift your test has to be able to see. If the lift lands far below 20%, keep $147; far above, switch. If it lands near 20%, the two prices earn about the same per visitor, and a wrong pick costs cents per visitor.
The calculator uses 95% confidence and 80% power and starts with that example: a 2.5% conversion rate at $147, a 20% lift to detect, 2,000 visitors a day and two prices.
Calculator
A/B test sample size calculator
How many visitors each version needs before a difference in conversion rate means something.
- Visitors in all
- Days at your traffic
- Orders per version at the current rate
95% confidence and 80% power, the usual defaults for comparing two conversion rates. Fix the sample before you start and read the result when it is reached: stopping the moment one version pulls ahead finds winners that aren’t there. With more than two versions, each is compared with the control and needs this many visitors.
At 2,000 visitors a day the test needs about 17 days, so round up to three full weeks. At 500 visitors a day it would take more than two months. Small price steps are worse: $147 against $145 means looking for a 1.9% lift, which takes about 1.7 million visitors per price. If your traffic is modest, test prices far enough apart to matter.
Two cautions. The calculator sizes the test on conversion rate, while you'll decide on contribution per visitor, which varies more from buyer to buyer when bundles and upsells are in play, so treat its number as a floor. And fix the sample before you start: checking every day and stopping the moment one price leads finds winners that aren't there. The VSL A/B testing guide explains significance and why peeking breaks a test.
Split: why a price test needs a split, not a before-and-after
The tempting way to test a price is to change it and compare this month with last month. It doesn't work. Ad costs, audiences, creatives and the season all shift between months, and any of them can move conversion more than the price did. A price test compares prices only when both run at the same moment, on the same traffic.
Three rules make the split fair:
- Assign visitors, not visits. Each new visitor gets one price at random and keeps it when they come back; a visitor who sees $129 on Monday and $147 on Thursday belongs to neither price. Random means random, too: giving phones one price and desktops another compares two audiences, not two prices.
- Change one thing. The video, the page, the checkout and the traffic stay the same; only the price differs.
- Run full weeks. Start and stop on the same weekday, so weekday and weekend buyers are counted in every arm.
On a VSL page there's a fourth rule: the video, the page and the checkout must agree. If your presenter says "just $147" out loud, an offer block showing $129 contradicts the video. Either keep spoken prices out of the script and let the offer block reveal them, or record one cut per price and split visitors between the cuts. Then click every buy button in every version: each must lead to a checkout that charges the price that version shows.
In RevvyScales, offer split tests (Growth plan and up) show different bundles, prices or offer headlines inside one VSL, each version by weight, with every viewer kept on the version they first saw (offer split tests). When the video itself names the price, a folder splits new visitors between two cuts of the VSL by weight, again keeping each visitor on one version (folders and A/B splitting):
- OrdersThe orders each version has taken so far. Don’t call the test before every version reaches the visitors you planned.
- RPVRevenue per visitor for each version. Multiply it by that price's contribution margin to compare what each version leaves.
- Traffic %The share of new visitors each version gets. A visitor keeps the version they first saw.
Whatever tool runs the split, check it after the first few days: each version should be getting roughly its share of new visitors, and orders should be arriving at the price each version shows.
The honesty rules for price testing
A price test is honest when every buyer sees one real price and pays it. These rules summarize the FTC's pricing guide and good practice; they aren't legal advice.
- One price per person. Keep each visitor on the price they first saw. If a buyer finds they paid more than someone else and asks, refund the difference without an argument. It costs far less than the dispute that could follow.
- No invented "was" prices. Under the FTC's Guides Against Deceptive Pricing, a former price gives a legitimate basis for a comparison if it's the actual, bona fide price at which the product was offered to the public on a regular basis for a reasonably substantial period. A price inflated so that a later reduction looks big is fictitious, and the bargain it advertises is a false one (16 CFR 233.1).
- Don't turn the other arm into a "was" price. In a test, $147 is a price half your visitors see this month. Showing it crossed out above $129 in the other arm is hard to square with the guide's idea of a price offered to the public on a regular basis.
- Don't raise a price in order to discount it. Lifting the price for a few weeks so a later "sale" looks bigger is the guide's textbook fictitious former price.
Anchoring against prices that really exist, your single bottle next to your bundles or a higher tier you sell, is a different matter, and the price anchoring post shows how to do it honestly.
Read: judge prices by contribution per visitor
Conversion rate is the wrong judge, because a lower price usually converts more visitors. The question is whether it converts enough more to pay for the money it leaves on each order. Contribution per visitor answers that in one number: conversion rate × contribution per order.
Suppose the brand also ran a third price, $165, and all three reached their planned samples. With three prices the calculator gives each arm the same 16,792 visitors, so at 2,000 visitors a day the test would take about 25 days, four full weeks. Here's how the three might come out:
Contribution per visitor at three prices
Illustrative test: three-bottle pack, before ad costs
The cheapest price sells to the most visitors and still earns 8 cents less per visitor than $147. It lifted conversion by 16%, short of the 20% it needed to tie, while $165 lost a fifth of its buyers when it could afford to lose 14%. Eight cents sounds small, but at 2,000 visitors a day it's about $160 of contribution a day.
Nothing in the result argues for a change, so $147, the price already running, stays. Ad cost per visitor is the same in every arm, so the price with the most contribution per visitor is also the most profitable one. The revenue per visitor guide covers the revenue side of this measure and how a few large orders can distort it.
Before you call it, run three more checks:
- Refunds by price. Read the result again once the guarantee window has closed on the last orders. A price can change who buys, and buyers who paid more may ask for their money back more often, or less.
- The mix. On a bundle offer, a new price on one tier moves buyers between tiers. Count contribution across every tier, not the tested one alone.
- The largest order. Take the single biggest order out of each arm and check that the winner still wins.
And resist slicing the result after the fact. Cut a finished test by device, source and country, and one slice is likely to show a "winner" by chance alone. If a segment matters to you, put it in the plan and size the test for it.
Roll out the winner, then test again
When a price wins on contribution per visitor, and still wins after refunds, make it permanent everywhere at once:
- Every place the price appears. The video, the offer block, the checkout products, the ads, the emails and any upsell that quotes it.
- The first weeks at full traffic. Compare them with the test's numbers. If they diverge, something about the traffic changed, and the result may not hold.
- The record. Prices, dates, visitors, orders, contribution per visitor and refunds for every arm. The next test starts from this baseline.
- People who already pay the old price. If you sell subscriptions or repeat orders, decide separately what happens to existing buyers, and tell them clearly before anything changes for them.
Then pick the next lever. Prices don't stay right forever: a new landed cost, a new traffic source or a new bundle changes the math, and the bundle pricing post shows how to set the steps between tiers before you test them.
Before you launch a price test
Tick these off before the first visitor sees a new price:
Before you launch a price test
0 of 8 done
Frequently asked questions
How long should a price test run?
Until each price reaches the sample you fixed before starting, and always in full weeks. Work the sample out from your conversion rate and the lift that would change your decision. At 2,000 visitors a day, detecting a 20% lift on a 2.5% conversion rate takes about 17 days, so you'd run three full weeks. Then wait for refunds before you make the change permanent.
Should I test a higher or a lower price first?
Work out both break-evens first. In this post’s example, $165 can lose about 14% of its buyers and still earn as much per visitor as $147, while $129 has to win 20% more buyers. Whichever change looks more plausible for your offer is the better first test, and the further apart the prices, the smaller the sample you need.
Can I test prices with a discount code instead of a split?
A code reaches only the people who get it, usually your most engaged buyers, so it measures their response rather than your traffic's. It also turns a price into a deal, which buyers read differently. For a clean read, split new visitors between two prices at random, keep each visitor on one price, and compare contribution per visitor.
What if a customer sees two different prices?
It happens when the same person uses two devices or clears their cookies. Keep it rare by assigning each visitor once and keeping them there. If a buyer who paid the higher price asks about the lower one, refund the difference promptly. It costs little next to a dispute or a public complaint, and it keeps the test honest.
Why not judge a price test by revenue per visitor?
Revenue per visitor is closer than conversion rate, but at different prices the same revenue leaves different amounts. A lower price sells more units for the same revenue, and every unit costs product and shipping. Turn revenue per visitor into contribution per visitor with each price’s own margin, after refunds, and compare that.