Guide
A creative testing framework in 5 simple rules
Updated
A creative testing framework is a set of rules for testing ads. This one has five rules that tell you what to test, how to judge it and how many ads you need.
In short
- A creative testing framework is a fixed set of rules for testing ads, so every test teaches you something.
- The five rules are: test one idea per ad, set your judging rule before launch, switch off losers quickly, log every test, and test enough ads.
- Switch off losers on early numbers like views or clicks, but name winners only on sales.
- One round of 3 to 5 test ads needs a daily budget of at least your target cost per sale for each ad.
- At a 7% win rate, 4 ads a month give you only 0.28 winners, so the number of tests matters.
- Winners you can expect from 4 ads a month at a 7% win rate
- 0.28
- Ads out of 100 that do not become winners, at a 7% win rate
- 93
- Share of Meta ads that become winners, per Motion
- 5-8%
What is a creative testing framework?
It is a fixed set of rules for testing ads. It says what each ad tests, how you judge it and how you record the result.
- It makes sure each ad teaches you something, whether it wins or loses.
- It stops you from calling an ad a winner when it was only lucky.
- It keeps a record, so you do not test the same idea twice.
The five rules, in order
Test one idea per ad
Write the idea in one sentence first
Decide how you will judge
The number you judge on, your target and how many sales you need
Switch off losers fast
Use views or clicks to switch off losers, and sales to pick winners
Write down every test
Including the ties
Test enough ads
More tests give you more winners
Rule 1: test one idea per ad
Before an ad goes live, write in one sentence what it tests. If you cannot, the result will not teach you anything.
- A good test idea is one sentence, such as "Opening with the price beats opening with the result."
- Another is "Showing the product in the first 2 seconds keeps more viewers than showing a face."
- The things you can change are the opening, the main selling point, the format, the proof, the length or the person on camera.
- Six ads that change the same thing, such as six different openings, make one clear test.
- Six ads that each change something different give you a broad first look, which is useful in your first month.
This is the whole editor
Highlight a phrase and a clip lands on those exact words. No timeline, no keyframes, no layers.
Rule 2: decide how you will judge before launch
Write down the number you judge on, your target and how many sales you need before you decide.
- AdManage.ai's creative testing guide advises at least 100 sales per ad before you call a winner.
- If you get too few sales for that, use faster numbers such as views or clicks to switch off losers.
- Write your early cut-off down before launch. For example, switch off an ad when far fewer people watch its first 3 seconds than watch your other ads.
Which number to use for what
| Number | How fast you get it | Use it to |
|---|---|---|
| 3-second view rate (share who watch 3 seconds) | Fast | Switch off clear losers |
| Share who watch to halfway | Fast | Switch off clear losers |
| Click-through rate | Fast | Switch off clear losers |
| Cost per sale | Slow: about 100 sales per ad | Name winners |
Number: 3-second view rate (share who watch 3 seconds)
- How fast you get it
- Fast
- Use it to
- Switch off clear losers
Number: Share who watch to halfway
- How fast you get it
- Fast
- Use it to
- Switch off clear losers
Number: Click-through rate
- How fast you get it
- Fast
- Use it to
- Switch off clear losers
Number: Cost per sale
- How fast you get it
- Slow: about 100 sales per ad
- Use it to
- Name winners
Rule 3: switch off losers fast, pick winners slowly
Switching off a bad ad early costs little. Calling a lucky ad a winner wastes budget and teaches you the wrong lesson.
- Switch an ad off early when its 3-second view rate is far below your other ads after the same spend.
- Do not call an ad a winner after only 6 sales, even if the cost per sale looks great.
- Most ads will fail. Motion, an ad analytics company, found that only 5 to 8% of 550,000+ Meta ads became winners.
7 winners in 100 ads, at a 7% win rate
93 of these 100 ads do not win. Switching them off early is normal.
Rule 4: write down every test
A spreadsheet is enough. Without one, you end up testing the same opening line twice.
- Keep the ties too. A tie shows that the change does not matter to your audience.
- After about 40 tests, plan with your own win rate instead of the 5 to 8% benchmark.
One row per ad
| Column | What to write |
|---|---|
| Idea | The one-sentence idea it tests |
| Launch date | The day it went live |
| What changed | How it differs from the other ads |
| Result | The numbers you chose in rule 2 |
| Verdict | Win, lose or tie |
| Lesson | One sentence on what it taught you |
Column: Idea
- What to write
- The one-sentence idea it tests
Column: Launch date
- What to write
- The day it went live
Column: What changed
- What to write
- How it differs from the other ads
Column: Result
- What to write
- The numbers you chose in rule 2
Column: Verdict
- What to write
- Win, lose or tie
Column: Lesson
- What to write
- One sentence on what it taught you
Rule 5: test enough ads
The winners you can expect each month are your win rate times the number of ads you test.
- At a 7% win rate, 4 ads a month give you 0.28 winners on average.
- 12 ads a month give you 0.84 winners on average.
- 30 ads a month give you 2.1 winners on average.
- So the cheaper each test ad is to make, the more tests you can afford.
Winning ads per month, at a 7% win rate
Number of ads × win rate = winners you can expect.
- 4 ads a month0.28
- 12 ads a month0.84
- 30 ads a month2.1
How much budget does one round of tests need?
Run 3 to 5 test ads at a time in one ad set. Give each one a daily budget of at least your target cost per sale.
- AdManage.ai's guide suggests 3 to 5 versions per round of tests.
- At a $30 target cost per sale, 3 ads need about $90 a day and 5 ads about $150 a day.
- Judging on sales takes much more. At 100 sales per ad and $30 a sale, one ad needs about $3,000 of spend.
- So on a small budget, use views and clicks to switch off losers, and let only the best ad run long enough to prove sales.
Daily budget for one round of tests
Each ad gets at least the target cost per sale a day. Meta will not split the money evenly.
| Your target cost per sale | 3 ads | 5 ads |
|---|---|---|
| $20 | $60 a day | $100 a day |
| $30 | $90 a day | $150 a day |
| $50 | $150 a day | $250 a day |
Your target cost per sale: $20
- 3 ads
- $60 a day
- 5 ads
- $100 a day
Your target cost per sale: $30
- 3 ads
- $90 a day
- 5 ads
- $150 a day
Your target cost per sale: $50
- 3 ads
- $150 a day
- 5 ads
- $250 a day
Where Cutroom fits
Cutroom makes each test ad cheaper to make, so one recording can become several test ads.
- You upload one video of yourself talking, up to 3 minutes long.
- You get a finished 9:16 MP4 with captions and b-roll in about a minute.
- You make a new version in the editor: write a new headline on screen, pick another caption or hook theme, or move the b-roll to other words.
- A 30-second ad costs 110 credits. Another version of the same video costs only its export, 10 credits.
- Your first video is free, with no card, and you have 7 days to make it.
Questions people ask
- What is a creative testing framework?
- It is a fixed set of rules for testing ads: what each ad tests, how you judge it, when you switch it off and how you record the result.
- Should I run a formal A/B test or launch all the ads together?
- Launching the ads together in one ad set is usually more practical, and cheaper. A formal A/B test splits the audience evenly, which gives cleaner results but needs more budget. Either way, set your target before you see any results.
- How many things should one ad test?
- One, if you want a lesson you can use again. Changing several things at once can find a winner, but it will not tell you why it won.
- How long should a creative test run?
- Until it reaches the number of results you set before launch, or an early number has clearly ruled it out. A fixed number of days leads to verdicts that come too early or too late.
- Should I test in one ad set or several?
- One ad set, with the same audience, budget and offer. Testing across ad sets mixes audience differences with ad differences, so you can no longer tell which change did it.
- What if an ad wins and I don't know why?
- Test it. Make the ad again without the part you think made it work. If it still wins, that part was not the reason.
- Does this framework work on a small budget?
- Yes, with one change. If you get too few sales to judge ads on sales, judge them on faster numbers such as clicks or 3-second views, and test fewer ads at a time. The other rules cost nothing.
Film one video and turn it into your next round of test ads. Your first video is free.