Measuring whether paid marketing adds customers

The Performance Marketing work group compares lift studies, holdouts and matched-market tests, then asks whether the extra customers pay back.

Updated October 13, 2026

Compare people or markets that see the ads with ones that don't, divide the lift by spend, then follow those customers until they repay what they cost.

Start from a counterfactual. Mark Fiske asks what sales would have been without the spend and uses a matched-market test or holdout to get close. Timothy Davis runs lift studies and reruns them, because the multiplier can move as audiences change.
Platform reports can overclaim. At Ancestry, Mark's quarterly mix model refresh could show a channel about 20% less efficient than its platform suggested, after spend grew 30 to 40%. He tests around the proposed spend increase, since incrementality can fall as a channel saturates.
Then check payback. Daniel Pearson tracks how long the incremental cohort takes to repay its cost, and checks LTV forecasts against realized revenue before raising the next budget. He tests the highest-spend channels first.
Test channels that run together. Mike Jacobs randomizes matched markets into a group that gets the extra spend and one that doesn't, with other channels unchanged, and reports a range when effects can't be separated. For lead gen, Timothy measures lift on qualified leads or paid accounts, rather than raw form fills.

Group chat with Timothy, Mark, Daniel, Mike, Micah — the Performance Marketing work group

Question:

How do you measure whether paid marketing is truly incremental?

Timothy DavisGrowth at ElevenLabs. Former Senior Lead, Growth Marketing at Shopify

I use a lift study to answer the budget question: how many additional customers or dollars did paid create compared with a group that did not see the ads? I then divide the incremental outcome by spend and use that result to decide whether to scale or pull back. We rerun lift studies because the multiplier can move as audience behavior changes.

YouTube

Platform attribution is useful for managing campaigns, but clicks alone give an incomplete view of value. I also separate paid and organic traffic so existing demand does not get credited to the campaign. For smaller budgets, test design matters: I’ve suggested roughly $50K a month in platform spend as a point to start the incrementality conversation, since below that you may lack enough signal for a useful test. That threshold depends on the business and the platform.

YouTube

Mark FiskeManaging Director at H.I.G. Capital. Former VP of Growth and Marketing at Credit Karma

I start with a counterfactual: what would sales and customer value have been without the spend? A matched-market test or holdout gets closest to that answer. Compare the change in total customers and revenue in exposed markets with the change in comparable unexposed markets, then divide the lift by the added spend. Check payback using the value of the customers you actually acquired.

The operating problem is that channel teams often manage to publisher-reported efficiency. Facebook can report a profitable return while taking credit for customers who would have arrived anyway. At Ancestry, we refreshed our marketing mix model quarterly; a channel manager could discover their program was 20% less efficient than the platform view suggested, especially after scaling spend 30 to 40%. That should change the next budget decision and the channel target. For smaller brands, I would triangulate tests, mix modeling where feasible, platform data, and carefully designed post-purchase surveys, with experiments doing the heavy lifting on causality.

YouTube

Timothy DavisGrowth at ElevenLabs. Former Senior Lead, Growth Marketing at Shopify

One practical check between formal lift studies is a controlled pause in a specific paid campaign. Watch total conversions alongside organic conversions during the pause; a drop in attributed paid conversions alone tells you little. I’ve used tactical pauses to isolate a channel’s effect when tracking is imperfect.

I’d also check customer value after the initial conversion. In subscription growth, revenue per user can fall as acquisition broadens, so the spread between revenue per user and CAC helps determine whether the extra customers are worth buying.

Mark FiskeManaging Director at H.I.G. Capital. Former VP of Growth and Marketing at Credit Karma

Timothy's controlled pause is useful if the comparison accounts for what else changed during that window. I would watch total business outcomes and use a comparable market when possible, because seasonality or a concurrent promotion can make a simple before-and-after read misleading.

The survey input needs care too. At one portfolio brand, customers described TikTok influencer content even though the team was buying paid ads; the ads used influencer-style creative. We changed the follow-up questions to distinguish the creator's own post from an ad on the brand's account. A survey can help explain where demand came from, but its wording can shift the apparent channel mix. I use that evidence to interpret a lift test, not to replace one.

YouTube

Daniel PearsonCo-founder of Bamboo. Advisor to HexClad Cookware

The harder call comes after the lift study: how long does the incremental cohort take to repay its acquisition cost? I’d track the cohort’s actual revenue over time alongside the test result, then update the spend decision as that revenue arrives. A week-one lift can look promising while the customers’ long-term value and payback tell a different story.

At Bamboo, I’ve argued for payback period as the user acquisition team’s primary KPI because it forces us to revisit historic cohorts and check whether we actually earned the ad investment back.

Substack For a business with a long payback window, forecasted LTV helps make an interim decision; I’d keep comparing those forecasts with realized cohort revenue rather than treat an early estimate as settled profit.

Timothy DavisGrowth at ElevenLabs. Former Senior Lead, Growth Marketing at Shopify

Agreed on the pause: I would treat an uncontrolled before-and-after result as a diagnostic, then use a comparable market or holdout for a budget decision. I’d also lock down the tracking before reading either test. Consistent source, medium, campaign, and content UTMs let us separate prospecting from retargeting and check whether an apparent shift in paid CAC came from a measurement change.

Mark FiskeManaging Director at H.I.G. Capital. Former VP of Growth and Marketing at Credit Karma

Daniel's cohort point has an immediate implication for bidding: feed the platforms a signal tied to expected customer value, then revise that signal as actual cohort revenue comes in. I care about payback because it tells us how quickly the business can reinvest in the next customer. A return-on-spend target based on the first order can push spend toward customers who convert cheaply and repay slowly. The incrementality test tells us how many additional customers the channel creates; cohort payback tells us whether acquiring them merits the cash.

YouTube

Daniel PearsonCo-founder of Bamboo. Advisor to HexClad Cookware

I would prioritize experiments where the budget decision is biggest. Test the highest-spend channels first, then require a holdout before expanding a new channel. That gives the team a measured lift estimate where an attribution error could move the most money. I’ve made the case for using experiments to calibrate a marketing mix model, rather than treating a model’s channel estimates as settled without a holdout.

Micah MoreauVP of Growth at OpenAI. Former VP of Growth Marketing at DoorDash

The denominator of marginal CAC should be the additional active, repeat buyers caused by the spend, rather than all buyers who showed up while spend rose. Use the holdout or matched-market lift to estimate that denominator. Retargeting and bottom-funnel search are particularly likely to claim buyers who would have purchased anyway.

Calculate marginal CAC as the extra spend divided by those lift-adjusted buyers. A Jet offsite example shows why the denominator matters: moving from $2M to $3M in spend took total buyers from 75,000 to 90,000, making marginal CAC $66.67 even though blended CAC was $33. That $66.67 is a spend-curve illustration, not a causal estimate; the experiment tells us how much higher the true marginal CAC is if some of the 15,000 added buyers would have arrived anyway.

Timothy DavisGrowth at ElevenLabs. Former Senior Lead, Growth Marketing at Shopify

For lead gen, I would define the lift outcome at the point where sales can use it, rather than count every form fill equally. At SoftLayer, we cut more than 10,000 low-quality leads per month from the sales team's workload and increased quality leads by 44%. A campaign that lifts raw leads while sending more noise to sales would fail that business test. I’d agree with sales and finance on the qualified-lead or paid-account definition before running the holdout, then follow the cohort through revenue.

Daniel PearsonCo-founder of Bamboo. Advisor to HexClad Cookware

Micah’s marginal CAC math is where I’d be especially careful with forecasted LTV. As spend scales, CAC can rise while newer cohorts monetize worse, so applying the value of an older cohort to the next dollar of spend can overstate returns on both sides. I’ve seen startups treat an LTV forecast as financial truth and discover the gap too late. I’d refresh the forecast against realized revenue from each new cohort before raising the next budget.

Mark FiskeManaging Director at H.I.G. Capital. Former VP of Growth and Marketing at Credit Karma

Micah's marginal CAC framing changes the scaling decision. Run the next test around the proposed increase in spend, not only the existing budget, because incrementality can decline as a channel saturates. I saw that in practice at Ancestry: after a program scaled 30 to 40% in a quarter, the model refresh could show a channel manager roughly 20% less efficiency than expected. The team then needs to reset its operating target to the marginal, lift-adjusted return it can actually earn at the new spend level.

YouTube

Daniel PearsonCo-founder of Bamboo. Advisor to HexClad Cookware

The cadence of the decision has to match the time it takes customers to pay back. For a first-purchase-profitable DTC brand, first-party financials and one-day-click reporting can be useful between experiments. A business with a long payback period needs a cohort forecast for near-term decisions, then a check against actual revenue as the cohort matures. I’ve used Adobe as an example of why waiting for financial payback before every media adjustment would be too slow.

Timothy DavisGrowth at ElevenLabs. Former Senior Lead, Growth Marketing at Shopify

The test only works if we can follow the same customer outcome across systems. At Aha!, I worked with engineering on backend reporting that traced a trial or demo user’s campaign and source through to a paid account. For a B2B holdout, I’d use that paid-account outcome and give conversions enough time to mature before comparing groups. Otherwise a channel with a longer sales cycle can look weak simply because its customers have yet to close.

Question:

Mike, Micah says the $66.67 marginal CAC in his Jet example is a spend-curve estimate, not a causal result. How would you test whether added buyers came from paid spend when several channels run at once?

Mike JacobsChief Growth Officer at Consortium Brand Partners. Former VP of Growth at Etsy

Micah is right: $66.67 describes the spend curve; it does not establish what caused the 15,000 additional buyers. I’d randomize comparable markets into a group that gets the proposed incremental spend and a group that keeps its current spend, while keeping the other channels’ plans the same in both. Compare the change in total new buyers and contribution margin across the groups. That measures the lift from the budget increase even when buyers encounter several channels.

If the channels cannot be held stable, test the combined media plan first. Then vary one channel at a time across matched markets, or use a design that varies channels independently, to see what each extra dollar contributes and where channels interact. I would report a range when the test cannot separate their effects. At Etsy, I co-developed media mix modeling with Analytics for a broader cross-channel read; I would calibrate that model against the experiments rather than let each platform claim the same buyer.

mikejacobs.brandyourself+1

Ask a follow-up

Performance Marketing

104

Performance Marketing

1M+ insights

Performance marketing leaders from Shopify, Etsy, Credit Karma & 1 more

by SuperMe

Frequently Asked Questions

What should you check before adding budget to flat paid social?
How the channel is judged. Many plateaus are measurement or payback-target problems: a channel graded on platform reporting, or on blended CAC instead of marginal payback. Then widen lookalike bands and rerun lift tests.
How do you find out why paid social costs went up?
Find the step where the cost rises. A higher CPM with steady CTR points to the auction, a falling CTR to tired creative, clicks that stop converting to the audience, and several audiences stalling at one step to the offer or checkout.
When is a marketing mix model worth building?
When a wrong split of the next budget would cost more than the model, the spend history varies enough to separate channels, and finance and channel owners will act on it. Check its estimates against holdout tests.

All paid acquisition questions →

Frequently Asked Questions

Measuring whether paid marketing adds customers

Performance Marketing

Frequently Asked Questions

What should you check before adding budget to flat paid social?

How do you find out why paid social costs went up?

When is a marketing mix model worth building?

Your ideas, super powered.

Measuring whether paid marketing adds customers

Performance Marketing

Frequently Asked Questions

What should you check before adding budget to flat paid social?

How do you find out why paid social costs went up?

When is a marketing mix model worth building?

Your ideas, super powered.