Often, every channel in a marketing report is the hero of its own story. Search claims the conversion, social claims the reach, and video claims it started the whole thing. But add up the credit and the same customer appears three times, so the report makes a poor guide to where next month’s budget should go. Worse, cut the channel that looks expensive, and you may find out it was creating the demand another channel was cashing in.

Marketing mix modelling was invented to settle exactly this argument. Rather than following individual clicks, it explains sales from the whole history of spending, promotions and seasonality at once, and asks what each channel adds that would not have happened anyway, which marketers call its incrementality. A Bayesian version goes one step further and also admits how sure it is. So instead of one attribution figure per channel, the team gets a range of plausible effects and can compare spending plans with the risk in plain view.

Yet there is one question the marketing model cannot answer, however good its data, because it has never seen the warehouse.

Who already budgets this way

Bayesian marketing mix modelling is no longer a research project. PyMC-Marketing is the open-source Python library behind much of this practice, and we contribute to it. Bolt describes using it for budgeting, for example, and PyMC Labs documents an engagement with Nürnberger Versicherung in which upper-funnel advertising creates demand that paid search later converts. After adopting a model of that funnel, Nürnberger reports a reduction in cost per lead of more than 27% over six months.

Those results belong to those teams, of course, but they show what the method is for. The model is not a one-off report. Instead, it becomes a recurring tool for deciding where the next euro goes, and it stays with the team, which adapts it as the business changes.

EUR 20,000, four weeks, three channels

To show the workflow end to end, we needed a business whose records we could publish, so we invented one. Kettle & Pine is an online homeware shop with 156 weeks of revenue, spending on search, social and video, and the demand, promotions and seasonality that move sales alongside advertising. And because we generated the data ourselves, we also know the true effect of every channel, which lets us check the model’s answers against the truth afterwards, something no real retailer can do.

Our story starts in week 130. The model learns from those 130 weeks only, while the revenue of the remaining 26 weeks stays hidden from it, so that we can later test its forecasts against what actually happened. The decision itself concerns the four weeks right after week 130.

For that campaign, Kettle & Pine has EUR 20,000. As usual, each channel manager has limits: search must receive EUR 1,000–2,600 per week, social EUR 500–3,000 and video up to EUR 2,500. Today the shop spends EUR 2,000, 1,800 and 1,200 respectively. Two familiar facts of advertising shape the model. First, an ad keeps working for a few weeks after the money is spent, an effect practitioners call adstock or carryover. Second, each additional euro buys a little less than the one before, because audiences saturate. Together they give every channel a response curve, and it is the slope of that curve, the return on the next euro or marginal ROAS, rather than the average return on ad spend, that decides where the money should go.

Two versions of Kettle & Pine's 156-week history. Each shows weekly revenue above weekly ad spend on search, social and video. In the first, social and video flights take turns; in the second, they run in the same weeks. In the final 26 shaded weeks, the model's forecast band follows the revenue that actually happened.
Two versions of Kettle & Pine's history. In the shaded weeks, the dots show the revenue that actually happened and the orange band shows what PyMC-Marketing forecast without seeing it. Foerster Marketing's forecast is practically identical.

We built two versions of that history to see how campaign timing affects what the model can learn. Search runs all year, whereas social and video run in flights, as marketers call bursts of a few weeks followed by a pause. In the first version, social and video take turns, so each channel’s effect shows up in weeks of its own. In the second, they overlap, as they do at many retailers. Reassuringly, the model’s forecasts for the hidden weeks track what actually happened in both versions: the actual revenue falls inside the 90% band in 23 of 26 weeks when the flights take turns, and in 22 of 26 when they overlap.

When the flights take turns, the model can tell the channels apart, and its advice is clear. At current spend, the next euro earns more in social and video than in search, so the optimiser moves roughly EUR 600 per week out of search and into the other two. Across the campaign and its carryover, that change is worth an expected EUR 1,300 in additional revenue.

PlanSearchSocialVideo
Current2,0001,8001,200
Recommended1,3862,0161,598

The gain is not guaranteed, however. Its 90% interval runs from about EUR −900 to EUR +3,200, so in some plausible versions of Kettle & Pine the change would do slightly worse. This interval is exactly what a dashboard would leave out, and it is what a finance director should ask about first, because it separates a confident recommendation from a reasonable bet.

Two overlaid histograms of the recommended plan's revenue gain, one from PyMC-Marketing and one from Foerster Marketing. They almost coincide, peak around EUR 1,500, and 86 percent of the distribution lies above zero.
Each histogram shows the plausible revenue gains of the recommended plan under one implementation of the model. The two implementations were built independently and agree so closely that their bars blend into one shape.

When two channels always move together

In practice, many retailers launch social and video together, which is convenient for the media plan but awkward for the model. If the two always rise and fall at the same time, several splits of credit explain the same sales history equally well, and the data alone cannot choose between them. Statisticians call this an identifiability problem. For the marketer, it simply means the model is guessing between two channels. And it shows: in our overlapping version, the recommendation shifts towards social and keeps video near its current level, while the 90% interval for the gain widens to roughly EUR −1,200 to EUR +4,300.

Two overlaid histograms of the revenue gain of the plan recommended for the overlapping history. They coincide, are wider than in the staggered case, and range from about EUR minus 4,000 to plus 6,000.
The same comparison for the history in which social and video run together. The plan differs from the one above, and its spread of outcomes is wider.

The usual way to break such a tie is an experiment. A geo-holdout or similar lift test measures what one channel does when only that channel changes, for instance by pausing it in some regions and comparing them with the rest. Feeding such results into the model is known as calibrating the MMM with lift tests. Because an experiment’s effect also carries over, we give the model the experiment’s whole time path, the weeks before, during and after the treatment, rather than a single before-and-after contrast. So in the third version of our story, Kettle & Pine adds one independent video experiment to the same overlapping history.

Since we generated the data, we can score each recommendation against the plan that would truly have been best. With overlapping flights alone, the model moves money into social and leaves video where it is, and that plan realises only about EUR 1,700 of a possible EUR 3,100 gain. With the lift test, the recommendation moves back to EUR 1,355 / 2,000 / 1,645 for search, social and video, close to what the model chose when the flights took turns, and it realises about EUR 2,800. In other words, one experiment is worth roughly EUR 1,100 on this single campaign, and that benefit returns every time the budget is set.

Interestingly, the experiment barely narrows the uncertainty interval, which still runs from about EUR −1,100 to EUR +4,000. What it changes is where the money goes, and that is what pays. As a cross-check, Foerster Marketing fitted the calibrated model independently and agrees: its own optimiser lands within about EUR 30 a week of PyMC’s plan on every channel, and both libraries put the expected gain of PyMC’s plan at about EUR 1,560.

PlanSearchSocialVideoTrue gain
Overlapping flights1,3822,4061,2121,682
Overlapping flights + lift test1,3552,0001,6452,836
Best possible (hidden truth)1,0982,0851,8173,114

A real engagement would, of course, also ask whether the experiment’s market resembles the rest of the business and whether another study is worth its cost, which this example does not price.

Then the warehouse falls behind

Up to this point, every plan has been judged by the revenue it is expected to generate. But the orders behind that revenue still have to be picked, packed and shipped, and a marketing model fitted on sales history knows nothing about the state of the warehouse today.

So suppose a staffing shortage cut fulfilment capacity three weeks ago, and nearly 700 orders are already waiting. The operations team expects normal capacity to return in three weeks, but until then every extra order the campaign buys simply joins the queue. Some of those customers wait, some receive a voucher, and some cancel, in which case the retailer has paid to acquire a customer mainly in order to disappoint them. In this way, a campaign can be a marketing success and a customer-service problem in the same week.

To decide how much to spend, Kettle & Pine therefore needs two estimates at once: how much demand each budget creates, and how much of that demand the warehouse can absorb. The marketing model already supplies the first. For the second, the recent shipment reports reveal the reduced capacity, though only in weeks when orders were waiting, because a warehouse with spare capacity ships whatever arrives. Read without the backlog, the same reports would mean something different, which is why the inference has to start from the warehouse’s actual state.

We bring both estimates together in a simulation of the next nine weeks. There, each plausible version of demand meets each plausible capacity, starting from today’s backlog, and the simulation counts the orders shipped, delayed and cancelled. Margin, delay costs and advertising spend then turn every run into profit.

From just three weeks of shipment reports, the simulation infers that the warehouse currently ships about 550 orders a week instead of its usual 900, give or take 20. The true figure, which we hid from the inference, is 560. As it turns out, that is enough to change the answer.

Campaign profit at weekly budgets from EUR 0 to 7,500, compared with keeping EUR 5,000 per week. With a clear warehouse, profit peaks at EUR 3,750 and is practically level up to EUR 5,000. With the backlog, it peaks at EUR 2,500, about EUR 4,300 above the EUR 5,000 plan, and falls steeply at higher budgets.
Profit over the four campaign weeks and five weeks of carryover, compared with keeping EUR 5,000 per week. Lines are posterior means and bands are 90% intervals. Blue starts from an empty warehouse at normal capacity, orange from today's backlog of 681 orders and the reduced capacity inferred from the shipment reports.

With an empty warehouse, any budget between EUR 3,750 and 5,000 a week earns about the same, within EUR 250 over the whole campaign. With the backlog, however, the best budget falls to EUR 2,500. Compared with keeping EUR 5,000, it earns about EUR 4,300 more, and the 90% interval of that difference, EUR 3,000 to 5,400, stays well above zero. Pushing through the slump with EUR 7,500, on the other hand, would cost about EUR 6,800. And since we wrote the warehouse too, we can check against the truth once more: with the hidden capacity and the true advertising effects, EUR 2,500 is indeed the most profitable budget, beating the EUR 5,000 plan by EUR 4,400.

The recommendation changes because the business is in a different state, not because the marketing evidence changed. Once capacity recovers, the same models would recommend spending more again, so the analysis is worth rerunning whenever the operational picture moves.

This is where simulation earns its keep. Our Stuttgart demonstrator simulated a whole city to ask what would happen to its shops if a shopping centre moved. That showed the method at scale, but it answered a question few businesses face every month. Kettle & Pine’s question, by contrast, comes back every budget cycle, and simulating the part of the business that limits the outcome turns a revenue forecast into a profit decision. The same pattern applies wherever estimates from data meet operations, from inventory and staffing to service capacity and delivery promises.

Inference that runs where the business runs

For a Python analytics team, PyMC-Marketing covers the measurement and allocation half of this workflow. Its budget optimiser already accepts custom constraints and objectives. What no marketing model has, however, is the current state of the warehouse. We can help such teams set up PyMC-Marketing, validate it and fit it to the decisions they actually make.

The second half needs something different, because it runs against the live state of the business. This is what we built Foerster for. Our probabilistic programming library for the JVM runs inference over ordinary application code. Each hypothesis about capacity executes the retailer’s own fulfilment logic in a separate fork of the running application state, so the analysis can explore what might happen next without writing anything back to the live system. The same mechanism appears in a queue demonstration, where ignoring the work already in flight reverses the conclusion drawn from the same three observations.

Foerster Marketing then brings the marketing model into that setting. We implemented Kettle & Pine’s model in both libraries and checked that they agree, with identical data and priors, independently verified densities and gradients, matching posterior estimates and the same recommended plans. Foerster’s value lies less in the model itself than in what it connects to, namely business logic, managed state and simulations already running on the JVM. To be fair, PyMC still fits this model faster, and we are closing that gap while extending Foerster to the models and use cases our customers bring.

And because businesses do not stand still, Foerster also supports Sequential Core-Set Monte Carlo, which updates a posterior as new batches of observations arrive rather than refitting from scratch. We have not yet validated it on this marketing model, though.

Bring us a decision

A first engagement starts with one recurring decision and the person who owns it, rather than with a request for another dashboard. We look at the spending and outcome history, campaign changes, promotions and any lift experiments, and first ask whether those records contain enough independent variation to support the decision at all. What the team gets is a repeatable review: a documented model, checks against held-out data, feasible plans and the uncertainty about their differences, together with an agreed way to evaluate the recommendation in practice.

Where operations change the economics, we extend the same review with a simulation of the parts of the business that matter, whether that is fulfilment, inventory or service capacity, and evaluate plans against the state the business is actually in.

So if your team is preparing its next budget, get in touch with the decision, the history you have and the constraints that matter. We can help with PyMC-Marketing in Python, or with Foerster where the analysis needs to run inside your business processes on the JVM.

All data in this article are synthetic. The PyMC-Marketing results use version 1.3.0 with PyMC’s default NUTS sampler and four chains of 4,000 draws. The executable study, its validation records and the performance measurements are in the Foerster Marketing repository.