Kettle & Pine, our fictional online homeware shop, has EUR 20,000 for its next campaign. The marketing lead wants that budget to bring in customers the business can serve profitably. Search, social and video all have a place in the plan, and their past results give the team a starting point. Where would another euro do the most good?

To answer that, we need to understand how spending produces sales. The channels can help one another: advertising in one place may create demand that another channel converts. So we look at the whole history of spending, promotions, seasonality and sales together, and ask what each channel adds beyond what might have happened anyway. Marketing mix modelling estimates those contributions, which marketers call incrementality.

We use a Bayesian model because several explanations can fit that history. We keep a range of plausible effects and ask how a spending plan performs across them. Marketing and finance can then compare the expected gain with the uncertainty and agree on a plan with the same evidence in view. In our example, an experiment helps them improve the channel mix, and a simulation of fulfilment helps them choose how much demand to acquire.

Who already budgets this way

Bayesian marketing mix modelling is no longer a research project. PyMC-Marketing is the open-source Python library behind much of this practice, and we contribute to it. Bolt describes using it for budgeting, for example, and PyMC Labs documents an engagement with Nürnberger Versicherung in which upper-funnel advertising creates demand that paid search later converts. After adopting a model of that funnel, Nürnberger reports a reduction in cost per lead of more than 27% over six months.

Those results belong to those teams. What we’d carry into an engagement of our own is the way the model becomes part of the team’s work. The next budget comes round, someone questions an assumption, and the people making the decision can inspect and adapt the model. We want Kettle & Pine’s example to make that process concrete.

EUR 20,000, four weeks, three channels

We gave the shop 156 weeks of revenue, spending on search, social and video, and the demand, promotions and seasonality that move sales alongside advertising. Making up the business lets us check something we couldn’t check in a real shop: what every other budget would have earned. We know the true advertising effects, so we can compare each recommendation with the plan that would have worked best. The model doesn’t get to see that truth.

The decision comes in week 130, with four campaign weeks ahead. We fit the model to those first 130 weeks and keep the remaining 26 weeks’ revenue hidden, so we can later compare its forecasts with what happened. Before asking it to recommend a budget, we want to know whether it can predict sales it hasn’t seen.

Each channel manager has limits: search must receive EUR 1,000–2,600 per week, social EUR 500–3,000 and video up to EUR 2,500. Today the shop spends EUR 2,000, 1,800 and 1,200 respectively. PyMC-Marketing lets the analyst compare allocations within those limits, with uncertainty about what each change would earn.

Suppose search has earned a good return so far. Does that make it the best place for another euro? Only if it still has room to grow. As an audience saturates, each additional euro buys a little less, so the return on the next euro can differ from the average return on all the money spent before it. That next-euro return is the marginal return on ad spend, or marginal ROAS.

We also need to follow what happens after the spending stops. An ad can keep working for a few weeks, an effect practitioners call adstock or carryover. The model combines carryover and saturation into a response curve for each channel. We use those curves to compare what a budget would earn during the campaign and afterwards.

Two versions of Kettle & Pine's 156-week history. Each shows weekly revenue above weekly ad spend on search, social and video. In the first, social and video flights take turns; in the second, they run in the same weeks. In the final 26 shaded weeks, the model's forecast band follows the revenue that actually happened.
Two versions of Kettle & Pine's history. In the shaded weeks, the dots show the revenue that actually happened and the orange band shows what PyMC-Marketing forecast without seeing it. Foerster Marketing's forecast is practically identical.

Now consider two versions of the shop’s history. Search runs all year in both. Social and video run in flights, as marketers call bursts of a few weeks followed by a pause, but in the first history they take turns and in the second they overlap. Both versions forecast the hidden weeks well: the actual revenue falls inside the 90% band in 23 of 26 weeks when the flights take turns, and in 22 of 26 when they overlap. We can now examine what each model recommends doing with the money.

Start with the history in which the flights take turns. Each channel has weeks of its own, so the model has evidence for telling their effects apart. At current spend, it estimates that the next euro earns more in social and video than in search. The optimiser therefore moves roughly EUR 600 per week out of search and into the other two. Across the campaign and its carryover, that change is worth an expected EUR 1,300 in additional revenue.

PyMC-Marketing’s recommended weekly spending, in EUR:

PlanSearchSocialVideo
Current2,0001,8001,200
Recommended1,4032,0051,592

We’d take the uncertainty to the budget review alongside that gain. The 90% interval runs from about EUR −800 to EUR +3,200, so in some plausible versions of Kettle & Pine the change would do slightly worse. With both the expected EUR 1,300 gain and its downside in view, marketing and finance can decide whether the change is worth making.

Two overlaid histograms of the revenue gain of Foerster Marketing's closely matching plan, one from PyMC-Marketing and one from Foerster Marketing. They almost coincide, peak around EUR 1,500, and 86 percent of the distribution lies above zero.
As a cross-check, each histogram evaluates Foerster Marketing's closely matching plan under one implementation of the model. The two implementations were built independently and agree so closely that their bars blend into one shape.

When two channels always move together

Now switch to the history in which social and video run together. If sales rise after every joint campaign, how much credit should each channel get? One explanation gives most of it to social, another to video, and both can reproduce the sales we saw. That is why forecasting sales well doesn’t settle where to spend the next euro. We need to know what happens when we change one channel without the other.

Statisticians call this an identifiability problem. For the marketing lead, it means the data leave an important budget question open. In this version, the recommendation shifts towards social and keeps video near its current level, while the 90% interval for the gain widens to roughly EUR −1,100 to EUR +4,300. We’d want to understand what supports that shift before acting on it.

Two overlaid histograms of the revenue gain of Foerster Marketing's plan for the overlapping history. They coincide, are wider than in the staggered case, and range from about EUR minus 4,000 to plus 6,000.
The same cross-check of Foerster Marketing's plan for the history in which social and video run together. The plan differs from the one above, and its spread of outcomes is wider.

This gives us a purpose for an experiment. A geo-holdout or similar lift test measures what one channel does when only that channel changes, for instance by pausing it in some regions and comparing them with the rest. The model can then use that evidence to distinguish explanations the spending history couldn’t separate. This is known as calibrating the MMM with lift tests.

In the third version of our study, we add one independent video experiment to the overlapping history. Because its effect also carries over, we give the model the experiment’s whole time path, the weeks before, during and after the treatment. A single before-and-after contrast would leave out part of the process we’re trying to learn from.

Since we generated the data, we can score each recommendation against the plan that would truly have been best. With overlapping flights alone, the model moves money into social and leaves video where it is, and that plan realises only about EUR 1,700 of a possible EUR 3,100 gain. With the lift test, the recommendation moves back to EUR 1,355 / 2,000 / 1,645 for search, social and video, close to what the model chose when the flights took turns, and it realises about EUR 2,800. The experiment improves this campaign’s allocation by roughly EUR 1,100 under the hidden truth. The marketing lead has a reason to change the plan that the overlapping spending history alone could not supply.

Notice that the experiment barely narrows the uncertainty interval, which still runs from about EUR −1,100 to EUR +4,000. If we judged it only by the width of that interval, we’d miss its value. It helps the team put the money in a better place. As a cross-check, Foerster Marketing fitted the calibrated model independently and agrees: its own optimiser lands within about EUR 30 a week of PyMC’s plan on every channel, and both libraries put the expected gain of PyMC’s plan at about EUR 1,560.

Weekly spending and true campaign-plus-carryover gains in EUR, using PyMC-Marketing’s recommendations:

PlanSearchSocialVideoTrue gain
Overlapping flights1,3752,4021,2231,739
Overlapping flights + lift test1,3552,0001,6452,836
Best possible (hidden truth)1,0982,0851,8173,114

Before using an experiment this way in a real engagement, we’d also ask whether its market resembles the rest of the business and whether the benefit justifies the cost of the study. Our example doesn’t price that cost.

A budget for the business you have today

The marketing lead now has an allocation to take to finance. The model has been checked, the experiment gives the team a reason to improve the channel mix, and the uncertainty is there to inspect. Operations can help refine the plan further. Nearly 700 orders are waiting to ship, and the team expects capacity to recover in three weeks. How much demand is worth acquiring while those orders move through fulfilment?

In this extension of our fictional business, a staffing shortage cut fulfilment capacity three weeks ago. Delayed orders create compensation and support costs, and some customers cancel. Those costs affect what the campaign can earn. By including them in the planning, the team can choose a spending level that makes better use of the capacity available now and revisit it as fulfilment recovers.

To answer that, we need an estimate of capacity. Imagine two warehouses that both shipped 550 orders last week. The first had only 550 orders available. The second still has hundreds waiting. Would you draw the same conclusion about how much they could ship next week? In the first warehouse, the report tells us little about spare capacity. In the second, it gives us evidence of a bottleneck. So the backlog changes what the shipment count can tell us.

A shipment count is the end of a process. Orders arrive, the warehouse ships up to its capacity, some waiting orders cancel, and the rest carry into the next week. We use Foerster to run those rules under alternative capacity assumptions and compare the resulting shipments with the reports. Each hypothesis has to explain what happened from the state the warehouse was in when the disruption began.

From three weeks of shipment reports, Foerster estimates reduced capacity at about 550 orders a week, give or take 20, compared with the usual 900. The true figure, which we hid from the inference, is 560. We then call the same warehouse function to simulate the next nine weeks from today’s backlog, pairing draws from the marketing and capacity estimates. It counts orders shipped, delayed and cancelled, and we use margin, delay costs and advertising spend to calculate profit.

Campaign profit at weekly budgets from EUR 0 to 7,500, compared with keeping EUR 5,000 per week. With a clear warehouse, profit peaks at EUR 3,750 and is practically level up to EUR 5,000. With the backlog, it peaks at EUR 2,500, about EUR 4,300 above the EUR 5,000 plan, and falls steeply at higher budgets.
Profit over the four campaign weeks and five weeks of carryover, compared with keeping EUR 5,000 per week. Lines are posterior means and bands are 90% intervals. Blue starts from an empty warehouse at normal capacity, orange from today's backlog of 681 orders and the reduced capacity inferred from the shipment reports.

With an empty warehouse, any budget between EUR 3,750 and 5,000 a week earns about the same, within EUR 250 over the whole campaign. With today’s backlog, EUR 2,500 a week makes the best use of the campaign budget. Compared with keeping EUR 5,000, it earns about EUR 4,300 more, and the 90% interval of that difference, EUR 3,000 to 5,400, stays well above zero. The larger EUR 7,500 plan earns about EUR 6,800 less than keeping EUR 5,000. And since we wrote the warehouse too, we can check against the truth once more: with the hidden capacity and the true advertising effects, EUR 2,500 is indeed the most profitable budget, beating the EUR 5,000 plan by EUR 4,400.

We’ve kept the marketing evidence the same. Bringing fulfilment into the decision helps the team choose how much demand to buy today. Once capacity recovers, the same models would recommend spending more again. Marketing and operations now have a shared reason to bring the next shipment report back to the budget review: it helps them recognise when more advertising becomes worthwhile.

This is the kind of recurring decision we want simulation to help with. Our Stuttgart demonstrator simulated a whole city to ask what would happen to its shops if a shopping centre moved. Kettle & Pine needs a much smaller simulation, but it has a question that comes back every budget cycle. Following demand through fulfilment lets the team connect its campaign investment to the profit it can earn.

Inference that runs where the business runs

For a Python analytics team, PyMC-Marketing is a practical starting point for measurement and allocation. Its budget optimiser already accepts custom constraints and objectives, and an operational model can be built alongside it. We can help the team check whether its records support the allocation it wants to make, fit and validate the model, and bring experiments into the next budget review.

We kept the warehouse example small enough to check independently. It has a backlog, capacity and fixed rules for cancellations and delay costs, and we could have written a separate analytical model of it. We checked the capacity estimate against an independent numerical calculation, and a separate Python implementation reproduced the profit calculation.

What we want to carry further is the connection to the application. Suppose yours already decides which orders get stock first, whether to split a shipment and when to offer compensation. Those rules affect both what the shipment reports tell you and what another campaign would earn. We’d like inference and forecasts to run that code too, because every rule change otherwise gives you two implementations to keep in step.

Foerster is our probabilistic programming library for Clojure on the JVM. Clojure’s immutable, persistent data structures let different versions share unchanged data while keeping their updates separate. That is a useful foundation for exploring several possible futures of an application. Spindel, the runtime beneath Foerster, manages the execution state and its forks.

Foerster’s particle methods can start from the application’s managed state, execute compatible application logic in separate forks under uncertain assumptions, and compare the resulting histories with observations. The same logic can then evaluate proposed plans. In our synthetic warehouse, both steps call the same fulfilment function and leave the caller’s state unchanged. We haven’t integrated the example with a retailer’s production application.

A separate queue demonstration shows why the starting state matters. Hypotheses about service speed run through the application’s scheduling function, and including work already in flight changes which hypothesis the completion reports support. A future run can explore a different queue without changing the parent’s managed state. External actions such as issuing an actual refund still need to be kept out of speculative runs.

Foerster Marketing supplies the demand model in this JVM setting. We implemented Kettle & Pine’s model in both libraries and checked that they agree, with identical data and priors, independently verified densities and gradients, matching posterior estimates and closely matching recommended plans. PyMC still fits this model faster. We’d use Foerster where we want that model to work with the application’s logic and managed state.

Running the application’s code still leaves us modelling choices to check. In this study, we fit demand and capacity separately and treat them as independent, and we supply the cancellation and cost assumptions. At each candidate budget, we choose the channel mix for revenue, then use the warehouse simulation to compare budget levels. We’d need a further step to optimise channel mix and profit jointly. More application detail would earn its place when it changed the decision or exposed a failure in the simpler model.

Bring us a decision

We start with the investment your team wants to make and the decision it keeps coming back to. Then we look at the spending and outcome history, campaign changes, promotions and any lift experiments. Before fitting a model, we ask what those records can tell us and where another experiment would help. The aim is a review your team can repeat, with a documented model, checks against held-out data, feasible plans and the uncertainty about their differences. Agreeing on how to evaluate the recommendation in practice is part of that work.

To connect the campaign to its business outcome, we work with the people who own fulfilment or other relevant processes to identify the state and rules that affect the decision. A first simulation may be small, like Kettle & Pine’s warehouse. Where compatible application logic already exists, we can help it participate directly in inference and plan evaluation, and check the results against observed operations.

If your team is preparing its next budget, get in touch with the decision, the history you have and the constraints that matter. We can help with PyMC-Marketing in Python, or with Foerster where the analysis needs to run inside your business processes on the JVM.

All data in this article are synthetic. The PyMC-Marketing results use version 1.3.0 with PyMC’s default NUTS sampler and four chains of 4,000 draws. The executable study, its validation records and the performance measurements are in the Foerster Marketing repository.