I often get asked for a POV on this, so I guess a lot of people are asking the same question.

Recap: What are Priors?

In a Bayesian media mix model, priors are simply what you believe the results might be before you look at the data or do any modelling. Think of them as expressing real-world knowledge — like “it’s very unlikely that my sales go down when I spend more on advertising” — in mathematical language. [5].

My view, in summary: Priors introduce risks of bias which can undermine model integrity — particularly when the priors are derived from last‑touch, platform‑reported metrics or poorly specified experiments. The core issues are that last touch platform reporting is biased to last touch and therefore inaccurate, and experiments are rarely undertaken with the levels of statistical rigour required to make them reliable.

Potential Advantages of Using Priors

Let’s look at the potential advantages of using priors.

1. The biggest argument for Bayesian Priors is that they reduce uncertainty in an MMM environment. Bayesian MMM doesn’t just give you a single point estimate for the impact of each marketing channel. It provides a probability distribution of possible impacts… This allows you to understand the uncertainty associated with the estimates, leading to more informed decision-making [6].
2. Model outputs (e.g. CPA by channel) look “right” and are relatable in the real world: To a large extent, priors fix the model outcomes into sets of values that seem to match the marketing team’s expectations and experience.
3. Priors can help when the data available for modelling is limited: Sometimes there isn’t sufficient spend data to accurate estimate coefficients for media channel performance. Spend may be too low, or the spend in one channel is drowned out by higher spends in other channels at the same time, or the number of weeks with spend may be very low. In these situations, priors can help by preventing coefficients from collapsing toward zero because the data is weak.
4. Priors help when spend variables are highly correlated: Many media campaigns feature multiple channels running at the same time campaigns often take place over 2–8-week periods with media channels being combined to extend reach or delivery efficiency frequency. This means input variables are highly correlated with each other which is a problem for regression models – they can’t isolate the effects of correlated channels. Using priors in a model helps the model find clarity by providing guides to what channels effects should be.

Key Risks and Limitations

1. Priors are beliefs, not data, so they risk injecting bias into your model: A key issue is that Priors are beliefs about media performance. Because they are beliefs about media performance e.g. “our Last Touch Social CPA is always between £10 and £20, so let’s fix that range into our MMM results” – we risk using those findings to inform the model outcomes – clearly this is a form of confirmation bias [7].
2. Priors often rely on inaccurate last‑touch or platform‑reported ROI Data: In practice, most priors come from platform dashboards rather than controlled experiments. The problem with platforms is that they generally provide last touch attribution (sometimes, first etc, but never full funnel i.e. including trend and seasonality an marketing mix variables for example).  This means that from the “get-go” Last Touch reporting is inaccurate – our industry knows that- so why use these results as priors in MMM?  But obviously using Last Touch platform sourced priors is only going to transfer that uncertainty directly into your marketing mix model. This is ironic because most MMMs are commission to provide an alternative view to Last Touch reporting.
3. Priors can pull MMM results toward last‑touch, and away from more statistically reliable findings: One of the big arguments for using priors is that they keep MMM results grounded in reality. But if you are using priors, you are creating an artificial result. If the prior is biased (e.g. towards last touch reporting), the model output will also be biased. Whilst this can make the MMM appear more aligned with platform reporting, but that alignment is artificial and the results are erroneous.
4. Priors can destabilise other coefficients: Let’s go back to high school maths. We know that equations have to balance on both sides – output e.g. Sales on the left, inputs e.g. media spend on the right. If we fix one or more of the input components of our equation on the right, other parts of it will have to move in order to accommodate that prior fix. This means your model will almost certainly inflate some media channels, suppress other, channels especially upper‑funnel media and produce inaccurate results
5. Priors reduce transparency: Stakeholders often struggle to understand how much of a coefficient is “data‑driven” versus “prior‑driven,” which can undermine trust.
6. Priors can mask genuine performance shifts: If a channel’s true effectiveness changes (e.g., due to creative fatigue, privacy changes, or market dynamics), a strong prior can prevent the model from detecting it. Equally performance might improve – dramatically. Let’s say you launch a new social media campaign with a new offer and new creative. Your MMM Social CPA falls from between £10 and £20 to £5. That means your social campaign has become much more effective. But MMM priors would likely exclude that result – or at least suggest it is unrealistic.
7. Priors Do Not Include Adstock or Diminishing Returns: There is a common belief that priors somehow incorporate adstock or saturation assumptions. They do not. Adstock and diminishing returns are structural modelling choices — they define how media works overtime and at different spend levels. They are often subjectively judged but they can and should be extracted from the MMM dataset itself using a grid search loss minimisation technique [8].   This can’t be circumnavigated by experiments as Adstock and diminishing returns are critical parts of advertising evaluation [9]. They can only be determined from  long time-series datasets, not short‑term experiments.
8. Priors can come from experiments, but these are notoriously difficult to get right: Priors can be defensible when grounded in rigorous, repeatable experiments. However, this is a technically demanding area. Getting marketing experiements right requires:
   1. Clean treatment and control regions
   2. No spillover
   3. Stable delivery
   4. Repeatability
   5. Sufficient spend, time and statistical power

In practice, achieving this level of scientific discipline in marketing experiments is difficult. Geographical regions can be more fluid than they look on a map – we live in a mobile world when people can move from one region to another in very short periods of time. [10].

Real‑World Example: Identical Creative, Different Results

There have been cases where a team ran an A/B test using the same creative in both treatment and control. Despite being identical, the experiment produced different lift results for each “creative.”  This highlights how algorithmic targeting, user heterogeneity, and data aggregation conspire to confound the magnitude, and even the sign, of ad A/B test result [11]. If identical creatives can produce different results, it shows how fragile and noisy marketing experiments can be — and why they often lack the stability required to serve as robust priors.

Conclusion:

I conclude with these points:

1. Avoid using Bayesian priors if you want a high integrity MMM.
2. Not using priors may produce more challenging results – but isn’t that what you want? You are building your MMM to get a different perspective on your marketing performance; to identify hidden opporutunities and to improve marketing ROIs. Why gloss over that valuable insight?
3. This may result in a more challenging conversation with the C-suite – but it’s safter to report a challenging result from a high integrity model than any result from a flawed model.
4. If you use priors to shape the answer, then the answer will look better. That doesn’t mean it’s the right answer, in as much as any model can produce the right answer.
5. Priors introduce confirmation bias into your model. If you believe the enemy of good modelling is bias, you must question the use of priors in your MMM project. They may be acceptable to some, but not to others.
6. If you are going to use priors, you must be very careful about where you source them. There are three main sources – Experience, Platforms and Experiments, but all have flaws and all run the risk of introducing bias into your model.

Implications for Marketers – checklist

• If you are going to use priors you must be certain that they are accurate.
• Be aware that in most marketing contexts the risks of priors being inaccurate are high.
• Don’t rely on last touch data for priors.
• If you use experiments, ensure they are properly specified, and even then, use them with care.
• In cases where you have too little data for full MMM, be careful about using Bayesian priors to overcome this problem
• In data light situations, consider reviewing your data, accepting lower granularity or looking at a different attribution modelling technique like regularisation.

References

1. J Martin and P Perez, Frequentists vs Bayesians and Marketing Science, Quantified Nation, July 2024 – https://open.substack.com/pub/quantifiednation/p/qn9-frequentists-vs-bayesians-and
2. Hits and Misses of Meridian – A Thorough Deep Dive, Aryma Labs Feb 2025 https://arymalabs.substack.com/p/hits-and-misses-of-meridian-a-thorough
3. Duncan Stoddard, Is Bayesian MMM worth the faff? DS Analytics Blog, February 2024, https://dsanalytics.co.uk/thoughts/is-bayesian-mmm-worth-the-faff
4. Two key problems that ail Bayesian MMM – Aryma Labs April 2024 https://arymalabs.substack.com/p/two-key-problems-that-ails-bayesian
5. Marty Sanchez, What Are Priors in MMM – And Why They’re Difficult to Get Right (But You Need To) Get Recast Blog June 2025 – https://getrecast.com/what-are-priors-in-mmm-and-why-theyre-difficult-to-get-right-but-you-need-to/
6. Rohit Nair, What is Bayesian MMM & why use it? Medium April 2025 – https://medium.com/@rohitnair.inft/what-is-bayesian-mmm-why-use-it-942d0193e7eb
7. Cognitive bias and data: how human psychology impacts data interpretation – Penn LLPS Features October 2025 https://lpsonline.sas.upenn.edu/features/cognitive-bias-and-data-how-human-psychology-impacts-data-interpretation
8. Nephade, D., A Predictive Modeling Approach to Multi Objective Marketing Mix Optimization: Balancing Performance, Acquisition, and Efficiency – in International Journal on Science and Technology (IJSAT) Volume 16, Issue 1, January-March 2025.
9. Gijsenberg et al., Understanding the Role of Adstock in Advertising Decisions, SSRN, 2011.
10. Tyler Buffington, Eppo, The Bet test – Spotting Problems in Bayesian A/B Test Analysis Dec 2024, https://www.geteppo.com/blog/the-bet-test-problems-in-bayesian-ab-test-analysis
11. Braun and Schwartz, Where A/B Testing Goes Wrong: How Divergent Delivery Affects What Online Experiments Cannot (and Can) Tell You About How Customers Respond to Advertising, Journal of Marketing, American Marketing Association, August 2024. https://journals.sagepub.com/doi/abs/10.1177/00222429241275886

The post Using Bayesian Priors in MMM: Do Better‑Looking Results Mean You Have a Better Model? first appeared on Marketing IQ.

A marketing Mix Model (MMM) can produce challenging results, especially if you have been relying on last touch( LT) attribution.  When presented with challenging results your first question may well be “can I trust the model?”.

The answer to this depends largely on the quality of the model as expressed through a set of diagnostics. The problem is not all diagnostics are shared.

Let’s explore MMM diagnostics in some more detail so you are better placed to form your own conclusions on model quality.

What you should get from your MMM partner

Let’s start with the diagnostics you should see from your MMM partner. They should present you with these test diagnostics:

1. Exploratory Data Analysis (EDA)

• Correlation Analysis (before modelling): Identifies highly correlated variables early, helping avoid multicollinearity in the model build and guiding variable selection – this is important because correlated variables can destabilise any modelling and results.
• Trend & Seasonality Checks: These checks confirm long‑term trends and seasonal peaks are properly captured before modelling media effects – this is important because if you don’t isolate these components, you risk incorrectly misattributing their effects to media.
• ACF (Autocorrelation Function): Ensures time‑based patterns are understood and removed so the model isn’t biased by serial correlation – this is also important. Your sales or revenue data can include patterns or “tidal effects” where your sales in the current weeks are partly related to sales in the previous week. Again, if you don’t control these relationships, your model will produce erroneous results.

2. Model Fit & Accuracy

• R²: This is a diagnostic that most marketers will be familiar with – it shows how much of the sales movement the model can account for by the model – but beware this is by no means a full diagnostic of the model.
• Adjusted R²: A fairer version of R² that penalises unnecessary variables, but again, not a full diagnostic.
• Actual vs Fitted plot which shows how well the model’s estimates match the actual data.
• MAE (Mean Absolute Error): The average size of the model’s prediction error in real sales units. For example, if a typical week delivers 1,000 sales and the model predicts values between 900 and 1,100, the absolute error is 100 units.
• MAPE (Mean Absolute Percentage Error): The model’s average prediction error expressed as a percentage of actual sales. Using the example above, an absolute error of 100 on 1,000 sales corresponds to a MAPE of 10%.

3. Residual Diagnostics

• Durbin–Watson test: Checks whether any time‑based pattern is left in the errors; values near 2 indicate no remaining autocorrelation
• Breusch–Pagan test: This test checks whether error variance is stable; instability can affect model coefficient reliability

4. Model Stability & Interpretability

• VIF (Variance Inflation Factor): Helps you measure the correlation between channels in a model and helps ensure channels aren’t too correlated to separate cleanly. If your model contains variables that are highly correlated with each other, it is not likely to be reliable.

These tests are all mission critical for evaluating both the data going into the model and the  model itself. But while they diagnose the quality of the model, they do not evaluate its predictive quality.  Predictive quality tests are further tests that assess the model’s ability to predict outcomes on previously unseen test data. In some ways these validation tests are the  ‘acid test’ in model diagnostics. When you combine the standard diagnostics (1-4 above) with the predictive quality tests you will see:

1. Good results in both mean you are dealing with a reliable model.
2. Good results in one but not the other should be cause for concern.
3. Poor results in both diagnostics and validation tests mean the model should be ignored and re-specified.

Unfortunately, the predictive quality tests are not always shared by MMM providers – I’ll leave you to figure out why.

So, what are the predictive quality tests?

MMM validation route 1: The holdout test

• How does the test work? In this test, we test the model on a subset of the data, usually towards the end of the sample. Let’s assume you have a model built on three years’ worth of weekly data – that’s 156 observations. You can ‘slice off’ the last 26 weeks and use the first 130 observations to “train” the model. You can then apply the trained model to the last 26 weeks, also known as the “test” period.
• What should you see?  You will have the actual sales data for the 26 week test period. The key ‘acid test’ question for you is how well the model’s prediction for those 26 test  weeks match actual sales in the test period. If you model is predicting within +/- 20% that’s a strong result. If it’s predicting between +/- 10% that is a very strong result.

MMM validation route 2: Cross Fold validation

• How does this test work? In this test we slice your 156 weeks of data into 10 consecutive time “blocks” of 15-16 weeks. In a similar way to the holdout test, we provide the explanatory variables for each block  and ask the model to predict what it estimates sales to be in each of these blocks based on the model coefficients. We then examine the prediction vs the actual and the size of the error in each block.
• What should you see?  We will have the actual sales data for each of the blocks. Again, the key ‘acid test’ question for you is how well the model’s prediction for those blocks matches actual sales in each of those blocks. If your model is predicting within +/- 20% that’s a strong result. If it’s predicting between +/- 10% that is a very strong result.  It’s worth noting that in some categories the 20% might be too tight, you could relax it to say 20-30%, but you need to be close to that range to have a validated model.

What do these results mean? If you see a model prediction within 20% of the actual in this holdout period, then you have a model that is working well. It is unlikely to be ‘overfitted’. Overfitting is a way of making the model look good within the sample, but it  does not perform well outside the sample and would therefore fail these tests.

The post How reliable is your marketing mix model (MMM)? first appeared on Marketing IQ.

Effectiveness and efficiency are not the same but they are both critical in media strategy, planning and activation

Intro

Marketing effectiveness and campaign efficiency are intertwined terms across the open plan workspaces of both advertisers and agencies. But they mean very different things. Now is a good time to remind ourselves what these terms mean and to explore the differences between effectiveness and efficiency and how they apply in media investment.

Effectiveness is about doing the right thing. It is sometimes referred to as ‘goal orientation’- are we doing the right things to reach our goal?  At m/SIX we refer to these effectiveness options as “levers”.

Let’s look at some examples of effectiveness; if we want to build sales revenue by growing the market share of a brand we need to increase its market penetration. To increase penetration, we need to move our brand into the consideration and preference sets of more consumers and in order to achieve this goal we need to deliver increased reach. In this case the goal of increasing reach is our route to effectiveness.  Actual effectiveness is the degree to which our approach delivers proximity to the selected goals – increased market penetration through increased reach.

In another effectiveness example we may wish to increase revenues by repositioning our brand versus competitors. For example we may wish to position our brand as more environmentally friendly than other brands in the category. To do this we may need to change the way consumers view our brand and ask them to associate new meanings with it.  In order to do this we may need to change the memory structures associated with our brand which in turn may require the use of media channels capable of delivering that “change in memory structure” goal.

In a third example we might want to deliver revenue growth by increasing purchase frequency. To do this we might need to give consumers reasons to purchase more often by reframing the way they use the product. This would typically increase the number of usage  occasions that the product can contribute to. The decision to reframe the way the product is used, and our success in doing that is the measure of the campaign’s effectiveness.

Efficiency is about doing things right. Efficiency tends to be process or ways of working orientated. At m/SIX we refer to these efficiency options as “switches”.

Now let’s look at how the three examples above might benefit from increased efficiency.

In the case of increasing market penetration,  we would need to examine which channels are able to deliver reach most efficiently – typically, we might ask which channels can do this quickly, or which channels can do this in the most cost-efficient way – how much reach and attitudinal shift can be generated per pound or dollar invested. Another aspect of efficiency might be which creative assets we use, exactly when we use them, where we use them and the time and cost involved in producing them.

In the case of repositioning a brand, efficiency might be measured as the number of points of attitudinal shift per £pound or dollar invested. We know that some channels are more efficient at achieving this goal than others. We also know that certain ways of using those channels are more efficient – a moving image may be more efficient than a static image, a larger format ad may be more efficient than a smaller format ad. Higher frequency over a short time period may be more efficient than lower frequency – or vice versa.

In the case of increased purchase frequency, the most efficient route might be how an agency and marketing team can remind consumers with prompts or triggers to change their behaviour – this is usually signals-based targeting; it could also be a carefully planned search campaign to target recipe searches for example. Or it may be a signals-based media and creative optimisation to target active meal planners; if we know that a consumer is going to shop online, we need to deliver our prompts and triggers in the right way and at exactly the right planning moments.

Efficiency and effectiveness is not a binary choice between one approach or the other – we have to deliver both, but in the right measures

Now we have explored these two concepts, we need to emphasise that one without the other amounts to suboptimal marketing and media investment.  Making effective strategic decisions without efficient delivery is likely to be slower and more expensive than it needs to be. Delivering campaigns efficiently, does not necessarily deliver the best goal delivery – ie effectiveness outcomes.

At m/SIX we manage both effectiveness and efficiency;  the levers and the switches. We have teams of strategic planners who are able to focus on making the right goal choices to maximise marketing and media effectiveness. We have teams of audience planners who look for the audiences most likely to deliver our goal and the channels and targeting criteria that will deliver those audiences in the most efficient way. And we have teams of display, search, social and CRO ad CX specialists who help us ensure that the strategy is delivered efficiently.

But whilst the choice is not binary, the balance between maximizing effectiveness and efficiency has to be carefully considered – our strategists and analysts work on optimsing this balance so you can be assured that your budgets are being invested in ways that will maximize your overall business outcomes.

WE OFFER MEDIA ATTRIBUTION MODELLING TO OUR CLIENTS: FIND OUT MORE HERE

The post Maximising Media Effectiveness and Efficiency first appeared on Marketing IQ.

With MMM models built, we are able to run scenarios and forecast outcomes from different types of media activity. This enables advertisers to optimise their media investments to maximise sales returns from media or marketing budgets invested.

Why is it called Marketing Mix Modelling?

MMM derives its name from the traditional ‘marketing mix’. The name recognises that all the variables in the marketing mix have a role in generating sales. So, let’s remind ourselves of the traditional marketing mix – the so called 4 Ps – Product, Price, Place and Promotion.

Product: In MMM, we might attach some attributes to the product – fast, smooth, light, powerful, low CO2 etc. These might be significant drivers within the category.

Price: Price is always a key determinant in consumer behaviour. As a general rule, products priced competitively sell more, and products priced less competitively relative to a category or competitors sell less. If we are to avoid confusing price effects with media effects, we naturally have to isolate the effect of price and take it out of the effectiveness equation.

Place: i.e. Distribution variables, whether they’re online (like delivery times) or offline (like store opening times) distribution variables usually have an effect on sales. If more stores are open, more product tends to be sold. If delivery times are long, versus competitors, less product tends to be sold. If we are to avoid confusing distribution effects with media effects, we also have to isolate the effect of distribution and take it out of the effectiveness equation.

Promotion: This variable is the promotional or advertising media spend variable. It can include advertising in media channels like TV, online display, search, social, OOH, cinema and print media.

What does an MMM actually look like?

This is a question many clients want to ask. MMM is often presented as a complex black box, when in fact it is simply an equation that captures the impact of the different elements of the marketing mix outlined above.  A Marketing Mix Model equation looks like this:

Sales = base (the levels of sales with no marketing) + (product attribute * a coefficient) + (price * a coefficient) + (distribution* a coefficient) + (promotion * a coefficient) + an error term.

Think of the coefficient is a statistically determined response rate which captures the rate at which sales are generated from changes in investments or activities in each of the Ps.

In statistics, the above might be written like this:

y = B0 + B1xX1 + B2xX2 + B3xX3 + error

How is MMM used to optimise paid media investments?

When we know the rate at which each “P” in our model generates sales, we can set values for each of the Ps excluding media advertising and then adjust the media advertising investment levels to see how those changes impact sales. This is a powerful tool in the marketer’s portfolio because it permits strong cases for media investment to be made. And that’s essential if to you want to maintain or secure increased budget from your CFO.

WE OFFER MARKETING AND MEDIA MIX MODELLING (MMM) TO OUR CLIENTS: FIND OUT MORE HERE

The post What is Marketing Mix Modelling (MMM)? first appeared on Marketing IQ.