Media ROI Evaluation Techniques

Techniques for tracking advertising and media ROI are often discussed by both advertisers and agencies as they seek to identify and maximise the ROI effect of media budgets. Deciding on which techniques to use can raise a number of issues depending on the data and budgets available for advertising evaluation.

Before we get into the techniques themselves you will see that one recurring theme is the impact of ‘extraneous variables’ – that is the impact of factors beyond or outside the media campaign itself. The six main extraneous variables are:

1. Competitor spend – almost all companies and brands have competitors. Competitor spends can have an impact on your campaign, usually by taking business away from you. You will need to quantify this effect before you can make statements about your campaign’s effectiveness.
2. Distribution – if you have uneven distribution of a product or service or if you have a shortage of a given product, this will impact on the results you report.
3. Economy – two economic variables are known to have an effect on marketing and media performance are interest rates and consumer confidence  – both of which can have a positive or negative effect on consumer spending. When confidence is high consumers spend more and campaigns perform better. You will need to understand how this affects your own campaign performance.
4. Seasonality – most markets have inherent seasonality which can have a powerful effect on sales patterns. If you are evaluating activity in seasonal peaks or troughs, you must account for the seasonality effect.
5. Pricing – price remains an important influence on consumer behaviour. If you have a 50% off sale, you will generate higher traffic and sales response than in a period of normal pricing. If your competitors are using pricing aggressively, this will also have an impact on your media ROI. You will need to take account of this.
6. Weather – many product sales are influenced by weather. Good weather can increase sales, bad weather can decrease ales, or vice versa. Again you will need to take account of this in any meaningful analysis of media ROI.

Why are extraneous variables important?

Extraneous variables can have a major influence on the results your campaigns generate. At best they can render any top-line results erroneous. At worse, the impact of extraneous variables could lead you down the route of making suboptimal media investment decisions which could damage your ROI.

Here is a short summary of the main media advertising evaluation techniques available and a note on whether or not extraneous variables are considered:

1. Linear data reporting
   • Involves the use of response codes, phone number tracking, SMS number tracking, drop downs or tick box menus to ask consumers how they found you
   • Tends to report only the last touch or last click i.e. the last thing the consumer remembers seeing before they connected with you
   • This tends to favour lower funnel channels like PPC and DM
   • Findings tend to reflect what the consumer thinks motivated them to interact
   • Works very well if you are only running one channel and one campaign, but can produce dangerously misleading results in a multi-channel environment
   • Limited to observed data only
   • Does not account for extraneous variables eg competitor spend, economy, pricing, seasonality or weather
2. Descriptive data reporting 
   • Basic counting of descriptive results e.g. web traffic during a TV campaign
   • Plots traffic before, during and after campaigns
   • Should be possible to measure and plot uplifts in campaign period (e.g. YoY)
   • Incremental traffic can be plotted against TV spend to calculate incremental CPC
   • Limited to observed data only
   • Does not account for extraneous variables eg competitor spend, economy, pricing, seasonality or weather
3. Uplift Analysis
   • Similar to 2 above but looks at the effects of media spend within a sales funnel or database
   • So, did sales conversion rates increase or bounce rates fall?
   • Did enquiries from current customers increase?
   • Did churn rates fall?
   • Does not account for extraneous variables eg competitor spend, economy, pricing, seasonality or weather
4. Correlation and Regression
   • Looks for basic statistical relationships in-campaign between media spend and a response variable eg web traffic
   • Allows advertisers to measure relationship between spend and response e.g. for every £1000 spend 2,000 clicks appear to be delivered
   • Limited to observed data only
   • Does not account for extraneous variables eg competitor spend, economy, pricing, seasonality or weather
5. Multiple Regression
   • Uses statistical modelling to estimate (I use the word “estimate” in a  statistical sense) the effect of multiple independent variables (e.g. adspend by channel) on a dependent or target variable e.g. web traffic new users
   • Can be structured to account for extraneous variables eg competitor spend, economy, pricing, seasonality or weather
6. Non-Linear multiple regression (AKA econometrics or Media Mix Models)
   • More advanced version of 5 above which incorporates the estimation of non-linear effects
   • Examples of non-linear effects include AdStock (the rate at which advertising spend effect decays over time) and diminishing returns (the rate at which adspend becomes less efficient as spend is increased).
   • Requires time series data covering multiple years to incorporate seasonal patterns in data
   • Requires at least 100 observations of data (e.g. weeks)
   • Will account for extraneous factors such as seasonality, competition, pricing or weather

All approaches are data dependent which means that if you are not collecting response or sales data you will need to.

Costs for implementation can vary but should always be viewed in the context of the potential savings that can be made from subsequent optimisation. For example, if an econometric media mix model costs £25k, but can optimise a £2.5m budget to save £500k, then the £25k is money very well spent.