The recent news that more agencies are ditching AVE in favour of cost per thousand people reached (CPT) and other metrics is to be applauded.
Evaluation fit for digital
The great thing about CPT is that it acknowledges the established importance of digital; getting coverage on a specialist website might not be worth a lot as an advert but if it results in reaching lots of the right people few would argue that it is not a cause for celebration.
But there is more to this evaluation shake-up than CPT and to understand why, it’s important to explore the reasons for setting objectives and evaluating campaigns in the first place. Fundamentally it is to understand what is required/ desired so that we know the degree to which success has been achieved. There are two elements here:
- What is required/ desired?
- Have we have achieved success?
The role of marketing comms
If we take the first point, the problem with AVE is that it only makes sense if what is required/ desired is a cheaper way to buy advertising space. Whilst cheap ad space certainly has its appeal it is not the role of marketing comms which broadly speaking is:
Taking this definition, here are some of the metrics that allow us to evaluate whether comms are delivering:
This list is far from conclusive but starts to give a picture of the myriad of ways in which comms can be evaluated.
The case against AVE
So why no AVE? Surely it could be argued that AVE covers the ‘reasonable cost’ element?
The problem is that whilst AVE, when part of a range of evaluated metrics, should be relatively harmless it tends to dominate the focus of what matters.
Monetary ROI is important because PROs are hypersensitive to a perception that PR is somehow less commercially aware than other marketing disciplines. This means that they are also hyper-sensitive to making sure that monetary ROI KPIs are met. When the metric is AVE, this leads PROs to secure coverage that would have cost the most if bought as an ad. On the other hand, when the monetary ROI metric is CPT, PROs will focus on securing coverage that best reaches the audience. To paraphrase the old saying, ‘what is measured is managed.’
Within this context, we can see why so many agencies are ditching AVE entirely. We have been using CPT instead of AVE for years but have found the need for flexibility to ease some clients into the transition.
This is why even though the agency long ceased setting KPIs for AVE we still measure and report it if clients ask. The approach leaves us free to use CPT and focus on what is important but also give clients the AVE figure they sometimes need to provide internally (but with the caveat that not a great deal of store should be placed in it).
There is no doubt that the long-term future of evaluation is an AVE-free world but for the time-being at least it’s a compromise on the road to the Promised Land that many agencies find themselves making.
The importance of context
The second reason to set objectives and evaluate is ‘to determine if we have achieved success.’
In establishing which metrics to set we define ‘what is required/ desired,’ but to understand the degree to which we ‘have achieved success’ it’s important to understand why KPIs are set at the value decided and not just whether they have been hit.
For example, if Brand X desires the audience to believe that it is innovative, we might use Innovation Key Message Delivery as the metric and set 50% as the KPI. If 50% key message delivery is then achieved, how does anyone know whether that is good or bad? To someone without a PR background, only telling the audience that Brand X is positioned as innovative in half the coverage could be perceived as a poor result. However, if we add the why and reveal that the industry average shows anything above 40% key message delivery is ‘Excellent’* the results can be viewed in a clearer light. That isn’t to say that we shouldn’t always set our sights higher, just that it is equally important to understand what success looks like.
Context allows those without in-depth knowledge of a subject to have confidence in whether success has been achieved.
To illustrate this point, let’s take an example from a field we’re less familiar with. This is the Large Hadron Collider (LHC):
The LHC is a particle accelerator that is currently running at around half speed at a cost over £3bn and won’t reach full speed until 2014, over six years since it first began operating. Is that a good or bad result?
What if we consider that, even running at half speed, the LHC still set the world record for the highest-energy particle collisions ever made by man? Or that even at half speed the LHC helped make a new breakthrough that could lead us to understanding how the universe was created and why life exists? The context helps us to understand that reaching a 50% KPI is a phenomenal success and not an expensive failure.
So when we look at evaluation, the PR industry’s steps to focus on the right metrics are to be celebrated by all. But in order to understand whether the actual results are to be celebrated, let’s remember the importance of context and base KPIs on benchmarks that help everyone understand the degree to which success has been achieved.
*Beige works with independent evaluation agency Metrica, which sets benchmarks based on industry data and historical delivery.