Iris, the new standard from Ipsos and UKOM, marks the start of a new era for online measurement in the UK. There’s a new supplier, a new user-centric approach, richer data and opportunities for campaign measurement, all delivered with new and improved analysis software and dashboard. However, making a fundamental change to a currency marks a point of inevitable discontinuity, a point of departure from the old to the new. This makes attempting to compare data between the two measurement systems unwise.
Over the last couple of decades, I have been involved in the transition of many measurement currencies around the world, either to a different supplier, an improved methodology or often both. Before each launch a lot of work is put in behind the scenes to check that changes to measured audiences make sense in the context of the new ways those audiences are being measured. Nonetheless, it is understood that those audiences will not be the same. After all, if they were identical, then what would be the point of moving to a new system? In this case, the new system has at its heart for the first time a single source, multi-device panel, delivering richer data in an improved reporting tool. Some element of discontinuity is a small price to pay for a significant upgrade to the system that drives online media planning in the UK.
Even so, once the new data is published, it can be hard for users to resist the temptation to want to compare; ‘are we up or down?’ – it’s only human nature. And yes, there will be differences, but to really benefit from what the new system has to offer it’s important not to look back, nor to attempt to somehow retrofit the new data into what has come before.
As a result, UKOM is wisely advising users not to look at trend data, not to create a form of exchange rate between the new and the old data. This does not mean that the old data is somehow wrong. Through defining and governing iris, UKOM has moved from one estimate of audiences to a different way of measuring those audiences that gets us closer to ‘truth’, but neither measurement is the truth.
I was once phoned by a panicked client who had seen two surveys on a topic that day that seemed to contradict each other and demanded “So Richard which one is right?”. My answer was that actually neither of them was likely to be the truth, but one of them would be closer to the truth than the other. The degree to which he cursed my pedantry after he put the phone down is not recorded, but I stand by my answer. Good currencies represent our best attempts to get close to the truth using the most appropriate and state of the art solutions available. They do not stand still.
So UKOM are wise to advise to regard this new beginning as year zero and warn against trending data across the transition. Yes, there will be a step change in some estimates, but attempting to trend across the two different currencies runs the risk of giving the false impression that an audience has gone up or down. Or that somehow the old estimate was right – “the truth” as my client put it – as that is what we were used to.
There are strong precedents for taking this reboot approach when a currency has a fundamental change.
In 1981 when the new BARB system was introduced and again in 1991 with a new meter panel, the industry agreed not to compare directly with previous audience levels and even to this day you sometimes see references to these watersheds in articles about “the biggest TV audiences of all time”.
In 2010 when BARB moved contractors but kept the core methodology similar, there was similar advice not to compare directly before and after. As it happened, during the first couple of weeks of the new contract heavy snow obscured most of the UK and, by trapping people at home, high viewing levels helpfully obscured any differences between the new and old systems.
For the same reasons I would advise against the creation of formal or informal conversion rate cards.
A decade ago I was heavily involved in the roll out of portable electronic measurement for radio around the world, which marked a radical change from diary measurement. Countries approached these changes in different ways. In the US, conversion rate cards were issued and initially the new data was retrofitted to look like diary data with the same metrics and reporting. To me this always seemed a bit like buying a new BMW and then camouflaging it to look like a Ford Escort. Or buying a racing car and then only driving it to the shops. By way of contrast, when Norway adopted electronic radio measurement, they leaned into the change and took the opportunity to report radio alongside TV in the same systems, with new metrics and actually saw an increase in ad revenue as a result of exploiting what the new system had to offer.
Painting an apple orange doesn’t make it an orange. Sometimes it’s best to accept it’s an apple and enjoy it. I would argue it’s important to embrace the change and make the most of this new system. UKOM has adopted a new way of gathering and reporting data that will have huge benefits for the online industry. Resisting the temptation to compare audiences between the old and new system is a small price to pay for this leap forward.
And if a client asks whether their audience has gone down or up? My advice would be to say, “Our audience hasn’t changed, but the way UKOM measure it has”.
Richard Marks consults on media measurement with his company Research The Media and is Research Director for asi who run conferences focused on global measurement issues. He is a former CEO of Kantar Media.
This article first appeared on Mediatel
Got questions about Ipsos iris? We've compiled a Q&A here.
