Local broadcast and cable ratings increasingly are falling outside the traditional 10% benchmark error range that has served as the mainstay used in media buying negotiations. This, according to new study findings which show random error in Nielsen diaries, continues to grow.
Currently, the relative error for total-day in all diary-only household ratings for all TV stations falls within the 10%-benchmark range just 11.3% of the time. The ratings in primetime fall within the range 26% of the time and ratings for weekday evening and late newscasts fall within the range 18.1% and 20.7% of the time, respectively.
Moreover, as the diary population is subdivided into various age components (18-34, 18-49 and 25-54), the data become less stable than for household measurement, due to smaller sample sizes. The relative error is especially pronounced for lower-rated stations.
The study, for which Nielsen provided 11 years’ information covering 31 diary markets, along with posted advertising schedules, was conducted for the Council for Research Excellence (CRE), a diverse group of senior-level research professionals from throughout the media and advertising industries dedicated to advancing the knowledge and practice of audience measurement methodology.
The study, for which the diary market data was analyzed for the CRE by BIA/Kelsey, encompassed six May sweeps dating back to 2001.
Analyses included relative error (“RE”) ranges for all stations, affiliates, households and demographics; homes using television (“HUTs”) by daypart and demos; and RE by daypart, demos and effective sample size. The research team also conducted a series of interviews with local-television advertising buyers and sellers, and Nielsen experts.
“Our goal was to examine potential instability in ratings as evidenced by relative error,” said Billy McDowell, Vice President, Research, Raycom Media, who chairs the CRE’s Local Measurement Committee. “In using TV diary ratings information – still used in 30% of all television households – both buyers and sellers are challenged to predict ratings performance accurately. Diaries do provide an economical way of understanding the ‘uniqueness’ of each local market, but they are becoming less precise. Local TV needs measurement to evolve but we should also evolve the way current data are used. It is important to note this project did not offer remedies to the problems with diary measurement but to educate the industry regarding the current status of the diary. There are no ‘quick fixes’ for the diary methodology, although Nielsen continues to work on improvements such as the electronic diary and the newer ‘code reader.’ In the meantime, the statistical limitations should be realized when conducting business using these data.”
More highlights from the study:
–The industry, both buy and sell sides, accepts that Ratings should be viewed as point estimates with a margin of error to be associated with these estimates. By convention, the buying and selling on local television diary data accepts forecast Ratings bought and sold as equal to Ratings in the posting process if they are within +/- 10% of the Relative Standard Error, or the Standard Error as a Percentage of the Estimate Rating.
–Ratings delivered less than 10% of forecast trigger additional negotiation between buyers and sellers in successive buying rounds.
–Remedies may include pricing discounts or “make-goods” (i.e., Alternative Delivery Units (ADUs) or spots provided to compensate for under-delivery of promised audiences.
–In Total Day, the Relative Error of all diary-only household ratings falls within the acceptable range about 24% of the time, meaning three-fourths of all fluctuation can be attributed to statistical error.
–Because of generally larger ratings in prime time, they are more often within the sales benchmark, but still less than one-half of the time (46%).
–During the traditional local news windows (weekdays 6-7pm and 11-11:30pm) household ratings are virtually the same: within the sales range 39% of the time.
–The long-established +10% threshold for audience delivery guarantees are outdated given the diary’s increasing relative error rates.
The diary’s sweeps measurement frequency (4x/year) no longer reflects today’s year-round programming environment and makes it difficult to predict and report on programs that fall between the February, May, July and
November measurement periods.
–Solutions for reducing relative error rates possess financial barriers, lack the diary’s granularity and will be challenged by technology trends as consumers find more ways to watch more content.
–The buying and selling communities are still evaluating the viability of using Set Top Box data sources for transaction purposes.
–Some media buyers could consider replacing local TV with other media as TV diary accuracy erodes.
What can be done? Increased movement from phone-based sampling to address-based sampling has improved television audience-sample frames. However, as evidenced by differential response rates in sampled homes, non-response bias cannot be completely resolved by weighting measures and remains a persistent challenge.
“Nielsen’s address based sampling has reduced response bias but there remains a significant difference between metered and diary samples,” said Ceril Shagrin, executive vice president of Univision Communications, who serves as chair of the CRE as well as its Sample Quality Committee. “An improved sample frame improves representation of younger, cell phone-only homes — which are growing rapidly — but more needs to be done. Measures such as enlarging the sample size or weighting the results don’t adequately resolve the issue; you can weight demographically, but we’ve learned more clearly that you can’t sufficiently weight for non-response.”
