MRC meeting addresses AP Meter problems

TVBR Exclusive –

Will this affect accreditation?

Complaints about data reliability with Nielsen AP 2.0 (active/passive) Local People Meter (LPM) led to a July 24th meeting of the Media Rating Council (MRC) – specifically addressing problems reported in Boston and Phoenix. A study was conducted by Ernst & Young and the findings were presented. Among those present at the meeting/conference call were MRC Chairman George Ivie, ABC O&O Stations SVP/Research & Electronic Measurement Pat Liguori and Media General’s Director of Research Brad Nimmons. The system, which has become industry currency, may have a few problems that need to be addressed. Can the AP Meter accurately report viewing? Are the stations the problem, or is it the inability of the meters to pick up viewing from the assortment of boxes out there—i.e. DirecTV, EchoStar, Cox Cable, Qwest, etc?

A good part of the issues are software, others are simply whether or not it is installed properly and/or is correctly checked that it is validating/receiving all station codes in the market. Who is to blame when encoding problems occur?–it depends on whom you ask. We had also heard there were more problems than just in Boston and Phoenix—in a number of cities, in fact. Might the AP meter have been introduced prematurely because Nielsen was moving as fast as it could to keep up with the expanding digital environment, and the ever-increasing numbers of households buying DVRs? Nielsen may have had no choice—something had to get into the market to measure digital viewing. Some of the problems, as we’ll detail, actually lie with analog measurement.

Said Kim Ross, Nielsen SVP/Product Leader, A/P Platform, Media Product Leadership: “The introduction of the A/P meter was a significant and necessary step forward in measuring television the way people watch it in the digital era.  Without the A/P meter we would have had no way to measure DVRs, VOD and other digital television. As is sometimes the case with new product roll-outs, the initial A/P meter had some software glitches.  We established a special A/P meter team to address these issues and over the past year we have introduced new software enhancements that have addressed these issues.  We also discovered that a number of local stations around the country had been improperly encoding their programs, which had exposed them to potential miscrediting.  We implemented an aggressive education program to make sure all stations understand how to use the encoding equipment.  We also developed special diagnostics that can help us identify encoding issues at local stations.”

The Phoenix problem

The Phoenix market is a concern, especially with precipitous audience drops at Belo’s KTVK-TV Ch. 3 right after it went to the LPM. There were technical issues. Belo SVP Rick Keilty, who did not attend the MRC meeting/call, noted the problems in Phoenix were not just with their Channel 3. “Other folks were affected. And I do not think Nielsen did a particularly good job taking care of business in Phoenix, frankly. Just the idea that they had to create two ‘supersites’ (read below), I mean that is a highly unusual event. So therefore, where there’s smoke there’s fire.”

He added, “What you’ve heard about the AP Meters is exactly correct. It has limitations that, frankly, Nielsen is not necessarily willing to step up to the table and admit. However, that needs to be their position going forward and we’ve said as much. They need to be aggressively challenging those limitations the AP Meter has—because it is the technology of record and with markets going to 100% AP, that’s going to be the meter. There are problems with that meter, Nielsen has detected them, other companies have detected them. But more importantly, they need to get to all of that before they impact the marketplace–but that train has already left the station, so that’s not going to happen.

It’s reactionary. They have resisted the final rigorous scrutiny and seal of approval before implementing a new technology into the marketplace and they begin to manufacture currency. The jury is not in yet and it affects everybody who’s a stakeholder in the game. Indeed, they’ve proven with the situations such as the ones we’re talking about that it’s just not working correctly.”

Keilty says their POV is the burden of responsibility and quality control has shifted to the broadcaster and cable company that is buying the Nielsen service. “We’re the first company to admit that this is a challenging environment—it’s difficult today with all of the technologies at work and choices in viewing technology out in the marketplace. But they’re it. They’ve chosen to be this provider of currency that we use that supports an industry.”

Like Ross said, some of the issues include miscrediting of programming. However, Keilty had a different take: “They have their coding and encoding process which they certainly refer to. When there are gaps, they have methodology that they use for purposes of filling those gaps in. That’s a very proprietary and secretive process that few, if any, have gotten to look at. They write the code that basically says that if the technology doesn’t demonstrate that a certain pattern of viewing is going on then we have a way of determining it with some other variables. We’re left with unexplainable precipitous drops. We’ve demonstrated in Phoenix countless examples where they have miscoded programming—where it just wasn’t being credited to the right source. But I find that out and I tell Nielsen—it should work the other way around. They should have whatever processes in place to head that off. I shouldn’t have to be the watchdog. We have had to go, as a company, along with many other companies, through some fairly extensive additional costs and expense just to have checks and balances in place with Nielsen. That’s both an outside consultant human resource and we’ve had to go out and purchase technology that monitors what they do. We’ve installed that in all of our current LPM markets and we’ll install that same equipment as LPM comes online for us in our other markets because we simply feel the confidence level is not there.”

Ross tells us Phoenix was just not quite that big of a deal: “When we switched to LPMs in Phoenix, a small number of clients raised issues about potential A/P issues.  Because of these concerns, we spent a lot of time in Phoenix investigating every potential issue. We also established our first “supersite” in Phoenix.  In the end, we did find one small problem with the way one station was encoding a feed, which caused miscrediting during programs that were simulcast on other stations.  However, we found that the ratings impact of this and all other potential issues was very small.”

The Boston problems and super sites

A big issue in Boston surfaced in The Boston Herald where Nielsen made an error over New England Patriots ratings which were simulcast on Hearst-Argyle’s WCVB (ABC) and ESPN. WCVB found Nielsen was crediting 16 rating points of viewing to ESPN that actually belonged to WCVB. The discovery was made in the minute-by-minute ratings during local commercial breaks during the ESPN game when Nielsen would shift ratings to WCVB during the breaks and then go back to ESPN when the game came back on.

The ratings snafu involving three Patriots games simulcast on WCVB and ESPN "robbed the Boston TV station of advertising revenue estimated by one ad exec at potentially more than $2[M]," according to Jessica Heslam of the Boston Herald: “While manually recalculating ratings for the games, Nielsen researchers uncovered an encoding error that had wrongly credited ESPN with close to two million viewers who were actually watching on WCVB. Nielsen had previously credited viewers watching WCVB feeds of the game on cable, satellite and via telephone fiber, which represents 75[%] of the Boston market, to ESPN. The recalculation determined that WCVB’s household rating for Patriots-Ravens on December 3 should have been a 24.4, compared to the previously reported 8.5.”

Paul Donato, Chief Research Officer for the Nielsen Company, tells us that was a very unusual incident where people weren’t encoding. “People weren’t encoding and it was a simulcast.  It was a situation which I think brought to the attention to all of our clients the importance of ensuring they are encoding all of their distribution channels. Working with the clients what we’ve done is develop procedures.  We try to play now a very active role ensuring that everyone’s got all their distribution channels encoded correctly.”

The method Nielsen employs to check if there are problems in a market are super sites. They set one up in Boston for WCVB (as well as Phoenix) where they have televisions connected 24/7 to every possible receiving device/distribution source to make sure viewing gets recorded. Apparently that is where Nielsen found and acknowledged a problem for the station, which they now claim has been fixed. Meanwhile the February and March books were held up over it.

Donato said the two super sites were done as an ongoing interactive relationship with their clients and the MRC: "The MRC, I think, considers super sites a very effective way providing assurances that all the coding and everything is working correctly.  It’s always a matter of interpreting things but I think super sites are what I conceive would be a good thing to do.”

WCVB President Bill Fine declined any comment, because he’s part of the MRC process and isn’t allowed to, but does acknowledge that Nielsen’s ratings of WCVB have not been accurate and the ongoing situation is being addressed directly with Nielsen.

Ed Piette, GM, WBZ-TV Ch.3 Boston, also noted there is an issue in Boston as it relates to WCVB-TV Ch. 5 Boston (ABC) and WMUR-TV Ch 9 (ABC) Manchester, NH and how their ratings have been measured and parsed out between the two.

Says Ross on the issue: “In 2007 there were some occasional issues at [WCVB], which had been caused by improper encoding.  Once we discovered the improper coding, this issue was resolved.  Earlier this year, we also discovered a highly unusual technical glitch in which simulcast programming from one affiliate was, in a small number of households, being credited to another affiliate station in the Boston market.  This situation was resolved and a review of the data found that this glitch caused had only a small impact on the ratings for either affiliate.”

The IRC

Another problem is the "IRC" (Interrelated Harmonic Carrier) issue between channels 5 and 6. Cable companies often adjust the signal to remove "ghosting." However, the AP 2.0 meters have trouble measuring Ch. 5 in that system–it falls out of the range of their ability to measure it. It occurs only on homes where the cable is plugged directly into the back of the set, not via set top boxes.

Said Donato: “George Ivie and his staff in the interest of making sure our clients knew that we were buttoned up on this, were invited to participate in our monthly operations meeting for a day a few weeks ago. Quite frankly I can say this because it’s not audit-related: George wrote a very flattering note to the clients.  What the sentiment in the marketplace is and certainly based on stuff that I’ve read in the last couple of weeks I think there will be issues that pop up as we get new experiences in new markets.  We’ve just got to be really fast to identify any of those and make sure we’ve got them locked up immediately. I understand that we worked with our clients over the past several months especially as we go into new markets and work through idiosyncrasies as we characterize that market.”

Will this affect accreditation? We ask George Ivie

How will this affect MRC accreditation, if at all? Will they pull accreditation for the markets that are already LPMs and give accreditation going forward to markets about to become LPMs? We asked MRC Chairman George Ivie: Bottom line—with all of these glitches and “growing pains” could there be a danger of the AP Meter losing accreditation?

“I don’t think there’s danger of them losing accreditation,” Ivie explained. “First of all, that’s not really how we work.  It has to be an extreme circumstance where we would go to them and say we’re pulling accreditation.”

Well, look at the PPM with radio—Arbitron has failed to win accreditation in New York and Philadelphia. Houston is it, so far.

“Well that’s different–that’s a newly accredited service. I mean you wouldn’t believe how many thousands of hours…most people in the industry have no idea what we do to try to protect things.  But we spend thousands of hours with the PPM and we are working on the details with Arbitron and we’re just not going to accredit it until we feel like its right,” said Ivie.

But Nielsen is not in the same category?

“Nielsen is not in the same category and also mission-wise I’m telling you this company is trying to nail this,” Ivie stated.” If you think about their business they want to sell VOD measurement.  It is extremely complicated.  Nielsen has these stations buying encoders and that’s not trivial.  You need an encoder for all of your distribution paths so if you’re Rick Keilty and you’ve got a broadcast distribution path, you’ve got various digital streams.  You’ve got to buy an encoder for all of those and MRC doesn’t get into finances but you can probably find out how much an encoder costs.

Multiply that by the number of streams and if that encoder fails, the broadcaster understands that they’re reliant on the other engines [read below] in this meter to make credit.  There’s a reason why encoding is there because if the code is there, it’s nailed.  You don’t have to default to other engines.  You want to keep that encoder running.  These broadcasters are trying to work with Nielsen to make sure that this system is pretty failsafe.  I would say by and large it’s not an accreditation thing, it is about making the process move beyond minimum standards to real, real good. Because we’re all working for advertisers here. We want that business to be based on excellent numbers.  That’s really the kind of nuance that we’re dealing with. I can tell you that this has been a lot of energy for all of us.”

Ivie concluded, “I can tell you, the MRC does an amazing amount of good.  A lot of what you see coming out for rating services that we audit–and I’m not just talking about Nielsen, I’m talking about Arbitron, MRI, Simmons, Scarborough all the companies we audit–a lot of what you see being introduced by these companies we drive. Especially the quality improvement types of things.  What allows us to drive this improvement is the relationship that we have with these services where they cannot hide anything from us.  When we go to talk to Nielsen about the AP Meter we talked to them about incredible details related to this meter functionality that they wouldn’t tell anybody else.  We know exactly how the meter works.  We‘ve tested it in a lab every which way you can think of. We’ve tested it out in the field.  We go visit Nielsen households which as you know is a closely guarded secret.

We actively work with Nielsen and we are related through the AP Meter to improve quality and ensure accurate measurement. We’ve audited it, it’s in production in essentially all of Nielsen’s accredited meter services.  To say that it’s an accreditation barrier is not accurate.  We have accredited services that have AP Meters in them.  Having said that I think that Nielsen implemented the AP Meter because their old meter technology the Mark II Meter was not prepared.  It was designed in the day when digitalization, DVRs, On Demand and time shifting was not present to this extent. So they installed this AP Meter to address these concerns and in truth the MRC was pushing them to install this meter also. 

That meter had I think in excess of a decade of research and development behind it from Nielsen.  It has redundant engines in it so that if a station does not encode for a period of time there are backup engines that are supposed know primarily based on signature matching but if it’s not in a digital environment there’s also video codes.  There are backup engines that are supposed to be able to make credit with the absence of code.  As an industry I think we’ve all been focusing on that aspect of the meter to make sure that that works as well as say the pre-digitalization Mark II Meter.  Before all this digital stuff Nielsen’s Mark II Meter did a bang-up job.  It was installed and working on a television that thing it was extremely reliable and all we’re trying to do is make sure that we have the same level of reliability in the AP Meter.  They have an incredible amount of resources–it’s unprecedented as far as I know.  I’ve been associated with Nielsen as an auditor and as a director of the MRC for 25 years and the level of effort that Nielsen has devoted to making the capabilities of this AP Meter robust is unlike anything I’ve ever seen.  They have like half of their company really working on this.  They’ve also designated like a top guy within Nielsen who used to be their CIO.”

RBR/TVBR observation: Rest assured, most of the ratings falloff across all broadcast stations in any market cannot be attributed to AP meter problems. The sheer volume of added channels/new digital boxes/DVR, across the country are the biggest reasons, by far. The difference in sample is an issue as well. We hope the AP 2.0 meter problems will be solved as Nielsen continues to roll the system out nationwide. Currently, Nielsen has 10 LPM markets that are already accredited, another four that are actively in the process of being accredited and four others that are in the earlier stages of accreditation.

By the way, there is a new meter out there that could replace the 2.0. The question is whether or not Nielsen is going to buy it. We didn’t get much of a response on that issue, but will keep you informed.