The Spanish Radio Association (SRA) launched a new attack on Arbitron’s PPM, saying the organization is disappointed with Arbitron’s response to its concerns about Hispanic audience measurement under PPM. “It appears that in trying to rush out PPM, Arbitron is sacrificing the accuracy of the data,” the group charged.
SRA – whose members include Border Media Partners, Spanish Broadcasting System, Entravision Communications and Univision Radio – has four main complaints about PPM. It charges that Arbitron: 1) Is not taking into account the diversity of the Hispanic population; 2) Has not shared information on recruitment to ensure that the diversity of the Hispanic community is reflected; 3) Is knowingly under-sampling the "cell phone only homes," which impacts Hispanic representation disproportionately; and 4) Is using the same model across markets and not looking at the diversity and demographics of each in recruitment and measurement models.
Arbitron late yesterday sent this response to RBR/TVBR:
"Arbitron has great respect and appreciation for the critical and unique role that and Spanish-language radio plays as a voice for the diverse communities across the country. For those reasons, we have had a long-term commitment to working with Spanish-language broadcasters in a collaborative and supportive manner.
Arbitron’s role as an independent research company is to provide stations and advertisers with information that is based on the actual behavior of radio audiences. That is what PPM delivers today.
We are confident that a fair understanding of the PPM system will demonstrate that it produces objective, unbiased audience estimates. The PPM is a more reliable survey instrument than the paper and pencil diary, which relies heavily on memory and recall. A paper and pencil ratings method often allows survey participants to overstate, intentionally or unintentionally, the time they spend with individual radio stations.
Our PPM samples are designed to effectively represent the diversity of the markets we measure in terms of age, sex, race, ethnicity and language preference. Overall, Hispanics and African Americans have the highest listening levels in the PPM system. Broadcasters that serve ethnic audiences and who have embraced PPM are succeeding with the timely and detailed data that only PPM can deliver."
What beefs do the Spanish broadcasters have with PPM?
SRA provided this lengthy point-by-point rebuttal to claims made by Arbitron on the website HispanicAd.com:
Arbitron: “On August 13, 2008 four of the country’s top 10 Hispanic markets, including the number one market, officially received their first pre-currency PPM report. Broadcasters are now seeing that some Spanish stations increase their ratings and rank position significantly in PPM.”
SRA: This is the exception, but not the rule across the Radio First PPM markets. Changes in station rank are not our concern. We fully expect that market dynamics may change and we accept that. Changes caused by the inaccurate measurement, however, are not acceptable.
Arbitron: “Arbitron first debuted PPM ratings pre-currency reports in New York last October and while an average of 8 percent of our Hispanic panel has been replaced each month with brand new Hispanics, the ratings results in this market have not changed dramatically.”
SRA: The extreme variability of these ranking data make the above statement highly questionable.
The only thing stable is change. This makes local market planning and buying of radio advertising and the verification of radio audience delivery month to month tenuous, if not impossible.
Arbitron: “Today, the Hispanic PPM sample has also improved to a point where, for the past several months in New York, it has consistently out-performed the diary on nearly every measure.”
SRA: This is a very broad statement that needs further clarification. We invite Arbitron to provide specific examples of which metrics they are citing. Does that include response rates? We expect the PPM measurement service ultimately to be better than the diary measurement service.
Arbitron: “Our work is never done and we will continue to put forth an earnest effort to work together with Broadcasters and agencies to improve areas that can benefit from it.”
SRA: Is the advantage of commercialization a benefit to advertisers or to Arbitron? Considering the amount of work that needs to be done and Arbitron’s timetable for improvements, we urge Arbitron to reconsider its rush to mass commercialization of its PPM products.
Arbitron: “At the same time that we look to the future, I want to take this opportunity to point out the facts on PPM samples today;-In the wave of markets rolling out now, PPM has the largest local market electronic media ratings panel ever maintained. The Hispanic sample size is more than twice the size of the current Nielsen LPM Hispanic panels.”
SRA: This is comparing apples-to-oranges. Nielsen meter counts are based on households – not the number of persons or TV’s within a household. Sample size does not equate to accuracy. A large sample that improperly represents a market is not better than a smaller sample that properly represents the market. Nielsen has learned that when one is relying on panel measurement, it is very important to have accurate market representation. There are more local radio stations in a market than local TV stations, therefore it requires a larger sample. Additionally, a fair comparison with Nielsen would be to look at the same level of detail about the sample they provide. This was one of the SRA’s first requests from Arbitron and they have set no timetable for delivering that analysis.
Arbitron: “The Spanish Dominant Hispanic PPM sample across all 10 deployed markets including New York and L.A. is over-representative of the Hispanic population and performs higher than the diary. The average PPM proportionality in June 2008 was 119 percent where 100 is the goal. The diary method achieved 101 percent in the last survey available survey for all of these markets”
SRA: Arbitron claims a PPM average index of 119 proportionality in New York and Los Angeles. We have difficulty analyzing and interpreting Arbitron’s response because their numbers did not take into account the reissue of San Francisco and San Jose PPM data due to implementation of “outdated” language usage populations that were used to process the July 2008 data. According to Arbitron, this mistake affected ethnic weighting in these two markets. Additionally, there are certain embedded markets in which we do not have the data to individually verify. This is not the first time inaccurate data was loaded for PPM, thus resulting in a reissue. These events do not instill confidence in the PPM system within the ethnic broadcasting or advertising community.
Arbitron: “Hispanic Panelists agree to participate in our PPM panels in higher percentages than non-black, non-Hispanics. We cannot say the same thing about the diary service.”
SRA: Agree and participate are two different things. Arbitron is assuming their Hispanic sample is 100% representative of each market’s Hispanic community. Our research has shown it is not (in terms of proportionality). After numerous requests, we are still awaiting specific information detailing Arbitron’s Hispanic recruitment procedures in the PPM service.
Arbitron: “Spanish Dominant PPM panelists participate in higher percentages each day than any other panelists. We cannot say the same thing about the diary service.”
SRA: Arbitron is assuming their Hispanic sample is 100% representative of each market’s Hispanic population. Our research has shown it is not (i.e. proportionality). We have concerns with how Hispanics are recruited for PPM.
Arbitron: “PPM includes individuals in Cell-Phone-Only homes, whereas the diary does not. These panelists tend to be Hispanic and young. The Cell Phone only population is a fast growing segment of the population and will be an increasingly important part of our panel.”
SRA: Arbitron states in the PPM Description of Methodology that they knowingly under-sample the Cell Phone Only homes. Sample balancing by re-weighting is not the way to fix this inequity. We know that because of the considerably lower age of the Hispanic community, the cell phone is of greater importance than any other segments of the market.
Arbitron: “Hispanic 18-34 year olds PPM panelist counts in June are at or above target in 8 out of 9 of our panels. Six are over 100 percent, two are over 97 percent and only Nassau-Suffolk is below target but climbing. The diary service is not performing at this level.”
SRA: Arbitron sets its target at 75% of the total installs. In other words, 25% of each market’s meters can fail to report usable data but that is the threshold Arbitron has set for 100% in each market. Our concern with this statement is that Arbitron is using a low self-imposed target as their benchmark. This is another example where two separate methodologies are inappropriate for comparison.
Arbitron: “Our Hispanic customers have asked for additional panel characteristics and controls in PPM, such as country of origin, more discrete language weighting by age groups, and in-home, in-person coaching. These measures and controls do not exist in our diary service and they would be yet another way that our electronic service will surpass all former radio ratings collection systems.”
SRA: We were not consulted in the development and design phases about the unique characteristics of the Hispanic community and how to best measure their listening behavior. After the fact, we have suggested procedures that are critical to Arbitron’s new methodology in order to accurately represent the audience.
Arbitron: “Despite the facts noted above, a concern has been raised that PPM "must be fixed" and must "be made right.‰ Those who have not had an opportunity to see the facts but have seen the headlines in industry trade magazines might assume there is a bias against Hispanics in the ratings. The reality is that in PPM Hispanics have the highest listening levels of any ethnic group. Hispanic stations have among the highest time spent listening in PPM.”
We do not disagree that Hispanics listen longer. We disagree with the way Hispanics are recruited. Hispanics cannot be simply classified by language. We have been asking Arbitron for many of their “facts” . . . and, we are still waiting.
Arbitron: “Hispanic stations total weekly count of listeners increase virtually across the board in PPM. Rather than asking what is wrong with the ratings, perhaps the better question is, “why don’t my ratings look like the dairy? The difference in ratings between diary and PPM has been quantified and it is because of the measurement tool.”
SRA: This is not our point. We strongly believe that additional initiatives need to be implemented in panel management to maintain accurate Hispanic representation. We never expected the PPM results to look like the diary produced data. We invite Arbitron to share the data that quantifies the ratings differences are a function of the PPM instrument and not due to the sample alone.
Arbitron: “After electronic measurement, the Diary is the best radio survey method available and it does a terrific job of measuring core radio listening habits. Yet, anyone who needs to understand why differences in radio listening exist between diary and PPM need only come to Arbitron headquarters to view how the American public records their listening in the diaries. The diary gets the job done well, but there is better way to deliver more granular and timely results with accountability that advertisers demand.”
SRA: Our issues are not with the principals of electronic measurement but with specific areas of implementation by Arbitron as it sees to introduce the PPM. It is significant to note that the only PPM market with MRC accreditation is Houston. A market with in-person recruiting, which Arbitron has indicated they will not use in any other markets.
If any of the pieces are wrong, then the overall outcome will be wrong. Therefore, if Hispanic is not properly represented, then all of the stations in the market will be impacted.