RBR’s lead story today (6/30) has Arbitron defending its sample size. That’s a story we’ve heard for years.
For most of my career, Arbitron defended ratings produced by the diary method. They used sample sizes so small that some demographics in some dayparts had little statistical significance. Arbitron claimed that replication studies showed that their results were more reliable than standard statistical analysis, and that the diary method and Arbitron sample sizes produced an accurate picture of radio listening. Now they defend the PPM and in some cases even smaller sample sizes with equal vehemence. Arbitron wants us to believe that their PPM methods and sample produce an accurate picture of radio listening. The problem is that this “accurate picture” is wildly different from the one produced by the diary method. You recently pointed out how significantly the reported ratings in Seattle changed with the introduction of PPM. Many of the top stations in the diary report dropped down near the bottom. Some previously low rated stations moved up near the top. The two methods produced quite different results.
So here is a question for Arbitron. You told us the diary methods and samples were sound. Now you say that PPM methods and samples are sound. Since the results are so different, which time were you wrong?
That is a rhetorical question of course. The answer is “both times.”