IAB releases independent review of online site intercept studies


The Interactive Advertising Bureau (IAB) released “An Evaluation of Methods Used to Assess the Effectiveness of Advertising on the Internet,” which examines the methodology behind site intercept studies, a common form of consumer advertising effectiveness research. It is the first thorough, public assessment undertaken of the state of online ad effectiveness research.

The IAB also announced that it would be leading an industry effort to undertake additional research to ensure that brand-effectiveness studies adhere to established principles for the conduct of market and consumer research.

The report was written by Paul Lavrakas, Ph.D., a renowned research methodology expert, former Chief Research Methodologist for the Nielsen Company and author.

Through a detailed review of relevant academic literature and multiple interviews with experts at research vendors and media companies, Dr. Lavrakas concludes that three prevalent flaws in site intercept studies jeopardize their results:

•         The validity of Interactive Advertising Effectiveness (IAE) research is “threatened . . .by the extremely low response rates achieved in most IAE studies”

•         The research is also “threatened by the near-exclusive use of quasi-experimental research designs rather than classic experimental designs”

•         The research is compromised by “a lack of valid empirical evidence that the statistical weighting adjustments… adequately correct for the biasing effects” created by the two preceding issues.

Dr. Lavrakas said his interviews with researchers indicated they are aware of these methodological flaws and their implications, but have not implemented more rigorous methodologies because of the expense.

“If the phrase ‘marketing science’ is to have any meaning, participants in the ecosystem must demand that their vendors employ rigorous, tested research methodologies, even if doing so costs more,” said Sherrill Mane, Senior Vice President of Industry Services, IAB. “Ad effectiveness research is an important tool for marketers looking to assess the success of branding campaigns, and Dr. Lavrakas’s report is just the first step in the IAB’s efforts to improve the way brand advertising in digital media is measured.”

The IAB Research Council commissioned Dr. Lavrakas to provide a foundation for a comprehensive initiative that will improve online brand effectiveness research. As next steps, the IAB will:

•         Launch a cross-industry task force to help the interactive advertising ecosystem better understand how these studies impact the broader supply chain, and suggest ways to minimize potential inefficiencies they may create

•         Create a set of U.S. best practices for online brand impact ad effectiveness studies that will include recommendations for marketers, advertising agencies, research vendors and publishers

•         Use Dr. Lavrakas’s report to identify and develop valuable research for understanding and improving online ad effectiveness studies

“An Evaluation of Methods Used to Assess the Effectiveness of Advertising on the Internet” and more information on the IAB’s action plan for improving online brand impact ad effectiveness and can be found on the IAB website at www.iab.net/interceptstudies