In science, the results of a properly conducted study or experiment can always be reproduced. Though many cite the Insurance Institute for Highway Safety (IIHS) studies on red light cameras as definitive, a new peer-reviewed journal article finds the conclusions did not hold up when subjected to re-analysis. In the latest issue of Health Behavior and Policy Review, University of South Florida (USF) Professors Barbara Langland-Orban, Etienne E. Pracht and John T. Large examine the 2011 IIHS report that claimed automated ticketing machines saved lives nationwide (view IIHS study).

In the 2011 study, the insurance industry took fourteen cities with red light cameras and 48 that did not have them. It compared results between the "before" period of 1992 to 1996 and the "after" period of 2004 to 2008. The results were then adjusted for each city's land and population density. In re-crunching the numbers, the USF researchers discovered the photo ticketing cities started off with a 50 percent higher red light running fatality rate in the "before" period. This left the IIHS results vulnerable to the statistical error known as regression to the mean, which refers to how the abnormally high accident rate has a natural tendency to fall back to normal levels on its own. IIHS credited this fall to the cameras. For instance, fatalities were so numerous in Phoenix, Arizona from 1992 to 1996 that the results were significantly skewed.

"A single camera city accounted for over half of the observed variance that the IIHS study sought to explain in the sample," the USF researchers wrote. "For internal validity, red light camera and non-red light camera cities should have been matched such that the fatality rates were similar in the 'before' period."

On the other extreme, six of the cities without cameras had either zero or one fatal red light running accident in the before period. That made it impossible for those camera-free cities to report any significant decrease in the after period.

"Whereas extreme rates can be used in research, they should be selected such that the 2 groups (treated and comparison) are similarly extreme, either high or low," the researchers wrote.

After outlining the statistical flaws in the model used by IIHS, the USF researchers applied more rigorous statistical techniques to the original IIHS data. The revised analysis considered 31 cities without cameras that had fatal red light running crash rates equal to 13 camera cities in the before period (Phoenix was excluded as an outlier). With these changes, the purported benefits of cameras disappeared.

"When using models with greater statistical rigor to assess the association of red light cameras with both fatal red light running and total fatal crash rates at signalized intersections, red light cameras were not associated with differences in either fatal red light running or total fatal crash rates," the USF researchers concluded. "Further, the IIHS finding that red light cameras were associated with a significant reduction in total fatal crash rates at signalized intersections could not be corroborated because the replications of their model concluded that the camera variable was not statistically significant."

The USF researchers note that their findings are consistent with a number of methodologically sound and independent analyses of red light camera results (view studies).