A congressionally mandated study of how CSA scores are determined recommends improvements to the program, ranging from better collection of data on things such as driver compensation and vehicle miles traveled by state, to replacing the system.
The Safety Measurement System is used to identify commercial motor vehicle carriers at high risk for future crashes and is the basis for FMCSA’s Compliance, Safety, Accountability enforcement regime, known as CSA.
As reported by Heavy Duty Trucking:
A study by the National Academies of Sciences, Engineering, and Medicine recommended that over the next two years, FMCSA should develop a “more statistically principled approach” for the task, based on an item response theory (IRT) model. Also known as latent trait theory, it’s an approach that has been used for policy decisions in other areas such as hospital rankings, according to the panel.
Since it rolled out in 2010, the report noted that the system has been criticized for, among other things:
- Using highly variable assessments
- Not accounting for crashes where the motor carrier is not at fault
- Including carriers that have very different tasks in the same peer groups
- Using measures that are sensitive to effects from one or more individual states
- Using measures that are not predictive of a carrier’s future crash frequency
- Using measures that are not reflective of a carrier’s efforts to improve its safety performance over time.
The National Academies panel reported, “We have found, for the most part, that the current SMS implementation is defensible as being fair and not overtly biased against various types of carriers, to the extent that data on MCMIS can be used for this purpose.
“However, we believe some features of SMS implementation can be improved upon, and some of the details of the implementation are ad hoc and not fully supported by empirical studies. Many of these details of implementation would be easily addressed if the algorithm currently used were replaced by a statistical model that is natural to this sort of discrimination problem. “
Read full HDT story here.