IRT FAQ

Q: What is IRT?
A: IRT or item response theory, is a mathematical model established 50 years ago in the field of psychometrics.  It is used to measure the relationship between unobservable traits, e.g., safety culture, and their outcomes.
 
Q: What data is used to calculate IRT CSA scores?
A: IRT uses the same data as the SMS – inspections, violations, and state reported crashes.
 
Q: Does IRT use the same 24 months’ worth of data like SMS?
A: Yes, the FMCSA has indicated they will continue to use the same 24 months’ worth of CSA data and SambaSafety is following their guideline.
 
Q: How does IRT affect Compliance, Safety, Accountability (CSA) scores?
A: IRT replaces the current scoring methodology, Safety Measurement System (SMS), and uses data to measure carrier safety.  Carriers will have IRT scores per BASIC, but they will also have a single CSA Safety Culture Score.
 
Q: What does the CSA Safety Culture Score indicate?
A: The CSA Safety Culture Score represents a carriers’ overall safety culture.  The score allows carriers to see how their safety culture measures against other carriers.  It is determined by the carrier’s IRT BASIC scores and their Exposure Risk Index.
 
Q: What is the Exposure Risk Index?
A: The Exposure Risk Index represents a motor carriers driver population, number of driver inspections, power units, number of vehicle inspections, and vehicle miles travelled.
 
Q: Are there thresholds?
A: At this time, the FMCSA has not indicated a position on thresholds.
 
Q: What happens to Safety Event Groups?
A: Safety Event Groups no longer factor into calculating CSA scores.
 
Q: How are violation severity weights impacted?
A: Under the SMS methodology, violation severity weights are determined by expert opinion and remain mostly static.  Using the IRT model, severity weights are calculated from the data and adjusts as carrier safety behavior changes.
 
Q: How is the Pre-employment Screening Program (PSP) impacted by IRT?
A: At this time, IRT does not affect PSP since it does not score drivers.
 
Q: What affect does IRT have on Data Qs?
A: It is still very important to ensure that you review the data quality for your carrier.  Auditing CSA data and maintaining its accuracy maintains the integrity of the scores.

Q: How are crashes handled under IRT model?
A:  Crashes are scored using the IRT model and an IRT score for the Crash BASIC is provided; the crash data is the same used in SMS. 
 
Q: How will the new IRT scores affect the Inspection Selection System (ISS) score?
A: The FMCSA has not provided guidance on how IRT will or will not affect ISS.
 
Q: Will drivers be scored?
A: No, drivers will not be scored using IRT because the CSA program only measures motor carrier safety.