Normalizing CSA (via Vigillo)
posted in Alerts by Brian Gray
In the previous post from this series, I talked about the benefits of a consistent, reliable driver scoring system that lets you see drivers from the FMCSA’s perspective. In this post, we’ll look at the first – and arguably, the biggest – problem a CSA driver scoring system needs to get right: Normalizing CSA Points.
Let’s get clear on what CSA points are first. Then we’ll talk about normalizing.
What do CSA Points Represent?
Drivers and motor carriers get CSA points for accumulating violations and crashes. Violations are given a 1-10 severity point weight, with 10 being the maximum – meaning the highest crash risk within that BASIC. So far, so good? You would think you can just add up the points, compare drivers and whichever driver has the most points is the one you should review first, right? Wrong! This is the most common misconception about CSA points. Here’s why:
Reason #1: Violations are given a 1-10 severity weight to compare against other violations in the same BASIC. That means a 10-point violation in Unsafe Driving is not the same crash risk as a 10-point violation in another BASIC, such as Fatigued Driving. Violation points cannot be meaningfully compared across BASICs. According to the official FMCSA SMS Methodology (the “rules” of CSA):
“Severity weights from one BASIC should not be added, subtracted, equated or otherwise combined with the severity weight of a violation from any other BASIC.”
~ FMCSA SMS Methodology, version 2.2
Reason #2: Different BASICs have wildly different numbers of violations – and Vehicle Maintenance gets many more violations (and violation points) than any other BASIC for most carriers. Here’s a pie chart from all Vigillo customers showing the distribution if you simply add up all violation points from different BASICs:
· 
Notice how Vehicle Maintenance (in red) account accounts for nearly 60% of total CSA points. This means if you are adding up points and using raw CSA points totals to prioritize drivers for review, you are focused on Vehicle Maintenance violations – even if all points across BASICs represented the same crash risk (which they do not – see above). Is that a good idea? It is fairly clear from the FMCSA’s CSA enforcement that Vehicle Maintenance is, at best, the 4th most important BASIC – behind the Fatigued Driving, Unsafe Driving, and Crash BASICs (based on tougher intervention thresholds and similar items). So why treat Vehicle Maintenance as though it’s the most important? Because adding points up seems to make sense. A bit of additional knowledge shows that adding CSA points together may be easy but it actually is counter-productive to effectively managing CSA scores. In short – CSA points are raw scores that cannot, in themselves, be compared against other CSA point totals for other BASICs.
Normalizing Explained
So – we need a way to meaningfully compare driver CSA scores across BASICs. That brings us to the concept of normalizing. We need a way to compare driver scores across BASICs. In fact, the phrase “meaningfully comparable” is an excellent way to think about the problem of normalization. An now we know simply adding CSA points won’t work.
An example: Imagine two students with two different teachers. Suzie, in Miss Scarlet’s class has 78 points. Johnny in, Mr. Jackson’s class has 138 points. Who is the better student? Answer: There is absolutely no way to know with the information provided. The two different classes may have totally different tests, point totals, and grading rules. What you need is way to meaningfully compare Suzie to Johnny. What would work? Here are a few examples of normalized scores that can be meaningfully compared::
– Suzie has an A minus while Johnny has a B plus
– Suzie has a 97% while Johnny has a 86%
– Suzie has 3.8 GPA while Johnny has a 3.2 GPA
So if you can’t meaningfully compare drivers using only CSA points, what can you use? The good news is that the FMCSA has given us an answer: They explain in the Driver Safety Measurement System portion of the SMS Methodology that all drivers are given a meaningfully comparable score called a BASIC Percentile score of 0-100 for each of the 7 BASICs where they have sufficient data. Hallelujah! Problem solved, right? Sadly, no. I’ll explain more when we tackle Driver BASIC Percentiles in the next post in this series.
