Driver drowsiness ground truths compared: JDS, KSS, and EEG

From mid-2024, all new models of passenger vehicles in Europe require a driver monitoring system (DMS) that can detect drowsiness. Engineers across the automotive industry have generally used one of three ground truths to measure drowsiness in drivers.

Ground truth		Description
JDS	Johns Drowsiness Scale	Eyelid movements are a window into the cognitive state of an individual. Optalert’s Johns Drowsiness Scale (JDS) has been shown across dozens of studies to be the best predictor of a driver making an error due to drowsiness.
KSS	Karolinska Sleepiness Scale	Subjective measures such as the Karolinska Sleepiness Scale (KSS) can be used as proxy metrics that correlate somewhat with the increase in relative risk.
EEG	Electroencephalography	Electrical activity in the brain is measured by electroencephalography (EEG), which becomes compromised in driving scenarios. EEG usually requires electrodes to be connected to the scalp, so all EEG-based drowsiness detection systems must use data from a non-invasive sensor to estimate EEG signals.

These three ground truths are vastly different in what they measure and how well they relate to the effect they are trying to measure in drivers: impairment. For this purpose, impairment can be defined as the increase in relative risk of a performance failure.

JDS: Eyelid movements and impairment

Johns Drowsiness Scale (JDS)

The Johns Drowsiness Scale (JDS) was developed specifically to protect drivers in real time. It was initially modelled around the laboratory-based Johns Test of Vigilance (JTV), which measures lapses in reactions to sudden visual stimuli. It was then refined by tracking performance failures in drivers in the field. Previously it had been understood that eyelid movements were linked to drowsiness. But nobody could have predicted just how accurately they would quantify the increase in relative risk during driving tasks.

“

No technology other than Optalert’s JDS provides a method for quantifying the ‘relative risk of performance failure’ in drivers, especially at its early stages.

– Dr. Murray Johns

The JDS is derived from the driver’s eyelid movements as captured via an RGB-IR camera in the DMS. The algorithm underpinning the JDS derives 64 mathematical parameters from the eyelid signal and tracks them over time. Its predictive power is valid across all ages, genders, and ethnicities, and has been deemed by Harvard Medical School to be commensurate with gold standard laboratory measures. Further, across dozens of rigorously controlled independent studies it has been repeatedly shown to be the most accurate ground truth for predicting performance failures in driving tasks. One example was a study published in 2023 by a team at the Institute for Breathing and Sleep (IBAS):

Subjects were kept awake for 32-34 hours before a two-hour drive on a closed loop track.
JDS predicted lane departures with a third of the errors of KSS (4% incorrect versus 12% incorrect).
KSS generated three times as many false alerts as JDS (18% versus 6%) at a cut-off for both systems that optimises for sensitivity (i.e., detecting all instances of dangerous levels of drowsiness).

	Specificity	False positive rate
JDS	94%	6%
KSS	82%	18%

When opting for 100% sensitivity (i.e., no false negatives) the JDS has a third as many false alerts (6%) as KSS (18%) in predicting wheels drifting out of lane. The JDS was the most accurate in detecting severe end of drive impairment with an area under curve (AUC) of 0.99 versus an AUC of 0.93 for KSS.

Johns Test of Vigilance (JTV)

Optalert’s founder Dr. Murray Johns and his team of researchers spent many years developing the laboratory-based Johns Test of Vigilance (JTV). Everything from the timing of the stimuli to the size and colour contrast between visual elements was fine-tuned to detect the exact form of cognitive failure that occurs when a test subject is impaired from drowsiness. The JTV isolates this fundamental cognitive breakdown that causes domain-specific performance failures, which in driving include things like lane deviations or failing to stop at a stop sign.

Optalert recommends that any drowsiness detection system is first validated in a laboratory-based psychomotor vigilance test (PVT) to determine that it predicts when lapses will occur. This is less costly and cumbersome than a final on-track validation that requires a comprehensive set of safety measures.

A screenshot of the Johns Test of Vigilance (JTV), which measures attentional vigilance on a short time scale (i.e., one to two seconds). This is critical in tasks such as driving in which a lapse in attention even shorter than one second can lead to failure.

Other eyelid parameters

Some engineering teams have used more rudimentary eyelid parameters to measure drowsiness. The most prominent example of this is PERCLOS, which is the percentage of time in which the eyes are at least 80% closed. This often increases during the very late stages of drowsiness, at which a driver is already dangerously impaired, but it cannot pre-empt and thus prevent high-risk situations. In addition, microsleeps can occur while the eyes are open.

Other teams have attempted to build something akin to the JDS using a far smaller number of parameters, but none of these have been validated to predict performance failures anywhere near as accurately as the JDS.

KSS: Subjective questionnaires

The Karolinska Sleepiness Scale involves a driver self-assessing on a 9-point scale ranging from 1 (extremely alert) to 9 (very sleepy, great effort to keep awake, fighting sleep). It has some major issues in preventing road accidents:

“

In terms of the KSS, people are not very good at identifying when they should stop driving because of drowsiness.

– Dr. Murray Johns

People are not good at identifying when they should stop driving because of drowsiness. Even experts and people experienced with the KSS exhibit this subjective problem. In repeated studies we see examples of people reporting a high KSS who are unimpaired and safe to drive, as well as people who insist they have a low KSS but are impaired and unsafe to drive. No modifications to a study protocol can resolve this inherent issue.
Asking someone for their KSS score invariably wakes them up. One of the best countermeasures to drowsiness is engaging in conversation. Asking someone how sleepy they are affects their drowsiness in a way that would not occur in real-world conditions. A study conducted by Torbjörn Åkerstedt, the co-creator of KSS, acknowledged this “activating effect”, and opted to take drivers’ KSS scores no more frequently than every 10 minutes.
It is not stable across different subjects or even different timeframes. One person’s KSS 6 may be another person’s KSS 8. Even the same person at the same level of drowsiness will report a different KSS based on exercise-induced fatigue, external stressors, or anxiety. No level of individual expertise can overcome this.

1	extremely alert
2	very alert
3	alert
4	rather alert
5	neither alert nor sleepy
6	some signs of sleepiness
7	sleepy, but no effort to keep awake
8	sleepy, some effort to keep awake
9	very sleepy, great effort to keep awake, fighting sleep

The Karolinska Sleepiness Scale was originally developed in 1990 by Torbjörn Åkerstedt and Mats Gillberg.

It has low resolution across both time and level. A person’s KSS score can be captured at most every five minutes. And only three values (7 to 9) are of concern for driver drowsiness. This lack of granularity is a significant impediment to building accurate data models.
Any model trained on the KSS as its ground truth is inherently subjective, even if it uses objective biomarkers to estimate KSS.
Furthermore, such models simply estimate what the driver already knows. In terms of accuracy, the upper limit of any model is the KSS itself. The real-world systems will likely sound many more false alerts and annoy drivers.

Although the European Regulation 2021/1341 details the KSS as the primary reference sleepiness scale, it has provisions to accept other scales. Four European technical services firms with Euro NCAP accredited test laboratories have confirmed the JDS is a valid alternative measure for homologation.

EEG: Electrical activity in the brain

Electroencephalography (EEG) has been used for decades in sleep laboratories. Its primary use has been in measuring sleep, rather than quantifying drowsiness in an awake subject. When used as a ground truth in driver monitoring, it has several issues:

“

The EEG can be used to identify the onset of sleep but cannot reliably quantify the impairment of drivers at the beginning of drowsiness.

– Dr. Murray Johns

It has very weak correlation with lapses or performance failures in psychomotor vigilance tasks, especially in the early stages. This holds across all published work exploring drowsiness measurement via EEG. Like rudimentary eyelid measures, it can detect when someone is dangerously close to sleep, but it cannot pre-empt and thus prevent high-risk situations.
It is not stable across populations. The EEG shows changes during sleep onset, but they are not consistent between different people. This is especially true in the early stages of the sleep-onset process that are relevant to driver drowsiness.
If used in-vehicle, movement artefacts are a problem. Most EEG work is confined to the laboratory because of the vibrations in vehicles.
Any model trained on a subject’s EEG reading is a secondary estimate. It will use facial features and possibly other biomarkers to estimate a subject’s EEG reading at any time. But until a car ships with electrodes that the driver attaches to their head and the in-vehicle noise can be removed from the signal, the DMS can only ever provide a secondary estimate of its EEG ground truth (like KSS models).

Some tier 1s may assert they have solved the issues with EEG, but nothing has been published to support such claims.

How can Optalert help you with measuring drowsiness?

We encourage everyone in the automotive industry to contact us for a more comprehensive list of the published literature on each of the above ground truths.

If you’re an OEM, get in touch to learn how you can integrate Optalert’s technology into your DMS and enhance or replace the current drowsiness solution. Blepharometry is also unlocking a wealth of health monitoring features for the vehicle of the future.

If you’re a tier 1, reach out to learn how you can integrate our drowsiness measurement technology via a software development kit (SDK) into your driver monitoring system (DMS). We are confident that in a rigorous test protocol that pushes test subjects to failure, it will outperform any technology in existence at predicting performance failures. But don’t take our word for it – put it to the test!