How will neurotechnology and sensing impact automotive: part 2

Part 1 discussed how researchers use neurotechnology (magnetoencephalography or MEG) sensing to monitor adolescent drivers in simulated situations to determine how the inexperienced participants react to different driving conditions. Evaluating more experienced drivers in real-world situations is quite different. In the following examples, electroencephalography (EEG) sensing technology provides the analytical tool for in-car evaluation. While a great deal of electronics and software makes each one possible, the key distinction among them is the sensor.

Measuring driver attention

To study how driver attention or vigilance changes during longer car trips, the Mobile Neurotechnologies group of Fraunhofer Institute for Digital Media Technology (IDMT) in Oldenburg, Germany developed a system to measure brain activity. This technique also allows researchers to evaluate the differences on the driver between manual and autonomous driving and evaluate driver assistance systems.

In addition to an EEG-based system, other vital data, such as heart rate or blink rate, allows researchers to examine whether a change in the level of vigilance can be detected sooner through brain activity than via other signs of fatigue such as yawning. To make the measurements outside the laboratory in an actual vehicle and provide a comfortable sensor for the driver, they are developing flexible electrode grids to provide high-quality signals for long-term EEG recordings.

By combining around-the ear sensor technology with the mobile measurement system, researchers can measure and record data during a road test that can be evaluated afterwards or even in real time while driving. Making measurements using their specially developed flexible electrode grids requires a gel similar to clinical EEG measurements.

Figure 1. Fraunhofer IDMT Mobile Neurotechnologies group measures brain activity to determine the attention or vigilance level of the driver (Image: Fraunhofer IDMT).

Figure 2. Prior to donning his helmet, a race car driver has a flexible biosensing net placed on his head (Image: MindMaze).

EEGs for race car drivers

Tuning the reaction time of race car drivers takes brain sensing activity to another level.

Using a combination of virtual reality, motion capture, and EEG brain scans previously developed to assist stroke patients’ recovery, the Swiss company MindMaze worked with Formula 1 and INDYCAR SERIES racing teams to determine ways to enhance the race car driver’s performance and safety. Its MindDrive system monitors human performance by capturing brain data, both on and off the racetrack. A unique array of EEG and other sensors collect and transmit key cognitive driver responses to a trackside medical team on a real-time basis. The biosensing technique can also capture the pit crews’ neural signatures during the race to improve their responses as well.

A carmaker analyzes brain activity

Figure 3. In Nissan’s Brain-to-Vehicle technology, the driver wears a skull cap to measure their reactions to driving situations (Image: Nissan Motor Corporation).

To predict a driver’s actions and detect discomfort, Nissan Motor Company developed a system that detects, analyzes, and responds to driver’s brainwaves in real time. Its Brain-to-Vehicle technology can sense activity in advance of intentional movement (movement-related cortical potential (MRCP) as well as activity that reveals the difference between what the driver expects and what they are experiencing (error-related potentials (ErrP)). The sensed brainwave activity is analyzed and interpreted for immediate implementation by onboard autonomous systems.

By anticipating the driver’s intended movement, driver assistance systems can respond 0.2 to 0.5 seconds faster than the average human.

In-car EEG sensing — without a headset

Figure 4. With the sensor and its associated electronics concealed in the driver’s headrest, Neumo can measure brainwave activity as much as 12 inches (30.5 cm) away (Image: Neumo).

More recently, a US-based neurotechnology company announced an in-cabin sensing platform designed to transform driver safety and personalization for automakers. Without physical contact to the driver, Neumo’s EEG-based biofeedback system uses a patented brain activity analysis technique to assess and monitor the driver’s condition in real-time. Instead of being on the driver’s head, the sensor is mounted discreetly in the vehicle’s headrest to passively collect brainwave data. Developed over many years, the system detects raw signals, filters the raw signals to produce analysis signals relevant to EEG brain electrical activity, and attenuates unrelated frequency components. Established brainwave frequency bands (delta, theta, alpha, beta and gamma) range from less than 3 Hz to 100 Hz and are produced simultaneously and in combination, so band pass filtering separates the different bands for analysis.

With neuro data providing a level of insight into the driver’s condition and cognitive state that is not possible with a driver-facing camera, automotive manufacturers and fleet operators have enabling technology to significantly reduce accidents and personalize the driver’s in-cabin experience.