There is growing interest for implementing tools to monitor cognitive performance in naturalistic and everyday work environments. An emerging field of research, known as Neuroergonomics, promotes the use of wearable and portable brain monitoring sensors to investigate cortical activity in a variety of human tasks out of the laboratory. Recent technological progress has allowed the development of highly portable brain imaging systems. Moreover, advanced signal processing and machine learning techniques open promising possibilities for improving our understanding of cognition in real life situations and developing neuroadaptive technology to improve man-machine interactions.
Aviation operations constitute an ideal paradigm to implement this approach. Pilots deal with an uncertain environment and face complex interactions with the flight deck that can sometime exceed human capacity. For instance, accident analyses reveal that stress and emotion can impair attentional and decision-making capabilities to an extent that pilots perform irrational courses of action despite visual and auditory alerts. Research conducted in our lab at ISAE-SUPAERO, using fMRI, fNIRS or EEG, has started to identify the neural correlates underlying auditory alarm misperception or perseverative behaviors.
However, there is a need to investigate these phenomena in actual flying situations as laboratory or simulator settings cannot reproduce the real thrill of flying. It is also important to demonstrate that brain imaging techniques such as dry EEG systems can be used in such environments. Indeed, the cockpit environment is known to be particularly noisy due to electromagnetic interferences (e.g., GPS antenna, radio communication), vibrations (e.g., engines), and pilots’ muscular activity.
During one of our studies, we designed a scenario in which twenty-two pilots equipped with an Enobio dry EEG system (6 electrodes: Fz, Cz, Pz, Oz, P3 and FP) had to perform one low load and one high load traffic pattern along with a passive auditory oddball. In the low load condition, the participants were monitoring the flight handled by a flight instructor while they were flying the aircraft in the high load condition.
At the group level, statistical analyses disclosed higher P300 amplitude for the auditory target (Pz, P4 and Oz electrodes) along with higher alpha band power (Pz electrode), and higher theta band power (Oz electrode) in the low load condition as compared to the high load one. Single trial classification accuracy using both event-related potentials and event-related frequency features at the same time did not exceed chance level to discriminate between the two load conditions. However, when considering only the frequency features computed over the continuous signal, classification accuracy reached around 70% on average (for more details please see).
We also conducted a second study on detecting cognitive fatigue. Our objective was to test the possibility to combine dry EEG system with functional near-infrared spectroscopy (fNIRS) to improve the monitoring of this deleterious mental state. More precisely, we used engagement-related features (EEG engagement ratio) and connectivity features (wavelet coherence fNIRS based metrics) that are known to be particularly relevant to account for brain dynamics.
In this preliminary study, four participants were asked to fly four traffic patterns in a flight simulator and in an actual light aircraft. Along with the flying task, pilots had to perform a secondary auditory task. The two first traffic patterns were considered as the low cognitive fatigue class, whereas the two last traffic patterns were considered as the high cognitive fatigue class. As expected, the pilots missed more auditory targets in the second part than in the first part of the experiment, yielding that cognitive fatigue can impair auditory attention. Classification accuracy reached 87.2% in the flight simulator condition and 87.6% in the actual flight conditions when combining the two modalities. Taken together, these studies demonstrate that fNIRS and EEG-based passive BCIs can monitor mental states in operational and noisy environments (for more details, please report to).
