Assessment of the pilot’s functional state based on sensor signals of various nature

Abstract

The article presents the results of researches to identify signs of pilot fatigue. The correlations between the parameters of the psychophysiological state of the pilot and indicators of the quality of piloting were studied. To assess the pilot's state, signals of various natures were used: deviations of the aircraft control stick in the pitch and roll channels, electroencephalograms, estimates of emotional states and analysis of blink parameters from pilot’s face video images, determination of the position of the pilot's head. This is the peculiarity of this study, which is aimed at identifying the opportunities of information channels that are traditionally not used to assess the functional state of a pilot. For this reason, generally accepted medical indicators (pulse, blood pressure, temperature, galvanic skin response) were not considered.

Indicators of piloting quality were vertical and horizontal deviations from the glide path line during landing approaches on a aircraft flight simulator. To process experimental data, spectral densities of signals, frequency coherence functions, and the principal component method were used. Deep learning convolutional neural networks were used to analyze video images. As a result of the research, stable correlations were identified between indicators of piloting accuracy and operator characteristics such as frequency coherence functions of control signals, the ratio of the first and second principal components of electroencephalogram signals, the number of emotional states and the number of blinks. A set of parameters has been determined to estimate operator fatigue (with increasing fatigue, the degree of linearity of the relationship between the control signal (pitch control stick) and vertical overload decreases; when recognizing emotions and blinks, indicators of fatigue are the categories «Sadness», «Fear», «Angry», frequency and duration of blinks).