A system designed to help persons with motor disabilities use computers only through their eye movements may, perhaps in the not-so-distant future, help pilots of fighter jets lock-in on their targets faster and keep track of their mental fatigue.
Aircraft and Systems Testing Establishment (ASTE) and Indian Institute of Science (IISc.) are undertaking a proof-of-concept project to track eye movements of pilots and use it in an integrated dashboard where weapon control automatically follows the gaze.
The prototype at display at the Indian Air Force stall at Aero India 2019 includes a goggle fitted with infrared sensors that can, with relative accuracy, point where the pilot is gazing. Consequently, the cursor on their dashboard moves in the 3-D map. For a little more than a year, the two organisations have been working on algorithms to capture movement in real-time, despite challenges of vibrations and varying G-Force (Gravitational force) conditions of the cockpit.
Faster weapon firing
The objective is to cut down the time taken by pilots to perform tasks that require them manually moving their joysticks. “(For instance) To perform a weapon control task, the pilot shifts the eye focus point from external world outside windshield to a display inside aircraft cockpit and then uses the HOTAS control (a joystick that controls the dashboard). Eye gaze tracker can reduce time and pilot mental workload required for the task,” said Wing Commander M. Dilli Babu, a Flight Test Engineer from the Prototype Test Squadron at ASTE, who is conducting the research along with Pradipta Biswas, an assistant professor at Centre for Product Design and Manufacturing at IISc.
Testing of the concept has started, having covered sorties in fighter jets and the simulator in ASTE.
“There is a statistically-significant saving in terms of time, while with a head-mounted display half the time for the operation can be saved,” said Prof. Biswas.
The team at IISc had used similar techniques to make computers and e-learning more accessible to children with spasticity. The eye-gaze controlled interface allows children who have lost motor controls to control the cursor of computers to play games or for mental exercises.
“For children, we compromised on latency (that is, time for cursor movement to sync with the movement of the eyeball) while focussing on accuracy. For pilots, we have had to improve on latency too,” said Prof. Biswas.
Cognitive load
While faster firing of weapons for pilots is one aspect, another important aspect is to tabulate the cognitive load on pilots during tests.
Wg Cdr Babu says currently, flight test crew have to fill out a subjective questionnaire to assess mental loads a pilot may have in undertaking certain tasks.
“Tabulation of spare mental capacity, that is mind-space for a pilot to carry out operations apart from flying the plane, is important. There is no sensor to measure it yet. But, by tracking the eye movements of pilots, we can indirectly estimate the cognitive load. It opens a new paradigm in testing,” he said.
