Tracking pilots’ eyes
By Jon Kelvey|September 2023
Since 2013, software designed by Australian company Seeing Machines has alerted drivers of luxury cars and commercial trucks when they are drowsy or inattentive. Someday soon, that same technology could be installed in passenger, cargo and military aircraft.
Today, the software is paired with optical and infrared cameras on the dashboard of 250-ton mining trucks and Cadillac brand cars. If the cameras detect that a driver’s eyes are drifting from the road or closing for a certain amount of time, the software triggers a seat vibration to alert the driver.
A similar setup in cockpits could do more than monitor pilot drowsiness, says Mike Lenné, chief science and innovation officer at Seeing Machines.
“What we’ll be is the mechanism to provide an understanding of what the human is doing, what they’re looking at, where their attention is,” says Lenné. “It’s really important as the levels of autonomy continue to go up in the aviation industry that there is a system in there that isn’t only about automation, but that there’s a system in there that’s actually concentrating on the human.”
Work is underway with Collins Aerospace, whom Seeing Machines struck an exclusive intellectual property licensing agreement with in May. Plans call for Collins to develop the cameras and other avionics required to install the Seeing Machines technology in cockpits. Initially, the software would only monitor pilot performance, but Seeing Machines believes the technology could one day be modified to interact directly with the flight controls. In one such scenario, automation could be prompted to take control of the aircraft if the software determines a pilot is incapacitated or unaware of an imminent danger.
In addition to airlines, Seeing Machines also plans to market its eye-monitoring software to pilot trainers — and the company has already garnered some interest. Since late 2019, the software has been operating in a Qantas Airlines 787 flight simulator in Sydney. Two eye-tracking cameras built by L3Harris, one for each pilot, were installed in the coaming of the glare shield panel that shades the cockpit displays. From this position, the software measures where each pilot’s eyes are focused, how often blinking occurs and the position of their heads and bodies, among other metrics. By gauging not only how tired the pilots are but also where they are focusing their attention when alert, “They can actually see exactly where the pilot is looking and how they make those decisions,” says Pat Nolan, the Seeing Machines general manager for aviation.
And Seeing Machines isn’t the only organization that’s identified a need for such technology. Researchers at NASA’s Armstrong Flight Research Center in California are studying ways to monitor breathing patterns and other measures to predict pilot performance. Across the pond, the European Union Aviation Safety Agency since 2022 has been studying whether it is possible to reduce the number of pilots in the cockpit. Under one of these concepts, Single Pilot Operations, a lone pilot would fly a cargo aircraft aided by increased automation.
Nolan says that although Seeing Machines does not have a position on single-pilot proposals, the eye-tracking software could be a useful tool for airlines and regulators looking to quantify the effects of making any changes to flight crews, including practices such as controlled rest or reduced crews for long-haul flights.
“It’s certainly not something that we advocate,” Nolan says of shifts to smaller flight crews, but “unless you understand the existing, how do you support the change?”
Related Topics
Avionics and Electronics
