Autonomous flight software aims to make search and rescue drones more effective

Darren Keralla typically flies drones on search and rescue missions for lost hikers or those injured in some kind of accident. But in July, researchers at the University of Colorado Boulder enlisted him in a search for traffic cones scattered across their campus — part of a test of autonomous flight software for search and rescue drones.

A military airplane had collided with another plane, spilling radioactive debris represented by six orange cones, according to the fictional scenario presented to Keralla and 33 other test subjects who flew drones for the study from June to August, half of whom had no prior experience operating drones.

Keralla, head of the drone unit for the volunteer Douglas County Search and Rescue Team in Colorado, controlled the test drone with a computer tablet loaded with the software. He was given 40 minutes to find the cones, all while avoiding fake helicopter medical evacuation traffic, tending to “distraction” tasks designed to take his attention away from the drone’s camera, and listening to radio communications from mock ground teams to pick up clues about the cone locations.

This setup was meant to mimic the workload and chaos of actual search and rescue operations. Hunter Ray, an aerospace engineering post-doc at CU Boulder and lead graduate researcher on the project, said the goal in developing this SAFARI (Search Autonomy For Aerial Robotic Intelligence) software is to enable drones to become more responsive partners to humans in search and rescue situations. This would free the emergency response experts on scene to focus on high-level tasks, rather than the details of flying or monitoring drones.

In addition to his work on the SAFARI team of Ph.D., master’s and undergraduate students and advising professors, Ray founded a company, Odin Autonomy, to develop an “Odin terminal” tablet. This device would be loaded with his own software, developed from a clean slate but incorporating concepts from the SAFARI research.

His goal is to sell it to search and rescue squads, law enforcement and eventually military customers.

Automating drone operations

Flying a drone isn’t difficult; most people can learn the skill in three to 10 hours — even sooner if they’re good at video games, said Ray. He has first-hand experience as a volunteer on the Boulder Emergency Squad. He leads its drone team and is trained for rescues in swift-water, underwater and ice situations, as well as vehicle extractions and wildland firefighting.

“The challenge, the thing that takes a long time to learn is more of that coordination side,” he said.

Knowing where a drone should be positioned to detect a fire on a building, angling for the right perspective for a SWAT team incident, or coordinating search and rescue teams for the best coverage in mountainous terrain are all nuanced skills that require much more experience and training than operating a drone, Ray said.

“That’s ideally where people would be able to focus more of their time,” he said. “You just want to be able to say [to the drone], ‘Follow the stream up the hill and go south of the farm.’”

When Ray started developing SAFARI three years ago, he noticed that some search and rescue operations might require up to three people to work with a drone: one person to pilot it and two others to direct the drone’s search, coordinate and communicate with others involved in the response, analyze images and other data collected by the drone, and collect other information about the incident.

He wanted the software to enable a single person with little or no piloting skills to direct the search with the off-the-shelf drones typically used by the responders. “You need these algorithms where a user can tweak and tune and change on the fly, depending on the mission and even where you are in the mission,” said Ray, whose SAFARI project received funding from the National Science Foundation’s Center for Aerial Autonomy, Mobility and Sensing.

“You’re working with a team that has a lot of other information and context that you need to account for,” he said. “You can’t just shove all the data into a box and then plan some path to find your person.”

One of the biggest development hurdles was data sparsity: How could the software learn the user’s overall intent based on just a few basic instructions at the start of a search? To help the software with that task, the researchers built a digital “reward map” that could be assigned to any given search area. The user could instruct the drone and its software to “go to these four places and not to these two places,” for example, Ray said, so the software could then fill in the rest of the map with areas of high interest and low interest using a machine learning model.

Based on those initial prompts, the software “can learn, ‘Oh, you want me to go to trails that are far from buildings and are in the open with no tree cover,’” for example, he said. The map can also be updated as new information comes in during a search. Once the reward map has been established, a planner algorithm can draw up a flight path for the drone that aligns with what the user wants.

Putting the software to the test

For the radioactive debris test scenario, the test subjects had access to three types of planners, each with a different level of automation: one where the user could just set up a reward map and let the drone fly off on its search; a less automated planner where the user set up a few waypoints and the software would plan the best routes between them, such as following a complex trail network; and the least automated level, where the user could direct the drone by joystick to stop and investigate a particular area of interest.

For Keralla, who has flown drones on about 75 actual search and rescue missions, the mock scenario rang true and the automation software proved valuable — he managed to find the cones.

“Even with a challenging scenario, the automation took a lot of the tactical workload off the flying side and let me stay focused on the bigger picture,” he said.

Keralla is familiar with many of the latest drone features for search and rescue, including night vision, laser range finders and cargo drops. In March, he flew a drone to help rescue two teenage hikers lost in a state park as darkness set in. He located them using their heat signatures with an infrared camera, pinpointed their coordinates for rescuers to reach them on foot, had one of the teen’s fathers talk to them through a speaker and lit their way as they walked to safety.

Last year, he flew a drone to the badly injured victim of a dirt bike accident in a wilderness area, then found and marked the coordinates for a nearby landing zone for a helicopter to airlift the man to a hospital. In the same wilderness area, he flew a drone to locate a 12-year-old boy who got lost while riding an all-terrain vehicle, spoke to the boy by speaker, pinpointed the location and then returned to the site to drop a thermal blanket, water and protein bars for the boy to eat while he waited for the ground rescuers to arrive.

As for the overall findings of the radioactive debris scenario, Ray and his team learned that users like the ability to switch levels of automation, similar to having the option of switching cruise control on and off when driving a car, he said. Also, users who accessed the reward map automation level had significantly more situational awareness, located more cones and found the drone operation to be less complex.

“They like and respond really well to having a lot of control of the autonomy,” he said.

Ray plans to submit the findings by January for publication in an academic journal yet to be chosen, then roll out a prototype of the Odin terminal in mid-2026 for field use by the Boulder Emergency Squad. His plan is to incorporate lessons learned from that prototype into a beta version that’s shared with other local agencies by the end of 2026.

As for future SAFARI iterations, Ray said he’d next like to research adding two-way communication between the user and software, so the software can ask for clarification about the user’s search intent.

“That’s going to enable a much more interactive experience and it’s going to approach the experience of working with a real teammate and not just a tool to assist you on the mission,” he said.