Rescue
It is said that a survival rate declines greatly if the survivors are not rescued within 72 hours after the occurrence of a disaster. Therefore, the establishment of rapid search technology on survivors is an urgent matter. For this reason, we have been carrying out research to install a microphone array on the drone and apply the robot audition technique to it so that it can suppress the noise of the drone itself and noise such as wind, detect the voices of the survivors.
There are other searching technologies using a drone with a camera or camera-related device, but those approaches do not work at nighttime, in a dark place, and or when the survivors are under the rubble.
So far, in cooperation with Waseda University and Kumamoto University, we developed an all-weather microphone array and installed it in the drone to estimate the three dimensional positions of sound sources in real time. We obtain “sound visualization” interface by displaying the orders on the map.
We are also conducting research on “a technique to distinguish sounds related to human only from various sounds detected in the system.”
In the near future, we are hoping that drone equipped with this technology can be used under the noise to find the survivors in the disaster area, and it will be a promising tool.”
Offline Sound Source Localization with a Drone Embedded Microphone Array
Online Outdoor Sound Position Estimation with a Drone Embedded Microphone Array
In recent years, we have been studying a method that enables multiple drones to collaborate and perform sound source localization and tracking of multiple sound sources in the 3D coordinates. We have confirmed the effectiveness in the simulation.
Sound source tracking by multiple drones with multiple microphone arrays
Improving Automatic Speech Recognition under Drone Propeller Noise
Hearing Through the Noise
Before a drone can find someone by sound, it must first deal with one major problem: the drone itself is incredibly loud.
In this audio demonstration, you can see and hear the original recording, where a person’s voice is almost completely buried under rotor noise. Then, after our proposed processing method is applied, the voice becomes much clearer.
This is an example of sound source enhancement: making the important sound easier to hear by reducing unwanted noise.
Tracking a Moving Sound Source Indoors
Once the drone can hear useful sounds, the next challenge is to find where the sound is coming from.
In this demonstration, a small group of drones flies inside an indoor test arena. Each drone carries microphones. As a person moves around the floor while speaking, the drones work together to estimate the person’s position in three-dimensional space.
The chart in the corner shows the system’s live estimate of the person’s location, updating in real time as the person moves.
Finding Someone You Cannot See
In this demonstration, the sound source is hidden behind a wooden wall and is completely blocked from view. This is similar to a survivor trapped behind rubble.
A camera alone would not be able to find the person. However, because the drones are listening instead of only looking, they can still detect and track the hidden target using sound.
The display in the corner shows the system estimating the target’s position even though the target cannot be seen.
This is the central promise of drone audition:
hearing what the eyes cannot see.
A Full-Scale Rescue System Outdoors
This demonstration brings the system closer to a real rescue scenario.
Here, we use not only a swarm of drones but also a swarm of ground robots, creating a cooperative rescue robot system. Full-size drones fly overhead and listen for a person calling out across a field. The drones provide an approximate estimate of the person’s location.
This information is then passed to ground robots, including a wheeled robot and a four-legged robot dog. These robots move toward the estimated location and help pinpoint the target more accurately.
This shows how drones and ground robots could work together, guided by sound, to reach survivors faster and more safely than humans could alone.
Listening to the Ground
Until now, the drone’s loud rotor noise has mostly been seen as a problem — something we need to remove. But what if that noise could actually be useful?
When a drone flies over the ground, its rotor noise travels downward, bounces off the surface below, and returns to the microphones as an echo. Different surfaces reflect sound in different ways. Asphalt, soil, water, snow, gravel, and sand each change the sound differently.
By listening to the echo of its own noise, a drone can estimate what kind of ground is underneath it. This could be useful in fog, dust, smoke, or darkness, when cameras may not work well.
For example, a rescue team could use this technology to identify flooded ground, blocked roads, or unsafe areas without sending people there first.
We have tested this idea at a real outdoor site with many different surfaces, including water, gravel, snow, and sand. The results show that a drone can recognize different ground types using only the echo of its own rotor noise.
| Publications |
|---|
