MIT Researchers Develop AI Tech That Allows Cameras To See Around Corners
One of the challenges that exists for self-driving cars is being able to anticipate the unseen, such as a child running out from behind a tree and into the road. But what if there were cameras mounted to the car that could somehow see hidden objects? Researchers at Massachusetts Institute of Technology's (MIT) Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a clever algorithm that allows cameras to detect people and objects hidden behind corners.
The implication of this technology extends beyond autonomous vehicles and into a wide range of fields, including emergency response situations. Rather than wander into an area without known what might be found on the other side of a wall or building, a camera that can 'see' around corners could provide vital information about the terrain, along with who and how many people might be on the other side.
This nifty technology examines light reflections to parse together a scene. Say you are walking down an L-shaped hallway and there is a wall between you and some objects around the corner. Those objects reflect a small amount of light on the ground, creating a fuzzy shadow called a "penumbra." To the naked eye, there is not much to discern. But using video of the penumbra, the "CornerCameras" algorithm can stitch together a bunch of one-dimensional images to reveal information about what is on the other side.
"Even though those objects aren’t actually visible to the camera, we can look at how their movements affect the penumbra to determine where they are and where they’re going," says Katherine Bouman SM '13, PhD '17, who is lead author on a new paper about the system. "In this way, we show that walls and other obstructions with edges can be exploited as naturally-occurring ‘cameras’ that reveal the hidden scenes beyond them."
The neat thing about this technology it does not require expensive equipment or projecting light into a space. It works even with a typical smartphone camera, and in a variety of lighting situations, both indoor and outdoor, whether it is sunny outside or rainy.
There are some limitations, such as environments where there is no light or where the lighting changes, as can happen when clouds move across the sun. But overall, it is a promising technology with potential applications in a variety of situations, such as helping firefighters find people in burning buildings.
The implication of this technology extends beyond autonomous vehicles and into a wide range of fields, including emergency response situations. Rather than wander into an area without known what might be found on the other side of a wall or building, a camera that can 'see' around corners could provide vital information about the terrain, along with who and how many people might be on the other side.
This nifty technology examines light reflections to parse together a scene. Say you are walking down an L-shaped hallway and there is a wall between you and some objects around the corner. Those objects reflect a small amount of light on the ground, creating a fuzzy shadow called a "penumbra." To the naked eye, there is not much to discern. But using video of the penumbra, the "CornerCameras" algorithm can stitch together a bunch of one-dimensional images to reveal information about what is on the other side.
"Even though those objects aren’t actually visible to the camera, we can look at how their movements affect the penumbra to determine where they are and where they’re going," says Katherine Bouman SM '13, PhD '17, who is lead author on a new paper about the system. "In this way, we show that walls and other obstructions with edges can be exploited as naturally-occurring ‘cameras’ that reveal the hidden scenes beyond them."
The neat thing about this technology it does not require expensive equipment or projecting light into a space. It works even with a typical smartphone camera, and in a variety of lighting situations, both indoor and outdoor, whether it is sunny outside or rainy.
There are some limitations, such as environments where there is no light or where the lighting changes, as can happen when clouds move across the sun. But overall, it is a promising technology with potential applications in a variety of situations, such as helping firefighters find people in burning buildings.
