Robots that can see around corners using radio signals and AI

Penn researchers developed HoloRadar, a system that reconstructs hidden 3D spaces beyond robots’ line of sight.

University of Pennsylvania School of Engineering and Applied Science

FacebookXLinkedInWeChatBlueskyMessageWhatsAppEmail

 

image: 

HoloRadar uses radio waves to see around corners, allowing it to detect people at T-shaped intersections like the one pictured here. 

view more 

Credit: Sylvia Zhang, Penn Engineering

Penn Engineers have developed a system that lets robots see around corners using radio waves processed by AI, a capability that could improve the safety and performance of driverless cars as well as robots operating in cluttered indoor settings like warehouses and factories. 

The system, called HoloRadar, enables robots to reconstruct three-dimensional scenes outside their direct line of sight, such as pedestrians rounding a corner. Unlike previous approaches to non-line-of-sight (NLOS) perception that rely on visible light, HoloRadar works reliably in darkness and under variable lighting conditions.

“Robots and autonomous vehicles need to see beyond what’s directly in front of them,” says Mingmin Zhao, Assistant Professor in Computer and Information Science (CIS) and senior author of a paper describing HoloRadar, presented at the 39th annual Conference on Neural Information Processing Systems (NeurIPS). “This capability is essential to help robots and autonomous vehicles make safer decisions in real time.” 

Turning Walls Into Mirrors

At the heart of HoloRadar is a counterintuitive insight into radio waves. Compared to visible light, radio signals have much longer wavelengths, a property traditionally seen as a disadvantage for imaging because it limits resolution. Zhao’s team realized that, for peering around corners, those longer wavelengths are actually an advantage.

“Because radio waves are so much larger than the tiny surface variations in walls,” says Haowen Lai, a doctoral student in CIS and co-author of the new paper, “those surfaces effectively become mirrors that reflect radio signals in predictable ways.”

In practical terms, this means that flat surfaces like walls, floors and ceilings can bounce radio signals around corners, carrying information about hidden spaces back to a robot. HoloRadar captures these reflections and reconstructs what lies beyond direct view.

“It’s similar to how human drivers sometimes rely on mirrors stationed at blind intersections,” says Lai. “Because HoloRadar uses radio waves, the environment itself becomes full of mirrors, without actually having to change the environment.”

Designed for In-the-Wild Operations

In recent years, other researchers have demonstrated systems with similar capabilities, typically by using visible light. Those systems analyze shadows or indirect reflections, making them highly dependent on lighting conditions. Other attempts to use radio signals have relied on slow and bulky scanning equipment, limiting real-world applications. 

“HoloRadar is designed to work in the kinds of environments robots actually operate in,” says Zhao. “This system is mobile, runs in real time and doesn’t depend on controlled lighting.”

HoloRadar augments the safety of autonomous robots by complementing existing sensors rather than replacing them. While autonomous vehicles already use LiDAR, a sensing system that uses lasers to detect objects in the vehicles’ direct line of sight, HoloRadar adds an additional layer of perception by revealing what those sensors cannot see, giving machines more time to react to potential hazards.

Processing Radio With AI

A single radio pulse can bounce multiple times before returning to the sensor, creating a tangled set of reflections that are difficult to untangle using traditional signal-processing methods alone. 

To solve this problem, the team developed a custom AI system that combines machine learning with physics-based modeling. In the first stage, the system enhances the resolution of raw radio signals and identifies multiple “returns” corresponding to different reflection paths. In the second stage, the system uses a physics-guided model to trace those reflections backward, undoing the mirror-like effects of the environment and reconstructing the actual 3D scene.

“In some sense, the challenge is similar to walking into a room full of mirrors,” says Zitong Lan, a doctoral student in Electrical and Systems Engineering (ESE) and co-author of the paper. “You see many copies of the same object reflected in different places, and the hard part is figuring out where things really are. Our system learns how to reverse that process in a physics-grounded way.”

By explicitly modeling how radio waves bounce off surfaces, the AI can distinguish between direct and indirect reflections and determine the correct physical locations of a variety of objects, including people. 

From the Lab to the Real World

The researchers tested HoloRadar on a mobile robot in real indoor environments, including hallways and building corners. In these settings, the system successfully reconstructed walls, corridors and hidden human subjects located outside the robot’s line of sight.

Future work will explore outdoor scenarios, such as intersections and urban streets, where longer distances and more dynamic conditions introduce additional challenges.

“This is an important step toward giving robots a more complete understanding of their surroundings,” says Zhao. “Our long-term goal is to enable machines to operate safely and intelligently in the dynamic and complex environments humans navigate every day.”

This research was conducted in the Wireless, Audio, Vision and Electronics for Sensing (WAVES) Lab at the University of Pennsylvania School of Engineering and Applied Science, and was supported by the University of Pennsylvania.

---

Method of Research

Experimental study

Subject of Research

Not applicable

Article Title

Non-Line-of-Sight 3D Reconstruction with Radar

How HoloRadar Works [VIDEO] |

HoloRadar relies on compact and nimble scanning equipment, opening up real-world applications.

Credit

Sylvia Zhang, Penn Engineering

HoloRadar works by reconstructing 3D scenarios from the bounces of radio waves.

Credit

WAVES Lab, Penn Engineering

Open-source modular robot for understanding evolution

A cost-effective, customizable quadruped could help researchers discover the particular advantages related to the length and segmentation of animal limbs

University of Michigan

FacebookXLinkedInWeChatBlueskyMessageWhatsAppEmail

 

Photos of the robot

 

What is it about a cheetah's build that enables it to run so fast? What gives the wolf its exceptional endurance? 

 

While these questions can be partly answered through animal experiments, many contributing factors can't be isolated from one another. Now, a new tool has arrived: a highly customizable, open-source robot design called The Robot of Theseus, or TROT, developed at the University of Michigan.

 

Named in homage to Greek philosophy's "Ship of Theseus," the robot is composed of commercially available motors and 3D-printed parts, which can be rearranged to take on a broad array of designs. The plans address several pain points for animal researchers who might be able to harness robotics for biomechanical experiments, as well as for roboticists seeking more task-specific designs. Assuming access to 3D printers, the cost in parts and materials is under $4,000.

 

"In paleontology, we can go back and look at bones, but it's really difficult to understand how these changes in limb proportion, or in range of motion, may have affected the way an animal can move. There have been some really great insights on this question from robots that each mimic one extinct animal very precisely," said Talia Moore, assistant professor of robotics with a background in evolutionary biology and corresponding author of the study in Bionspiration and Biomimetics. "But each robot took years to design and construct.

 

"I wanted to make a robot that could easily shapeshift into several different extinct species proportions, so that we could compare them, and see how the evolution of those limb lengths and other features would affect their locomotion. With TROT, 60 million years of evolutionary changes in body size can happen in 20 minutes."

 

Usable, customizable and easy to measure

 

The modular robot plans and assembly guides offer three major benefits. First, they are usable by people without robotics degrees, with help from equipment that is available at many universities. As Moore pointed out, robotics offers insights into biological questions, but not many evolutionary labs have the benefit of robotics expertise.

 

Second, the robot's shape is highly customizable. While the published study focuses on four-legged designs, experimenters can change nearly any body segment—adding and removing parts, changing the range of motion and more. This means TROT can model most mammals and enable direct comparisons of variations on the same structure—for instance, between closely related extant and extinct species. And they can try out theoretical designs to determine whether they are biomechanically unfavorable or just untried by evolution.

 

Third, researchers mimicked the springiness and stiffness of muscular structures without actual springs or elastics, which can muddy measurements. TROT simulates this biological energy storage and return mechanism with backdrivable motors, which recover energy as they are driven backwards.

 

"Traditional robots are designed with an emphasis on industrial applications and are expensive to make. TROT was designed with ease of fabrication in mind," said Karthik Urs, a recent master's graduate in robotics and first author of the study.

 

"The overall part count is kept low, and most of the parts only fit together one way. That means that scientists can make most of the robot parts in-house with commodity 3D printers, assemble them and get to experimenting faster. It also makes the iteration process quick—key to enabling exploration in both robot and experimental design."

 

Isolating biomechanical factors that are tough to measure in animals

 

Moore was first inspired to make this robot when reading a 1974 experiment on running cheetahs and goats. Because the leg swings from the hip like a pendulum, physics holds that legs with more mass away from the hip, known as a greater moment of inertia, require more energy to redirect than legs that weigh the same but have most of the mass near the hip. This concept has informed the interpretation of evolutionary changes in legs—increasingly tapered limbs are likely associated with more efficient running.

 

However, the 1974 experiment showed that although a cheetah has a more favorable moment of inertia in its limbs, running costs nearly the same amount of energy as it does for a goat. Because so much else was different between these animals, Moore explained, the benefit from a lower moment of inertia was basically unmeasurable. In contrast, Moore's group varied only the weight distribution in their robot's limbs and was able to isolate the exact amount of energetic cost or benefit associated with that change. 

 

TROT is designed for research and teaching rather than for operational robot work—while some 3D-printed parts break easily, they are also easy to repair and replace. Still, the results of future studies with this robot could inform commercial designs. At present, most commercial quadrupeds have fore and hind legs of the same length and style, but this test robot could reveal how to optimize the legs for the robot's intended purposes and terrains, and quantify whether the gains are worth the increase in manufacturing costs.

 

Researchers and enthusiasts can download the plans for the robot from U-M. The printing instructions for the parts are largely written for typical resin 3D printers, known as fused deposition modeling printers, with a stereolithography printer needed for a couple of components.

 

Urs is now the lead spacecraft engineer at Argo Space.

 

Study: The Robot of Theseus: A modular robotic testbed for legged locomotion (DOI: 10.1088/1748-3190/ae3ec1)

 

---

Journal

Bioinspiration & Biomimetics

DOI

10.1088/1748-3190/ae3ec1 

 

Ancient American pronghorns were built for speed

U-M study shows American pronghorns evolved for speed long before the American cheetah arrived on the scene

University of Michigan

FacebookXLinkedInWeChatBlueskyMessageWhatsAppEmail

ANN ARBOR—The fastest land animal in North America is the American pronghorn, and previously, researchers thought it evolved its speed because of pressure from the now-extinct American cheetah.

But recently, that theory has come under fire. Now, a University of Michigan study examining fossilized ankle bones of ancient relatives of the American pronghorn has shown that the pronghorn was evolving to be faster more than 5 million years before the American cheetah appeared on the continent. The study, supported by the Michigan Society of Fellows and the U-M Rackham Graduate School, is published in the Journal of Mammalogy.

"There was a long-standing idea that pronghorn are so much faster than every predator in North America because of a predator-prey arms race between the pronghorn and the American cheetah," said U-M paleontologist Anne Kort and co-author of the study. "What our work was able to add to this story was that not only was the American cheetah not as cheetah-like as previously thought, but that pronghorn have this build for running that existed well before the American cheetah came about."

The researchers say this sheds light on how current artiodactyl relatives—artiriodactyls include camels, cows, deer and antelopes—and other mammals may adapt as humans push farther into wild landscapes and as the climate warms. First author Fabian Hardy, assistant professor at Slippery Rock University, says the findings have implications for how we develop future wildlife and livestock management practices, as well as conservation practices.

"This tells us something about what animals are going to succeed going forward, and where we are going to find them. Is the modern pronghorn going to continue throughout the region that it's found in now?" said Hardy, who completed the work as a graduate student and postdoctoral researcher at U-M. "They're still traveling long distances. They were set up for that 12 million years ago. So they'll probably do all right, even with urbanization and fragmentation, because they can move around efficiently."

The study focused on fossils found in the Mojave Desert's Dove Spring Formation, which was deposited from 8 million to 12.5 million years ago. During this time, the region underwent great environmental change. In a valley called the El Paso Basin, the landscape shifted from once unbroken forest into a patchy mosaic of woodlands separated by more arid grasslands. 

Examining the ankle bones of early pronghorn relatives, the researchers expected to see the bones shorten in comparison to the point of rotation of the ankle. This would suggest that they adapted to traveling efficiently across grasslands and between patches of forest. Instead, the researchers found that their ankle bones, specifically a blocky bone in the middle of the joint called the astragalus, remained unchanged. This suggests the pronghorns were able to move between patches of forest effectively rather than adapt to open grassland.

Working in a fossil assemblage from a geologic epoch called the Miocene, Hardy collected a range of bones from a span of about 4 million years. He met with Kort, who studies the interaction between environmental change and mammals' locomotor adaptations, to develop a project centered around them.

"From about 30 million years ago to present, we see mammals becoming increasingly cursorial, which means they're adapted to running," Kort said. "I was curious to see if you could detect that in this kind of smaller scale pattern of drying and opening environments."

The researchers compared the astragalus using a study that had been done on modern artiodactyls such as antelope and cows. That study showed that these animals, which lived in an open habitat, had a shorter astragalus, adapted for efficient running.

"We expected to see longer astragali in the beginning, and then it would transition to more running-adapted astragali in the end. But we did not find that," Kort said. "Instead, what we found was a community that just stays the same through the whole section."

The pronghorn relatives, which were the size of a slender, long-legged beagle, were large enough that they could move in and out of the El Paso Basin, Kort said, seeking patches of forest elsewhere that were still suitable habitat.

"Our guess is that they could just move around to compensate for this aridification and landscape change," Kort said.

Although the astragalus had not changed over the time period Kort and Hardy were examining, they did show evidence of already adapting to running—5 million years before the cheetah appeared. In fact, the earlier pronghorns have "effectively the same ankle ratio as the modern pronghorn. So they're really built very similarly, and really built for speed well before this American cheetah shows up," Kort said.

Eventually, toward the end of the Miocene, the small pronghorns died off—perhaps as a result of a "tipping point" in the ecosystem that led to irrevocable changes in the ecosystem.

"It's easy to expect evolution and this gradual change over time, but I think this idea that you might not even see a problem until it's too late is a good reminder of how these things work," Kort said. "Our findings also may provide contextual information for biologists who are in wildlife management and doing direct conservation. They may not directly use it, but it's almost like having a historical context to understand a political problem."

---

Journal

Journal of Mammalogy

DOI

10.1093/jmammal/gyaf089 

Article Title

Family-level ecometrics reveal functional trait stability in Miocene artiodactyls despite environmental change in the Mojave region