High-speed video of experiments on a robotic hula hooper, whose hourglass form holds the hoop up and in place.
Some version of the Hula-Hoop has been around for millennia, but the popular plastic version was introduced by Wham-O in the 1950s and quickly became a fad. Now, researchers have taken a closer look at the underlying physics of the toy, revealing that certain body types are better at keeping the spinning hoops elevated than others, according to a new paper published in the Proceedings of the National Academy of Sciences.
“We were surprised that an activity as popular, fun, and healthy as hula hooping wasn’t understood even at a basic physics level,” said co-author Leif Ristroph of New York University. “As we made progress on the research, we realized that the math and physics involved are very subtle, and the knowledge gained could be useful in inspiring engineering innovations, harvesting energy from vibrations, and improving in robotic positioners and movers used in industrial processing and manufacturing.”
Ristroph’s lab frequently addresses these kinds of colorful real-world puzzles. For instance, in 2018, Ristroph and colleagues fine-tuned the recipe for the perfect bubble based on experiments with soapy thin films. In 2021, the Ristroph lab looked into the formation processes underlying so-called “stone forests” common in certain regions of China and Madagascar.
In 2021, his lab built a working Tesla valve, in accordance with the inventor’s design, and measured the flow of water through the valve in both directions at various pressures. They found the water flowed about two times slower in the nonpreferred direction. In 2022, Ristroph studied the surpassingly complex aerodynamics of what makes a good paper airplane—specifically, what is needed for smooth gliding.
And last year, Ristroph’s lab cracked the conundrum of physicist Richard Feynman’s “reverse sprinkler” problem, concluding that the reverse sprinkler rotates a good 50 times slower than a regular sprinkler but operates along similar mechanisms. The secret is hidden inside the sprinkler, where there are jets that make it act like an inside-out rocket. The internal jets don’t collide head-on; rather, as water flows around the bends in the sprinkler arms, it is slung outward by centrifugal force, leading to asymmetric flow.
The RAVEN walks, it flies, it hops over obstacles, and it’s efficient.
The RAVEN in action. Credit: EPFL/Alain Herzog
Most drones on the market are rotary-wing quadcopters, which can conveniently land and take off almost anywhere. The problem is they are less energy-efficient than fixed-wing aircraft, which can fly greater distances and stay airborne for longer but need a runway, a dedicated launcher, or at least a good-fashioned throw to get to the skies.
To get past this limit, a team of Swiss researchers at the École Polytechnique Fédérale de Lausanne built a fixed-wing flying robot called RAVEN (Robotic Avian-inspired Vehicle for multiple ENvironments) with a peculiar bio-inspired landing gear: a pair of robotic bird-like legs. “The RAVEN robot can walk, hop over obstacles, and do a jumping takeoff like real birds,” says Won Dong Shin, an engineer leading the project.
Smart investments
The key challenge in attaching legs to drones was that they significantly increased mass and complexity. State-of-the-art robotic legs were designed for robots walking on the ground and were too bulky and heavy to even think about using on a flying machine. So, Shin’s team started their work by taking a closer look at what the leg mass budget looked like in various species of birds.
It turned out that the ratio of leg mass to the total body weight generally increased with size in birds. A carrion crow had legs weighing around 100 grams, which the team took as their point of reference.
The robotic legs built by Shin and his colleagues resembled a real bird’s legs quite closely. Simplifications introduced to save weight included skipping the knee joint and actuated toe joints, resulting in a two-segmented limb with 64 percent of the weight placed around the hip joint. The mechanism was powered by a standard drone propeller, with the ankle joint actuated through a system of pulleys and a timing belt. The robotic leg ended with a foot with three forward-facing toes and a single backward-facing hallux.
There were some more sophisticated bird-inspired design features, too. “I embedded a torsional spring in the ankle joint. When the robot’s leg is crouching, it stores the energy in that spring, and then when the leg stretches out, the spring works together with the motor to generate higher jumping speed,” says Shin. A real bird can store elastic energy in its muscle-tendon system during flexion and release it very rapidly during extension for a jumping takeoff. The spring’s job was to emulate this mechanism, and it worked pretty well—“It actually increased the jumping speed by 25 percent,” Shin says.
In the end, the robotic legs weighed around 230 grams, way more than the real ones in a carrion crow, but it turned out that was good enough for the RAVEN robot to walk, jump, take off, and fly.
Crow’s efficiency
The team calculated the necessary takeoff speed for two birds with body masses of 490 grams and a hair over 780 grams; these were 1.85 and 3.21 meters per second, respectively. Based on that, Shin figured the RAVEN robot would need to reach 2.5 meters per second to get airborne. Using the bird-like jumping takeoff strategy, it could reach that speed in just 0.17 seconds.
How did nature’s go-to takeoff procedure stack up against other ways to get to the skies? Other options included a falling takeoff, where you just push your aircraft off a cliff and let gravity do its thing, or standing takeoff, where you position the craft vertically and rely on the propeller to lift it upward. “When I was designing the experiments, I thought the jumping takeoff would be the least energy-efficient because it used extra juice from the battery to activate the legs,” Shin says. But he was in for a surprise.
“What we meant by energy efficiency was calculating the energy input and energy output. The energy output was the kinetic energy and the potential energy at the moment of takeoff, defined as the moment when the feet of the robot stop touching the ground,” Shin explains. The energy input was calculated by measuring the power used during takeoff.
The RAVEN takes flight.
“It turned out that the jumping takeoff was actually the most energy-efficient strategy. I didn’t expect that result. It was quite surprising”, Shin says.
The energy cost of the jumping takeoff was slightly higher than that of the other two strategies, but not by much. It required 7.9 percent more juice than the standing takeoff and 6.9 percent more than the falling takeoff. At the same time, it generated much higher acceleration, so you got way better bang for the buck (at least as far as energy was concerned). Overall, jumping with bird-like legs was 9.7 times more efficient than standing takeoff and 4.9 times more efficient than falling takeoff.
One caveat with the team’s calculations was that a fixed-wing drone with a more conventional design, one using wheels or a launcher, would be much more efficient in traditional takeoff strategies than a legged RAVEN robot. “But when you think about it, birds, too, would fly much better without legs. And yet they need them to move on the ground or hunt their prey. You trade some of the in-flight efficiency for more functions,” Shin claims. And the legs offered plenty of functions.
Obstacles ahead
To demonstrate the versatility of their legged flying robot, Shin’s team put it through a series of tasks that would be impossible to complete with a standard drone. Their benchmark mission scenario involved traversing a path with a low ceiling, jumping over a gap, and hopping onto an obstacle. “Assuming an erect position with the tail touching the ground, the robot could walk and remain stable even without advanced controllers,” Shin claims. Walking solved the problem of moving under low ceilings. Jumping over gaps and onto obstacles was done by using the mechanism used for takeoff: torsion springs and actuators. RAVEN could jump over an 11-centimeter-wide gap and onto an obstacle 26-centimeter-high.
But Shin says RAVEN will need way more work before it truly shines. “At this stage, the robot cannot clear all those obstacles in one go. We had to reprogram it for each of the obstacles separately,” Shin says. The problem is the control system in RAVEN is not adaptive; the actuators in the legs perform predefined sets of motions to send the robot on a trajectory the team figured out through computer simulations. If there was something blocking the way, RAVEN would have crashed into it.
Another, perhaps more striking limitation is that RAVEN can’t use its legs to land. But this is something Shin and his colleagues want to work on in the future.
“We want to implement some sensors, perhaps vision or haptic sensors. This way, we’re going to know where the landing site is, how many meters away from it we are, and so on,” Shin says. Another modification that’s on the way for RAVEN is foldable wings that the robot will use to squeeze through tight spaces. “Flapping wings would also be a very interesting topic. They are very important for landing, too, because birds decelerate first with their wings, not with their legs. With flapping wings, this is going to be a really bird-like robot,” Shin claims.
All this is intended to prepare RAVEN for search and rescue missions. The idea is legged flying robots would reach disaster-struck areas quickly, land, traverse difficult terrain on foot if necessary, and then take off like birds. “Another application is delivering parcels. Here in Switzerland, I often see helicopters delivering them to people living high up in the mountains, which I think is quite costly. A bird-like drone could do that more efficiently,” Shin suggested.
John is Ars Technica’s science editor. He has a Bachelor of Arts in Biochemistry from Columbia University, and a Ph.D. in Molecular and Cell Biology from the University of California, Berkeley. When physically separated from his keyboard, he tends to seek out a bicycle, or a scenic location for communing with his hiking boots.
There’s a common popular science demonstration involving “soap boats,” in which liquid soap poured onto the surface of water creates a propulsive flow driven by gradients in surface tension. But it doesn’t last very long since the soapy surfactants rapidly saturate the water surface, eliminating that surface tension. Using ethanol to create similar “cocktail boats” can significantly extend the effect because the alcohol evaporates rather than saturating the water.
That simple classroom demonstration could also be used to propel tiny robotic devices across liquid surfaces to carry out various environmental or industrial tasks, according to a preprint posted to the physics arXiv. The authors also exploited the so-called “Cheerios effect” as a means of self-assembly to create clusters of tiny ethanol-powered robots.
As previously reported, those who love their Cheerios for breakfast are well acquainted with how those last few tasty little “O”s tend to clump together in the bowl: either drifting to the center or to the outer edges. The “Cheerios effect is found throughout nature, such as in grains of pollen (or, alternatively, mosquito eggs or beetles) floating on top of a pond; small coins floating in a bowl of water; or fire ants clumping together to form life-saving rafts during floods. A 2005 paper in the American Journal of Physics outlined the underlying physics, identifying the culprit as a combination of buoyancy, surface tension, and the so-called “meniscus effect.”
It all adds up to a type of capillary action. Basically, the mass of the Cheerios is insufficient to break the milk’s surface tension. But it’s enough to put a tiny dent in the surface of the milk in the bowl, such that if two Cheerios are sufficiently close, the curved surface in the liquid (meniscus) will cause them to naturally drift toward each other. The “dents” merge and the “O”s clump together. Add another Cheerio into the mix, and it, too, will follow the curvature in the milk to drift toward its fellow “O”s.
Physicists made the first direct measurements of the various forces at work in the phenomenon in 2019. And they found one extra factor underlying the Cheerios effect: The disks tilted toward each other as they drifted closer in the water. So the disks pushed harder against the water’s surface, resulting in a pushback from the liquid. That’s what leads to an increase in the attraction between the two disks.
On Wednesday, researchers reported that they had developed a drone they’re calling the CoulombFly, which is capable of self-powered hovering for as long as the Sun is shining. The drone, which is shaped like no aerial vehicle you’ve ever seen before, combines solar cells, a voltage converter, and an electrostatic motor to drive a helicopter-like propeller—with all components having been optimized for a balance of efficiency and light weight.
Before people get excited about buying one, the list of caveats is extensive. There’s no onboard control hardware, and the drone isn’t capable of directed flight anyway, meaning it would drift on the breeze if ever set loose outdoors. Lots of the components appear quite fragile, as well. However, the design can be miniaturized, and the researchers built a version that weighs only 9 milligrams.
Built around a motor
One key to this development was the researchers’ recognition that most drones use electromagnetic motors, which involve lots of metal coils that add significant weight to any system. So, the team behind the work decided to focus on developing a lightweight electrostatic motor. These rely on charge attraction and repulsion to power the motor, as opposed to magnetic interactions.
The motor the researchers developed is quite large relative to the size of the drone. It consists of an inner ring of stationary charged plates called the stator. These plates are composed of a thin carbon-fiber plate covered in aluminum foil. When in operation, neighboring plates have opposite charges. A ring of 64 rotating plates surrounds that.
The motor starts operating when the plates in the outer ring are charged. Since one of the nearby plates on the stator will be guaranteed to have the opposite charge, the pull will start the rotating ring turning. When the plates of the stator and rotor reach their closest approach, thin wires will make contact, allowing charges to transfer between them. This ensures that the stator and rotor plates now have the same charge, converting the attraction to a repulsion. This keeps the rotor moving, and guarantees that the rotor’s plate now has the opposite charge from the next stator plate down the line.
These systems typically require very little in the way of amperage to operate. But they do require a large voltage difference between the plates (something we’ll come back to).
When hooked up to a 10-centimeter, eight-bladed propeller, the system could produce a maximum lift of 5.8 grams. This gave the researchers clear weight targets when designing the remaining components.
Ready to hover
The solar power cells were made of a thin film of gallium arsenide, which is far more expensive than other photovoltaic materials, but offers a higher efficiency (30 percent conversion compared to numbers that are typically in the mid-20s). This tends to provide the opposite of what the system needs: reasonable current at a relatively low voltage. So, the system also needed a high-voltage power converter.
Here, the researchers sacrificed efficiency for low weight, arranging a bunch of voltage converters in series to create a system that weighs just 1.13 grams, but steps the voltage up from 4.5 V all the way to 9.0 kV. But it does so with a power conversion efficiency of just 24 percent.
The resulting CoulombFly is dominated by the large cylindrical motor, which is topped by the propeller. Suspended below that is a platform with the solar cells on one side, balanced out by the long, thin power converter on the other.
Meet the CoulombFly.
To test their system, the researchers simply opened a window on a sunny day in Beijing. Starting at noon, the drone took off and hovered for over an hour, and all indications are that it would have continued to do so for as long as the sunlight provided enough power.
The total system required just over half a watt of power to stay aloft. Given a total mass of 4 grams, that works out to a lift-to-power efficiency of 7.6 grams per watt. But a lot of that power is lost during the voltage conversion. If you focus on the motor alone, it only requires 0.14 watts, giving it a lift-to-power efficiency of over 30 grams per watt.
The researchers provide a long list of things they could do to optimize the design, including increasing the motor’s torque and propeller’s lift, placing the solar cells on structural components, and boosting the efficiency of the voltage converter. But one thing they don’t have to optimize is the vehicle’s size since they already built a miniaturized version that’s only 8 millimeters high and weighs just 9 milligrams but is able to generate a milliwatt of power that turns its propeller at over 15,000 rpm.
Again, all this is done without any onboard control circuitry or the hardware needed to move the machine anywhere—they’re basically flying these in cages to keep them from wandering off on the breeze. But there seems to be enough leeway in the weight that some additional hardware should be possible, especially if they manage some of the potential optimizations they mentioned.
Enlarge/ Right now, the software doesn’t do arms, so don’t go taking on any aliens with it.
20th Century Fox
Exoskeletons today look like something straight out of sci-fi. But the reality is they are nowhere near as robust as their fictional counterparts. They’re quite wobbly, and it takes long hours of handcrafting software policies, which regulate how they work—a process that has to be repeated for each individual user.
To bring the technology a bit closer to Avatar’s Skel Suits or Warhammer 40k power armor, a team at North Carolina University’s Lab of Biomechatronics and Intelligent Robotics used AI to build the first one-size-fits-all exoskeleton that supports walking, running, and stair-climbing. Critically, its software adapts itself to new users with no need for any user-specific adjustments. “You just wear it and it works,” says Hao Su, an associate professor and co-author of the study.
Tailor-made robots
An exoskeleton is a robot you wear to aid your movements—it makes walking, running, and other activities less taxing, the same way an e-bike adds extra watts on top of those you generate yourself, making pedaling easier. “The problem is, exoskeletons have a hard time understanding human intentions, whether you want to run or walk or climb stairs. It’s solved with locomotion recognition: systems that recognize human locomotion intentions,” says Su.
Building those locomotion recognition systems currently relies on elaborate policies that define what actuators in an exoskeleton need to do in each possible scenario. “Let’s take walking. The current state of the art is we put the exoskeleton on you and you walk on a treadmill for an hour. Based on that, we try to adjust its operation to your individual set of movements,” Su explains.
Building handcrafted control policies and doing long human trials for each user makes exoskeletons super expensive, with prices reaching $200,000 or more. So, Su’s team used AI to automatically generate control policies and eliminate human training. “I think within two or three years, exoskeletons priced between $2,000 and $5,000 will be absolutely doable,” Su claims.
His team hopes these savings will come from developing the exoskeleton control policy using a digital model, rather than living, breathing humans.
Digitizing robo-aided humans
Su’s team started by building digital models of a human musculoskeletal system and an exoskeleton robot. Then they used multiple neural networks that operated each component. One was running the digitized model of a human skeleton, moved by simplified muscles. The second neural network was running the exoskeleton model. Finally, the third neural net was responsible for imitating motion—basically predicting how a human model would move wearing the exoskeleton and how the two would interact with each other. “We trained all three neural networks simultaneously to minimize muscle activity,” says Su.
One problem the team faced is that exoskeleton studies typically use a performance metric based on metabolic rate reduction. “Humans, though, are incredibly complex, and it is very hard to build a model with enough fidelity to accurately simulate metabolism,” Su explains. Luckily, according to the team, reducing muscle activations is rather tightly correlated with metabolic rate reduction, so it kept the digital model’s complexity within reasonable limits. The training of the entire human-exoskeleton system with all three neural networks took roughly eight hours on a single RTX 3090 GPU. And the results were record-breaking.
Bridging the sim-to-real gap
After developing the controllers for the digital exoskeleton model, which were developed by the neural networks in simulation, Su’s team simply copy-pasted the control policy to a real controller running a real exoskeleton. Then, they tested how an exoskeleton trained this way would work with 20 different participants. The averaged metabolic rate reduction in walking was over 24 percent, over 13 percent in running, and 15.4 percent in stair climbing—all record numbers, meaning their exoskeleton beat every other exoskeleton ever made in each category.
This was achieved without needing any tweaks to fit it to individual gaits. But the neural networks’ magic didn’t end there.
“The problem with traditional, handcrafted policies was that it was just telling it ‘if walking is detected do one thing; if walking faster is detected do another thing.’ These were [a mix of] finite state machines and switch controllers. We introduced end-to-end continuous control,” says Su. What this continuous control meant was that the exoskeleton could follow the human body as it made smooth transitions between different activities—from walking to running, from running to climbing stairs, etc. There was no abrupt mode switching.
“In terms of software, I think everyone will be using this neural network-based approach soon,” Su claims. To improve the exoskeletons in the future, his team wants to make them quieter, lighter, and more comfortable.
But the plan is also to make them work for people who need them the most. “The limitation now is that we tested these exoskeletons with able-bodied participants, not people with gait impairments. So, what we want to do is something they did in another exoskeleton study at Stanford University. We would take a one-minute video of you walking, and based on that, we would build a model to individualize our general model. This should work well for people with impairments like knee arthritis,” Su claims.
Enlarge/ A kitty named Clover prepares to play with a robot arm in the Cat Royale “multi-species” science/art installation .
Blast Theory – Stephen Daly
Cats and robots are a winning combination, as evidenced by all those videos of kitties riding on Roombas. And now we have Cat Royale, a “multispecies” live installation in which three cats regularly “played” with a robot over 12 days, carefully monitored by human operators. Created by computer scientists from the University of Nottingham in collaboration with artists from a group called Blast Theory, the installation debuted at the World Science Festival in Brisbane, Australia, last year and is now a touring exhibit. The accompanying YouTube video series recently won a Webby Award, and a paper outlining the insights gleaned from the experience was similarly voted best paper at the recent Computer-Human Conference (CHI’24).
“At first glance, the project is about designing a robot to enrich the lives of a family of cats by playing with them,” said co-author Steve Benford of the University of Nottingham, who led the research, “Under the surface, however, it explores the question of what it takes to trust a robot to look after our loved ones and potentially ourselves.” While cats might love Roombas, not all animal encounters with robots are positive: Guide dogs for the visually impaired can get confused by delivery robots, for example, while the rise of lawn mowing robots can have a negative impact on hedgehogs, per Benford et al.
Blast Theory and the scientists first held a series of exploratory workshops to ensure the installation and robotic design would take into account the welfare of the cats. “Creating a multispecies system—where cats, robots, and humans are all accounted for—takes more than just designing the robot,” said co-author Eike Schneiders of Nottingham’s Mixed Reality Lab about the primary takeaway from the project. “We had to ensure animal well-being at all times, while simultaneously ensuring that the interactive installation engaged the (human) audiences around the world. This involved consideration of many elements, including the design of the enclosure, the robot, and its underlying systems, the various roles of the humans-in-the-loop, and, of course, the selection of the cats.”
Based on those discussions, the team set about building the installation: a bespoke enclosure that would be inhabited by three cats for six hours a day over 12 days. The lucky cats were named Ghostbuster, Clover, and Pumpkin—a parent and two offspring to ensure the cats were familiar with each other and comfortable sharing the enclosure. The enclosure was tricked out to essentially be a “utopia for cats,” per the authors, with perches, walkways, dens, a scratching post, a water fountain, several feeding stations, a ball run, and litter boxes tucked away in secluded corners.
Enlarge/ (l-r) Clover, Pumpkin, and Ghostbuster spent six hours a day for 12 days in the installation.
E. Schneiders et al., 2024
As for the robot, the team chose the Kino Gen3 lite robot arm, and the associated software was trained on over 7,000 videos of cats. A decision engine gave the robot autonomy and proposed activities for specific cats. Then a human operator used an interface control system to instruct the robot to execute the movements. The robotic arm’s two-finger gripper was augmented with custom 3D-printed attachments so that the robot could manipulate various cat toys and accessories.
Each cat/robot interaction was evaluated for a “happiness score” based on the cat’s level of engagement, body language, and so forth. Eight cameras monitored the cat and robot activities, and that footage was subsequently remixed and edited into daily YouTube highlight videos and, eventually, an eight-hour film.
Reinforcement-learning algorithms in systems like ChatGPT or Google’s Gemini can work wonders, but they usually need hundreds of thousands of shots at a task before they get good at it. That’s why it’s always been hard to transfer this performance to robots. You can’t let a self-driving car crash 3,000 times just so it can learn crashing is bad.
But now a team of researchers at Northwestern University may have found a way around it. “That is what we think is going to be transformative in the development of the embodied AI in the real world,” says Thomas Berrueta who led the development of the Maximum Diffusion Reinforcement Learning (MaxDiff RL), an algorithm tailored specifically for robots.
Introducing chaos
The problem with deploying most reinforcement-learning algorithms in robots starts with the built-in assumption that the data they learn from is independent and identically distributed. The independence, in this context, means the value of one variable does not depend on the value of another variable in the dataset—when you flip a coin two times, getting tails on the second attempt does not depend on the result of your first flip. Identical distribution means that the probability of seeing any specific outcome is the same. In the coin-flipping example, the probability of getting heads is the same as getting tails: 50 percent for each.
In virtual, disembodied systems, like YouTube recommendation algorithms, getting such data is easy because most of the time it meets these requirements right off the bat. “You have a bunch of users of a website, and you get data from one of them, and then you get data from another one. Most likely, those two users are not in the same household, they are not highly related to each other. They could be, but it is very unlikely,” says Todd Murphey, a professor of mechanical engineering at Northwestern.
The problem is that, if those two users were related to each other and were in the same household, it could be that the only reason one of them watched a video was that their housemate watched it and told them to watch it. This would violate the independence requirement and compromise the learning.
“In a robot, getting this independent, identically distributed data is not possible in general. You exist at a specific point in space and time when you are embodied, so your experiences have to be correlated in some way,” says Berrueta. To solve this, his team designed an algorithm that pushes robots be as randomly adventurous as possible to get the widest set of experiences to learn from.
Two flavors of entropy
The idea itself is not new. Nearly two decades ago, people in AI figured out algorithms, like Maximum Entropy Reinforcement Learning (MaxEnt RL), that worked by randomizing actions during training. “The hope was that when you take as diverse set of actions as possible, you will explore more varied sets of possible futures. The problem is that those actions do not exist in a vacuum,” Berrueta claims. Every action a robot takes has some kind of impact on its environment and on its own condition—disregarding those impacts completely often leads to trouble. To put it simply, an autonomous car that was teaching itself how to drive using this approach could elegantly park into your driveway but would be just as likely to hit a wall at full speed.
To solve this, Berrueta’s team moved away from maximizing the diversity of actions and went for maximizing the diversity of state changes. Robots powered by MaxDiff RL did not flail their robotic joints at random to see what that would do. Instead, they conceptualized goals like “can I reach this spot ahead of me” and then tried to figure out which actions would take them there safely.
Berrueta and his colleagues achieved that through something called ergodicity, a mathematical concept that says that a point in a moving system will eventually visit all parts of the space that the system moves in. Basically, MaxDiff RL encouraged the robots to achieve every available state in their environment. And the results of first tests in simulated environments were quite surprising.
Racing pool noodles
“In reinforcement learning there are standard benchmarks that people run their algorithms on so we can have a good way of comparing different algorithms on a standard framework,” says Allison Pinosky, a researcher at Northwestern and co-author of the MaxDiff RL study. One of those benchmarks is a simulated swimmer: a three-link body resting on the ground in a viscous environment that needs to learn to swim as fast as possible in a certain direction.
In the swimmer test, MaxDiff RL outperformed two other state-of-the-art reinforcement learning algorithms (NN-MPPI and SAC). These two needed several resets to figure out how to move the swimmers. To complete the task, they were following a standard AI learning process divided down into a training phase where an algorithm goes through multiple failed attempts to slowly improve its performance, and a testing phase where it tries to perform the learned task. MaxDiff RL, by contrast, nailed it, immediately adapting its learned behaviors to the new task.
The earlier algorithms ended up failing to learn because they got stuck trying the same options and never progressing to where they could learn that alternatives work. “They experienced the same data repeatedly because they were locally doing certain actions, and they assumed that was all they could do and stopped learning,” Pinosky explains. MaxDiff RL, on the other hand, continued changing states, exploring, getting richer data to learn from, and finally succeeded. And because, by design, it seeks to achieve every possible state, it can potentially complete all possible tasks within an environment.
But does this mean we can take MaxDiff RL, upload it to a self-driving car, and let it out on the road to figure everything out on its own? Not really.
3D model of a 100-year-old shipwreck off the western coast of Australia. Credit: Daniel Adams, Curtin University HIVE.
A small underwater drone called Hydrus has located the wreckage of a 100-year-old coal hulk in the deep waters off the coast of western Australia. Based on the data the drone captured, scientists were able to use photogrammetry to virtually “rebuild” the 210-foot ship into a 3D model (above). You can explore an interactive 3D rendering of the wreckage here.
The use of robotic submersibles to locate and explore historic shipwrecks is well established. For instance, researchers relied on remotely operated vehicles (ROVs) to study the wreckage of the HMS Terror, Captain Sir John S. Franklin‘s doomed Arctic expedition to cross the Northwest Passage in 1846. In 2007, a pair of brothers (printers based in Norfolk) discovered the wreck of the Gloucester, which ran aground on a sandbank off the coast of Norfolk in 1682 and sank within the hour. Among the passengers was James Stuart, Duke of York and future King James II of England, who escaped in a small boat just before the ship sank.
In 2022, the Falklands Maritime Heritage Trust and National Geographicannounced the discovery of British explorer Sir Ernest Shackleton‘s ship Endurance. In 1915, Shackleton and his crew were stranded for months on the Antarctic ice after the ship was crushed by pack ice and sank into the freezing depths of the Weddell Sea. The wreckage was found nearly 107 years later, 3,008 meters down, roughly four miles (6.4 km) south of the ship’s last recorded position. The wreck was in pristine condition partly because of the lack of wood-eating microbes in those waters. In fact, the lettering “ENDURANCE” was clearly visible in shots of the stern.
And just last year, an ROV was used to verify the discovery of the wreckage of a schooner barge called Ironton, which collided with a Great Lakes freighter called Ohio in Lake Huron’s infamous “Shipwreck Alley” in 1894. The wreck was so well-preserved in the frigid waters of the Great Lakes that its three masts were still standing and its rigging still attached. That discovery could help resolve unanswered questions about the ship’s final hours.
Enlarge/ Deployment of one of Advanced Navigation’s Micro Autonomous Underwater Vehicles (AUV).
Advanced Navigation
According to Advanced Navigation, there are some 3 million undiscovered shipwrecks around the world—1,819 recorded wrecks lying off the coast of Western Australia alone. That includes the Rottnest ship graveyard just southwest of Rottnest Island, with a seabed some 50 to 200 meters below sea level (164 to 656 feet). The island is known for the number of ships wrecked near its shore since the 17th century. The Rottnest graveyard is more of a dump site for scuttling obsolete ships, at least 47 of which would be considered historically significant.
However, this kind of deep ocean exploration can be both time-consuming and expensive, particularly at depths of more than 50 meters (164 feet). Hydrus was designed to reduce the cost of this kind of ocean exploration significantly. One person can deploy the drone because of its compact size, so there is no need for large vessels or complicated launch systems. And Hydrus can capture georeferenced 4K video and still images at the same time. Once this latest expedition realized they had found a shipwreck, they were able to deploy a pair of the drones to take a complete survey in just five hours.
Hydrus captured this footage of the 210-foot wreck of a 19th-century coal hulk. Credit: Advanced Navigation
Ross Anderson, curator of the Western Australia Museum, was able to identify the wreck as an iron coal hulk once used in Freemantle Port to service steamships, probably built in the 1860s–1890s and scuttled in the graveyard sometime in the 1920s. The geolocation data provided to scientists at Curtin University HIVE enabled them to use photogrammetry to convert that data into a 3D digital model. “It can’t be overstated how much this structure in data assists with constraining feature matching and reducing the processing time, especially in large datasets,” Andrew Woods, a professor at the university, said in a statement.
The expedition team’s next target using the Hydrus technology is the wreck of the luxury passenger steamship SS Koombana, which disappeared somewhere off Port Hedland en route to Broome during a tropical cyclone in 1912, with 150 on board presumed to have perished. The only wreckage recovered at the time was part of a starboard bow planking, a stateroom door, a panel from the promenade deck, and a few air tanks. There were a couple of reports in the 1980s of “magnetic anomalies” in the seabed off Bedout Island, part of the route the Koombana would have taken. But despite several deep-water expeditions in the early 2010s, to date the actual shipwreck has not been found.
ANYmal can do parkour and walk across rubble. The quadrupedal robot went back to school and has learned a lot.
Meet ANYmal, a four-legged dog-like robot designed by researchers at ETH Zürich in Switzerland, in hopes of using such robots for search-and-rescue on building sites or disaster areas, among other applications. Now ANYmal has been upgraded to perform rudimentary parkour moves, aka “free running.” Human parkour enthusiasts are known for their remarkably agile, acrobatic feats, and while ANYmal can’t match those, the robot successfully jumped across gaps, climbed up and down large obstacles, and crouched low to maneuver under an obstacle, according to a recent paper published in the journal Science Robotics.
The ETH Zürich team introduced ANYmal’s original approach to reinforcement learning back in 2019 and enhanced its proprioception (the ability to sense movement, action, and location) three years later. Just last year, the team showcased a trio of customized ANYmal robots, tested in environments as close to the harsh lunar and Martian terrain as possible. As previously reported, robots capable of walking could assist future rovers and mitigate the risk of damage from sharp edges or loss of traction in loose regolith. Every robot had a lidar sensor. but they were each specialized for particular functions and still flexible enough to cover for each other—if one glitches, the others can take over its tasks.
For instance, the Scout model’s main objective was to survey its surroundings using RGB cameras. This robot also used another imager to map regions and objects of interest using filters that let through different areas of the light spectrum. The Scientist model had the advantage of an arm featuring a MIRA (Metrohm Instant Raman Analyzer) and a MICRO (microscopic imager). The MIRA was able to identify chemicals in materials found on the surface of the demonstration area based on how they scattered light, while the MICRO on its wrist imaged them up close. The Hybrid was more of a generalist, helping out the Scout and the Scientist with measurements of scientific targets such as boulders and craters.
As advanced as ANYmal and similar-legged robots have become in recent years, significant challenges still remain before they are as nimble and agile as humans and other animals. “Before the project started, several of my researcher colleagues thought that legged robots had already reached the limits of their development potential,” said co-author Nikita Rudin, a graduate student at ETH Zurich who also does parkour. “But I had a different opinion. In fact, I was sure that a lot more could be done with the mechanics of legged robots.”
Enlarge/ The quadrupedal robot ANYmal practices parkour in a hall at ETH Zürich.
ETH Zurich / Nikita Rudin
Parkour is quite complex from a robotics standpoint, making it an ideal aspirational task for the Swiss team’s next step in ANYmal’s capabilities. Parkour can involve large obstacles, requiring the robot “to perform dynamic maneuvers at the limits of actuation while accurately controlling the motion of the base and limbs,” the authors wrote. To succeed, ANYmal must be able to sense its environment and adapt to rapid changes, selecting a feasible path and sequence of motions from its programmed skill set. And it has to do all that in real time with limited onboard computing.
The Swiss team’s overall approach combines machine learning with model-based control. They split the task into three interconnected components: a perception module that processes the data from onboard cameras and LiDAR to estimate the terrain; a locomotion module with a programmed catalog of movements to overcome specific terrains; and a navigation module that guides the locomotion module in selecting which skills to use to navigate different obstacles and terrain using intermediate commands.
Rudin, for example, used machine learning to teach ANYmal some new skills through trial and error, namely, scaling obstacles and figuring out how to climb up and jump back down from them. The robot’s camera and artificial neural network enable it to pick the best maneuvers based on its prior training. Another graduate student, Fabian Jenelten, used model-based control to teach ANYmal how to recognize and negotiate gaps in piles of rubble, augmented with machine learning so the robot could have more flexibility in applying known movement patterns to unexpected situations.
Enlarge/ ANYmal on a civil defense training ground.
ETH Zurich / Fabian Jenelten
Among the tasks ANYmal was able to perform was jumping from one box to a neighboring box up to 1 meter away. This required the robot to approach the gap sideways, place its feet as close as possible to the edge, and then use three legs to jump while extending the fourth to land on the other box. It could then transfer two diagonal legs before bringing the final leg across the gap. This meant ANYmal could recover from any missteps and slippage by transferring its weight between the non-leaping legs.
ANYmal also was able to climb down from a 1-meter-high box to reach a target on the ground, as well as climbing up the box. It can also crouch down to reach a target on the other side of a narrow passage, lowering its base and adapting its gait accordingly. The team also tested ANYmal’s walking abilities, in which the robot successfully traversed stairs, slopes, random small obstacles and so forth.
ANYmal still has its limitations when it comes to navigating real-world environments, whether it be a parkour course or the debris of a collapsed building. For instance, the authors note that they have yet to test the scalability of their approach to more diverse and unstructured scenarios that incorporate a wider variety of obstacles; the robot was only tested in a few select scenarios. “It remains to be seen how well these different modules can generalize to completely new scenarios,” they wrote. The approach is also time-consuming since it requires eight neural networks that must be tuned separately, and some of the networks are interdependent, so changing one means changing and retraining the others as well.
Still, ANYmal “can now evolve in complex scenes where it must climb and jump on large obstacles while selecting a nontrivial path toward its target location,” the authors wrote. Thus, “by aiming to match the agility of free runners, we can better understand the limitations of each component in the pipeline from perception to actuation, circumvent those limits, and generally increase the capabilities of our robots.”
Sending 1 kilogram to Mars will set you back roughly $2.4 million, judging by the cost of the Perseverance mission. If you want to pack up supplies and gear for every conceivable contingency, you’re going to need a lot of those kilograms.
But what if you skipped almost all that weight and only took a do-it-all Swiss Army knife instead? That’s exactly what scientists at NASA Ames Research Center and Stanford University are testing with robots, algorithms, and highly advanced building materials.
Zero mass exploration
“The concept of zero mass exploration is rooted in self-replicating machines, an engineering concept John von Neumann conceived in the 1940s”, says Kenneth C. Cheung, a NASA Ames researcher. He was involved in the new study published recently in Science Robotics covering self-reprogrammable metamaterials—materials that do not exist in nature and have the ability to change their configuration on their own. “It’s the idea that an engineering system can not only replicate, but sustain itself in the environment,” he adds.
Based on this concept, Robert A. Freitas Jr. in the 1980s proposed a self-replicating interstellar spacecraft called the Von Neumann probe that would visit a nearby star system, find resources to build a copy of itself, and send this copy to another star system. Rinse and repeat.
“The technology of reprogrammable metamaterials [has] advanced to the point where we can start thinking about things like that. It can’t make everything we need yet, but it can make a really big chunk of what we need,” says Christine E. Gregg, a NASA Ames researcher and the lead author of the study.
Building blocks for space
One of the key problems with Von Neumann probes was that taking elements found in the soil on alien worlds and processing them into actual engineering components was resource-intensive and required huge amounts of energy. The NASA Ames team solved that with using prefabricated “voxels”—standardized reconfigurable building blocks.
The system derives its operating principles from the way nature works on a very fundamental level. “Think how biology, one of the most scalable systems we have ever seen, builds stuff,” says Gregg. “It does that with building blocks. There are on the order of 20 amino acids which your body uses to make proteins to make 200 different types of cells and then combines trillions of those cells to make organs as complex as my hair and my eyes. We are using the same strategy,” she adds.
To demo this technology, they built a set of 256 of those blocks—extremely strong 3D structures made with a carbon-fiber-reinforced polymer called StattechNN-40CF. Each block had fastening interfaces on every side that could be used to reversibly attach them to other blocks and form a strong truss structure.
A 3×3 truss structure made with these voxels had an average failure load of 900 Newtons, which means it could hold over 90 kilograms despite being incredibly light itself (its density is just 0.0103 grams per cubic centimeter). “We took these voxels out in backpacks and built a boat, a shelter, a bridge you could walk on. The backpacks weighed around 18 kilograms. Without technology like that, you wouldn’t even think about fitting a boat and a bridge in a backpack,” says Cheung. “But the big thing about this study is that we implemented this reconfigurable system autonomously with robots,” he adds.
Enlarge/ Grasshoppers, beware! Robopteryx is here to flush you from your hiding place.
Jinseok Park, Piotr Jablonski et al., 2024
Scientists in South Korea built a robotic dinosaur and used it to startle grasshoppers to learn more about why dinosaurs evolved feathers, according to a recent paper published in the journal Scientific Reports. The results suggest that certain dinosaurs may have employed a hunting strategy in which they flapped their proto-wings to flush out prey, and this behavior may have led to the evolution of larger and stiffer feathers.
As reported previously, feathers are the defining feature of birds, but that wasn’t always the case. For millions of years, various species of dinosaurs sported feathers, some of which have left behind fossilized impressions. For the most part, the feathers we’ve found have been attached to smaller dinosaurs, many of them along the lineage that gave rise to birds—although in 2012, scientists discovered three nearly complete skeletons of a “gigantic” feathered dinosaur species, Yutyrannus huali, related to the ancestors of Tyrannosaurus Rex.
Various types of dino-feathers have been found in the fossil record over the last 30 years, such as so-called pennaceous feathers (present in most modern birds). These were found on distal forelimbs of certain species like Caudipteryx, serving as proto-wings that were too small to use for flight, as well as around the tip of the tail as plumage. Paleontologists remain unsure of the function of pennaceous feathers—what use could there be for half a wing? A broad range of hypotheses have been proposed: foraging or hunting, pouncing or immobilizing prey, brooding, gliding, or wing-assisted incline running, among others.
Enlarge/ Mounted Caudipteryx zoui skeleton at the Löwentor Museum in Stuttgart, Germany.
Co-author Jinseok Park of Seoul National University in South Korea and colleagues thought the pennaceous feathers might have been used to flush out potential prey from hiding places so they could be more easily caught. It’s a strategy employed by certain modern bird species, like roadrunners, and typically involves a visual display of the plumage on wings and tails.
There is evidence that this flush-pursuit hunting strategy evolved multiple times. According to Park et al., it’s based on the “rare enemy effect,” i.e., certain prey (like insects) wouldn’t be capable of responding to different predators in different ways and would not respond effectively to an unusual flush-pursuit strategy. Rather than escaping a predator, the insects fly toward their own demise. “The use of plumage to flush prey could have increased the frequency of chase after escaping prey, thus amplifying the importance of plumage in drag-based or lift-based maneuvering for a successful pursuit,” the authors wrote. “This, in turn, could have led to the larger and stiffer feathers for faster movements and more visual flush displays.”
To test their hypothesis, Park et al. constructed a robot dinosaur they dubbed “Robopteryx,” using Caudipteryx as a model. They built the robot’s body out of aluminum, with the proto-wings and tail plumage made from black paper and plastic ribbing. The head was made of black polystyrene, the wing folds were made of black elastic stocking, and the whole contraption was covered in felt. They scanned the scientific literature on Caudipteryx to determine resting posture angles and motion ranges. The motion of the forelimbs and tail was controlled by a mechanism controlled by custom software running on a mobile phone.
Enlarge/ Robopteryx faces off against a grasshopper and prepares to flap its wings.
Jinseok Park, Piotr Jablonski et al., 2024
Park et al. then conducted experiments with the robot performing motions consistent with a flush display using the band-winged grasshopper (a likely prey), which has relatively simple neural circuits. They placed a wooden stick with scale marks next to the grasshopper and photographed it to record its body orientation relative to the robot, and then made the robot’s forelimbs and tail flap to mimic a flush display. If the grasshopper escaped, they ended the individual test; if the grasshopper didn’t respond, they slowly moved the robot closer and closer using a long beam. The team also attached electrodes to grasshoppers in the lab to measure neural spikes as the insects were shown projected Cauderyx animations of a flush display on a flat-screen monitor.
The results: around half the grasshoppers fled in response to Robopteryx without feathers, compared to over 90 percent when feathered wings flapped. They also measured stronger neural signals when feathers were present. For Park et al., this is solid evidence in support of their hypothesis that a flush-pursuit hunting strategy may have been a factor in the evolution of pennaceous feathers. “Our results emphasize the significance of considering sensory aspects of predator-prey interactions in the studies of major evolutionary innovations among predatory species,” the authors wrote.
Not everyone is convinced by these results. “It seems to me to be very unlikely that a structure as complex as a pennaceous feather would evolve for such a specific behavioral role,” Steven Salisbury of the University of Queensland in Australia, who was not involved with the research, told New Scientist. “I am sure there are lots of ways to scare grasshoppers other than to flap some feathers at it. You can have feathers to scare grasshoppers and you can have them to insulate and incubate eggs. They’re good for display, the stabilization of body position when running, and, of course, for gliding and powered flight. Feathers help for all sorts of things.”
VR robots are slowly moving into the mainstream with applications that go beyond the usual manufacturing processes. Robots have been in use for years in industrial settings where they perform automated repetitive tasks. But their practical use has been quite limited. Today, however, we see some of them in the consumer sector delivering robotic solutions that require customization.
Augmented by other technologies such as AR, VR, and AI, robots show improved efficiency and safety in accomplishing more complex processes. With VR, humans can supervise the robots remotely to enhance their performance. VR technology provides human operators with a more immersive environment. This enables them to interact with robots better and view the actual surroundings of the robots in real time. Consequently, this opens vast opportunities for practical uses that enhance our lives.
Real-Life Use Cases of VR Robots
1. TX SCARA: Automated Restocking of Refrigerated Shelves
Developed by Telexistence, TX SCARA is powered by three main technologies—robotics, artificial intelligence, and virtual reality. This robot specializes in restocking refrigerated shelves in stores. It relies on GORDON, its AI system, to know when and where to place products. When issues arise due to external factors or system miscalculation, Telexistence employees use VR headsets to control the robot remotely and address the problem.
TX SCARA is present in 300 FamilyMart stores in Japan. Plans to expand their use in convenience stores in the United States are already underway. With TX SCARA capable of working 24/7 with a pace of up to 1,000 bottles or cans per day, it can replace up to three hours of human work each day for a single store alone.
2. Reachy: A Robot That Shows Emotions
Reachy gives VR robots a human side. An expressive humanoid platform, Reachy mimics human expressions and body language. It conveys human emotions through its antennas and motions.
Reachy
Users operate Reachy remotely using VR equipment that shows the environment surrounding the robot. They can move Reachy’s head, arms, and hands to manipulate objects and interact with people around the robot. They can also control Reachy’s mobile base to move around and explore its environment.
Since it can be programmed with Python and ROS to perform almost any task, its use cases are virtually limitless. It has applications across various sectors, such as research (to explore new frontiers in robotics), healthcare (to replace mechanical tasks), retail (to enhance customer experiences), education (to make learning more immersive), and many others. Reachy is also fully customizable, with many different configurations, modules, and hardware options available.
3. Robotic VR: Haptic Technology for Medical Care
A team of researchers co-led by the City University of Hong Kong has developed an advanced robotic VR system that has great potential for use in healthcare. Robotic VR, an innovative human-machine interface (HMI), can be used to perform medical procedures. This includes conducting swab tests and caring for patients with infectious diseases.
Doctors, nurses, and other health practitioners control the VR robot using a VR headset and flexible electronic skin that enables them to experience tactile sensations while interacting remotely with patients. This allows them to control and adjust the robot’s motion and strength as they collect bio-samples or provide nursing care. Robotic VR can help minimize the risk of infection and prevent contagion.
4. Skippy: Your Neighborhood Delivery Robot
Skippy elevates deliveries to a whole new level. Human operators, called Skipsters, control these VR robots remotely. They use VR headsets to supervise the robots as they move about the neighborhood. When you order food or groceries from a partner establishment, Skippy picks it up and delivers it to your doorstep. Powered by AI and controlled by Skipsters, the cute robot rolls through pedestrian paths while avoiding foot traffic and obstacles.
Virtual reality is an enabling technology in robotics. By merging these two technologies, we’re bound to see more practical uses of VR-enabled robots in the consumer market. As the technologies become more advanced and the hardware required becomes more affordable, we can expect to see more VR robots that we can interact with as we go through our daily lives.
Developments in VR interface and robotics technology will eventually pave the way for advancements in the usability of VR robots in real-world applications.
