While the Protoclone is a twitching, dangling robotic prototype right now, there’s a lot of tech packed into its body. Protoclone’s sensory system includes four depth cameras in its skull for vision, 70 inertial sensors to track joint positions, and 320 pressure sensors that provide force feedback. This system lets the robot react to visual input and learn by watching humans perform tasks.
As you can probably tell by the video, the current Protoclone prototype is still in an early developmental stage, requiring ceiling suspension for stability. Clone Robotics previously demonstrated components of this technology in 2022 with the release of its robotic hand, which used the same Myofiber muscle system.
Artificial Muscles Robotic Arm Full Range of Motion + Static Strength Test (V11).
A few months ago, Clone Robotics also showed off a robotic torso powered by the same technology.
Torso 2 by Clone with Actuated Abdomen.
Other companies’ robots typically use other types of actuators, such as solenoids and electric motors. Clone’s pressure-based muscle system is an interesting approach, though getting Protoclone to stand and balance without the need for suspension or umbilicals may still prove a challenge.
Clone Robotics plans to start its production with 279 units called Clone Alpha, with plans to open preorders later in 2025. The company has not announced pricing for these initial units, but given the engineering challenges still ahead, a functional release any time soon seems optimistic.
There’s a difference between knowing a word and knowing a concept.
Large language models like ChatGPT display conversational skills, but the problem is they don’t really understand the words they use. They are primarily systems that interact with data obtained from the real world but not the real world itself. Humans, on the other hand, associate language with experiences. We know what the word “hot” means because we’ve been burned at some point in our lives.
Is it possible to get an AI to achieve a human-like understanding of language? A team of researchers at the Okinawa Institute of Science and Technology built a brain-inspired AI model comprising multiple neural networks. The AI was very limited—it could learn a total of just five nouns and eight verbs. But their AI seems to have learned more than just those words; it learned the concepts behind them.
Babysitting robotic arms
“The inspiration for our model came from developmental psychology. We tried to emulate how infants learn and develop language,” says Prasanna Vijayaraghavan, a researcher at the Okinawa Institute of Science and Technology and the lead author of the study.
While the idea of teaching AIs the same way we teach little babies is not new—we applied it to standard neural nets that associated words with visuals. Researchers also tried teaching an AI using a video feed from a GoPro strapped to a human baby. The problem is babies do way more than just associate items with words when they learn. They touch everything—grasp things, manipulate them, throw stuff around, and this way, they learn to think and plan their actions in language. An abstract AI model couldn’t do any of that, so Vijayaraghavan’s team gave one an embodied experience—their AI was trained in an actual robot that could interact with the world.
Vijayaraghavan’s robot was a fairly simple system with an arm and a gripper that could pick objects up and move them around. Vision was provided by a simple RGB camera feeding videos in a somewhat crude 64×64 pixels resolution.
The robot and the camera were placed in a workspace, put in front of a white table with blocks painted green, yellow, red, purple, and blue. The robot’s task was to manipulate those blocks in response to simple prompts like “move red left,” “move blue right,” or “put red on blue.” All that didn’t seem particularly challenging. What was challenging, though, was building an AI that could process all those words and movements in a manner similar to humans. “I don’t want to say we tried to make the system biologically plausible,” Vijayaraghavan told Ars. “Let’s say we tried to draw inspiration from the human brain.”
Chasing free energy
The starting point for Vijayaraghavan’s team was the free energy principle, a hypothesis that the brain constantly makes predictions about the world based on internal models, then updates these predictions based on sensory input. The idea is that we first think of an action plan to achieve a desired goal, and then this plan is updated in real time based on what we experience during execution. This goal-directed planning scheme, if the hypothesis is correct, governs everything we do, from picking up a cup of coffee to landing a dream job.
All that is closely intertwined with language. Neuroscientists at the University of Parma found that motor areas in the brain got activated when the participants in their study listened to action-related sentences. To emulate that in a robot, Vijayaraghavan used four neural networks working in a closely interconnected system. The first was responsible for processing visual data coming from the camera. It was tightly integrated with a second neural net that handled proprioception: all the processes that ensured the robot was aware of its position and the movement of its body. This second neural net also built internal models of actions necessary to manipulate blocks on the table. Those two neural nets were additionally hooked up to visual memory and attention modules that enabled them to reliably focus on the chosen object and separate it from the image’s background.
The third neural net was relatively simple and processed language using vectorized representations of those “move red right” sentences. Finally, the fourth neural net worked as an associative layer and predicted the output of the previous three at every time step. “When we do an action, we don’t always have to verbalize it, but we have this verbalization in our minds at some point,” Vijayaraghavan says. The AI he and his team built was meant to do just that: seamlessly connect language, proprioception, action planning, and vision.
When the robotic brain was up and running, they started teaching it some of the possible combinations of commands and sequences of movements. But they didn’t teach it all of them.
The birth of compositionality
In 2016, Brenden Lake, a professor of psychology and data science, published a paper in which his team named a set of competencies machines need to master to truly learn and think like humans. One of them was compositionality: the ability to compose or decompose a whole into parts that can be reused. This reuse lets them generalize acquired knowledge to new tasks and situations. “The compositionality phase is when children learn to combine words to explain things. They [initially] learn the names of objects, the names of actions, but those are just single words. When they learn this compositionality concept, their ability to communicate kind of explodes,” Vijayaraghavan explains.
The AI his team built was made for this exact purpose: to see if it would develop compositionality. And it did.
Once the robot learned how certain commands and actions were connected, it also learned to generalize that knowledge to execute commands it never heard before. recognizing the names of actions it had not performed and then performing them on combinations of blocks it had never seen. Vijayaraghavan’s AI figured out the concept of moving something to the right or the left or putting an item on top of something. It could also combine words to name previously unseen actions, like putting a blue block on a red one.
While teaching robots to extract concepts from language has been done before, those efforts were focused on making them understand how words were used to describe visuals. Vijayaragha built on that to include proprioception and action planning, basically adding a layer that integrated sense and movement to the way his robot made sense of the world.
But some issues are yet to overcome. The AI had very limited workspace. The were only a few objects and all had a single, cubical shape. The vocabulary included only names of colors and actions, so no modifiers, adjectives, or adverbs. Finally, the robot had to learn around 80 percent of all possible combinations of nouns and verbs before it could generalize well to the remaining 20 percent. Its performance was worse when those ratios dropped to 60/40 and 40/60.
But it’s possible that just a bit more computing power could fix this. “What we had for this study was a single RTX 3090 GPU, so with the latest generation GPU, we could solve a lot of those issues,” Vijayaraghavan argued. That’s because the team hopes that adding more words and more actions won’t result in a dramatic need for computing power. “We want to scale the system up. We have a humanoid robot with cameras in its head and two hands that can do way more than a single robotic arm. So that’s the next step: using it in the real world with real world robots,” Vijayaraghavan said.
Jacek Krywko is a freelance science and technology writer who covers space exploration, artificial intelligence research, computer science, and all sorts of engineering wizardry.
This isn’t the first time scientists have looked to the mantis shrimp as an inspiration for robotics. In 2021, we reported on a Harvard researcher who developed a biomechanical model for the mantis shrimp’s mighty appendage and built a tiny robot to mimic that movement. What’s unusual in the mantis shrimp is that there is a one-millisecond delay between when the unlatching and the snapping action occurs.
The Harvard team identified four distinct striking phases and confirmed it’s the geometry of the mechanism that produces the rapid acceleration after the initial unlatching by the sclerites. The short delay may help reduce wear and tear of the latching mechanisms over repeated use.
New types of motion
The operating principle of the Hyperelastic Torque Reversal Mechanism (HeTRM) involves compressing an elastomeric joint until it reaches a critical point, where stored energy is instantaneously released. Credit: Science Robotics, 2025
Co-author Kyu-Jin Cho of Seoul National University became interested in soft robotics as a graduate student, when he participated in the RoboSoft Grand Challenge. Part of his research involved testing the strength of so-called “soft robotic manipulators,” a type often used in assembly lines for welding or painting, for example. He noticed some unintended deformations in the shape under applied force and realized that the underlying mechanism was similar to how the mantis shrimp punches or how fleas manage to jump so high and far relative to their size.
In fact, Cho’s team previously built a flea-inspired catapult mechanism for miniature jumping robots, using the Hyperelastic Torque Reversal Mechanism (HeTRM) his lab developed. Exploiting torque reversal usually involves incorporating complicated mechanical components. However, “I realized that applying [these] principles to soft robotics could enable the creation of new types of motion without complex mechanisms,” Cho said.
Now he’s built on that work to incorporate the HeTRM into a soft robotic arm that relies upon material properties rather than structural design. It’s basically a soft beam with alternating hyperelastic and rigid segments.
“Our robot is made of soft, stretchy materials, kind of like rubber,” said Cho. “Inside, it has a special part that stores energy and releases it all at once—BAM!—to make the robot move super fast. It works a bit like how a bent tree branch snaps back quickly or how a flea jumps really far. This robot can grab things like a hand, crawl across the floor, or even jump high, and it all happens just by pulling on a simple muscle.”
Fast and complex multi-finger movements generated by the hand exoskeleton. Credit: Shinichi Furuya
When it comes to fine-tuned motor skills like playing the piano, practice, they say, makes perfect. But expert musicians often experience a “ceiling effect,” in which their skill level plateaus after extensive training. Passive training using a robotic exoskeleton hand could help pianists overcome that ceiling effect, according to a paper published in the journal Science Robotics.
“I’m a pianist, but I [injured] my hand because of overpracticing,” coauthor Shinichi Furuya of Kabushiki Keisha Sony Computer Science Kenkyujo told New Scientist. “I was suffering from this dilemma, between overpracticing and the prevention of the injury, so then I thought, I have to think about some way to improve my skills without practicing.” Recalling that his former teachers used to place their hands over his to show him how to play more advanced pieces, he wondered if he could achieve the same effect with a robotic hand.
So Furuya et al. used a custom-made exoskeleton robot hand capable of moving individual fingers on the right hand independently, flexing and extending the joints as needed. Per the authors, prior studies with robotic exoskeletons focused on simpler movements, such as assisting in the movement of limbs stabilizing body posture, or helping grasp objects. That sets the custom robotic hand used in these latest experiments apart from those used for haptics in virtual environments.
A helping robot hand
A total of 118 pianists participated in three different experiments. In the first, 30 pianists performed a designated “chord trill” motor task with the piano at home every day for two weeks: first simultaneously striking D and F keys with the right index and ring fingers, then striking the E and G keys with the right middle and little fingers. “We used this task because it has been widely recognized as technically challenging to play quickly and accurately,” the authors explained. It appears in such classical pieces as Chopin’s Etude Op. 25. No. 6, Maurice Ravel’s “Ondine,” and the first movement of Beethoven’s Piano Sonata No. 3.
High-speed video of experiments on a robotic hula hooper, whose hourglass form holds the hoop up and in place.
Some version of the Hula-Hoop has been around for millennia, but the popular plastic version was introduced by Wham-O in the 1950s and quickly became a fad. Now, researchers have taken a closer look at the underlying physics of the toy, revealing that certain body types are better at keeping the spinning hoops elevated than others, according to a new paper published in the Proceedings of the National Academy of Sciences.
“We were surprised that an activity as popular, fun, and healthy as hula hooping wasn’t understood even at a basic physics level,” said co-author Leif Ristroph of New York University. “As we made progress on the research, we realized that the math and physics involved are very subtle, and the knowledge gained could be useful in inspiring engineering innovations, harvesting energy from vibrations, and improving in robotic positioners and movers used in industrial processing and manufacturing.”
Ristroph’s lab frequently addresses these kinds of colorful real-world puzzles. For instance, in 2018, Ristroph and colleagues fine-tuned the recipe for the perfect bubble based on experiments with soapy thin films. In 2021, the Ristroph lab looked into the formation processes underlying so-called “stone forests” common in certain regions of China and Madagascar.
In 2021, his lab built a working Tesla valve, in accordance with the inventor’s design, and measured the flow of water through the valve in both directions at various pressures. They found the water flowed about two times slower in the nonpreferred direction. In 2022, Ristroph studied the surpassingly complex aerodynamics of what makes a good paper airplane—specifically, what is needed for smooth gliding.
And last year, Ristroph’s lab cracked the conundrum of physicist Richard Feynman’s “reverse sprinkler” problem, concluding that the reverse sprinkler rotates a good 50 times slower than a regular sprinkler but operates along similar mechanisms. The secret is hidden inside the sprinkler, where there are jets that make it act like an inside-out rocket. The internal jets don’t collide head-on; rather, as water flows around the bends in the sprinkler arms, it is slung outward by centrifugal force, leading to asymmetric flow.
The RAVEN walks, it flies, it hops over obstacles, and it’s efficient.
The RAVEN in action. Credit: EPFL/Alain Herzog
Most drones on the market are rotary-wing quadcopters, which can conveniently land and take off almost anywhere. The problem is they are less energy-efficient than fixed-wing aircraft, which can fly greater distances and stay airborne for longer but need a runway, a dedicated launcher, or at least a good-fashioned throw to get to the skies.
To get past this limit, a team of Swiss researchers at the École Polytechnique Fédérale de Lausanne built a fixed-wing flying robot called RAVEN (Robotic Avian-inspired Vehicle for multiple ENvironments) with a peculiar bio-inspired landing gear: a pair of robotic bird-like legs. “The RAVEN robot can walk, hop over obstacles, and do a jumping takeoff like real birds,” says Won Dong Shin, an engineer leading the project.
Smart investments
The key challenge in attaching legs to drones was that they significantly increased mass and complexity. State-of-the-art robotic legs were designed for robots walking on the ground and were too bulky and heavy to even think about using on a flying machine. So, Shin’s team started their work by taking a closer look at what the leg mass budget looked like in various species of birds.
It turned out that the ratio of leg mass to the total body weight generally increased with size in birds. A carrion crow had legs weighing around 100 grams, which the team took as their point of reference.
The robotic legs built by Shin and his colleagues resembled a real bird’s legs quite closely. Simplifications introduced to save weight included skipping the knee joint and actuated toe joints, resulting in a two-segmented limb with 64 percent of the weight placed around the hip joint. The mechanism was powered by a standard drone propeller, with the ankle joint actuated through a system of pulleys and a timing belt. The robotic leg ended with a foot with three forward-facing toes and a single backward-facing hallux.
There were some more sophisticated bird-inspired design features, too. “I embedded a torsional spring in the ankle joint. When the robot’s leg is crouching, it stores the energy in that spring, and then when the leg stretches out, the spring works together with the motor to generate higher jumping speed,” says Shin. A real bird can store elastic energy in its muscle-tendon system during flexion and release it very rapidly during extension for a jumping takeoff. The spring’s job was to emulate this mechanism, and it worked pretty well—“It actually increased the jumping speed by 25 percent,” Shin says.
In the end, the robotic legs weighed around 230 grams, way more than the real ones in a carrion crow, but it turned out that was good enough for the RAVEN robot to walk, jump, take off, and fly.
Crow’s efficiency
The team calculated the necessary takeoff speed for two birds with body masses of 490 grams and a hair over 780 grams; these were 1.85 and 3.21 meters per second, respectively. Based on that, Shin figured the RAVEN robot would need to reach 2.5 meters per second to get airborne. Using the bird-like jumping takeoff strategy, it could reach that speed in just 0.17 seconds.
How did nature’s go-to takeoff procedure stack up against other ways to get to the skies? Other options included a falling takeoff, where you just push your aircraft off a cliff and let gravity do its thing, or standing takeoff, where you position the craft vertically and rely on the propeller to lift it upward. “When I was designing the experiments, I thought the jumping takeoff would be the least energy-efficient because it used extra juice from the battery to activate the legs,” Shin says. But he was in for a surprise.
“What we meant by energy efficiency was calculating the energy input and energy output. The energy output was the kinetic energy and the potential energy at the moment of takeoff, defined as the moment when the feet of the robot stop touching the ground,” Shin explains. The energy input was calculated by measuring the power used during takeoff.
The RAVEN takes flight.
“It turned out that the jumping takeoff was actually the most energy-efficient strategy. I didn’t expect that result. It was quite surprising”, Shin says.
The energy cost of the jumping takeoff was slightly higher than that of the other two strategies, but not by much. It required 7.9 percent more juice than the standing takeoff and 6.9 percent more than the falling takeoff. At the same time, it generated much higher acceleration, so you got way better bang for the buck (at least as far as energy was concerned). Overall, jumping with bird-like legs was 9.7 times more efficient than standing takeoff and 4.9 times more efficient than falling takeoff.
One caveat with the team’s calculations was that a fixed-wing drone with a more conventional design, one using wheels or a launcher, would be much more efficient in traditional takeoff strategies than a legged RAVEN robot. “But when you think about it, birds, too, would fly much better without legs. And yet they need them to move on the ground or hunt their prey. You trade some of the in-flight efficiency for more functions,” Shin claims. And the legs offered plenty of functions.
Obstacles ahead
To demonstrate the versatility of their legged flying robot, Shin’s team put it through a series of tasks that would be impossible to complete with a standard drone. Their benchmark mission scenario involved traversing a path with a low ceiling, jumping over a gap, and hopping onto an obstacle. “Assuming an erect position with the tail touching the ground, the robot could walk and remain stable even without advanced controllers,” Shin claims. Walking solved the problem of moving under low ceilings. Jumping over gaps and onto obstacles was done by using the mechanism used for takeoff: torsion springs and actuators. RAVEN could jump over an 11-centimeter-wide gap and onto an obstacle 26-centimeter-high.
But Shin says RAVEN will need way more work before it truly shines. “At this stage, the robot cannot clear all those obstacles in one go. We had to reprogram it for each of the obstacles separately,” Shin says. The problem is the control system in RAVEN is not adaptive; the actuators in the legs perform predefined sets of motions to send the robot on a trajectory the team figured out through computer simulations. If there was something blocking the way, RAVEN would have crashed into it.
Another, perhaps more striking limitation is that RAVEN can’t use its legs to land. But this is something Shin and his colleagues want to work on in the future.
“We want to implement some sensors, perhaps vision or haptic sensors. This way, we’re going to know where the landing site is, how many meters away from it we are, and so on,” Shin says. Another modification that’s on the way for RAVEN is foldable wings that the robot will use to squeeze through tight spaces. “Flapping wings would also be a very interesting topic. They are very important for landing, too, because birds decelerate first with their wings, not with their legs. With flapping wings, this is going to be a really bird-like robot,” Shin claims.
All this is intended to prepare RAVEN for search and rescue missions. The idea is legged flying robots would reach disaster-struck areas quickly, land, traverse difficult terrain on foot if necessary, and then take off like birds. “Another application is delivering parcels. Here in Switzerland, I often see helicopters delivering them to people living high up in the mountains, which I think is quite costly. A bird-like drone could do that more efficiently,” Shin suggested.
John is Ars Technica’s science editor. He has a Bachelor of Arts in Biochemistry from Columbia University, and a Ph.D. in Molecular and Cell Biology from the University of California, Berkeley. When physically separated from his keyboard, he tends to seek out a bicycle, or a scenic location for communing with his hiking boots.
There’s a common popular science demonstration involving “soap boats,” in which liquid soap poured onto the surface of water creates a propulsive flow driven by gradients in surface tension. But it doesn’t last very long since the soapy surfactants rapidly saturate the water surface, eliminating that surface tension. Using ethanol to create similar “cocktail boats” can significantly extend the effect because the alcohol evaporates rather than saturating the water.
That simple classroom demonstration could also be used to propel tiny robotic devices across liquid surfaces to carry out various environmental or industrial tasks, according to a preprint posted to the physics arXiv. The authors also exploited the so-called “Cheerios effect” as a means of self-assembly to create clusters of tiny ethanol-powered robots.
As previously reported, those who love their Cheerios for breakfast are well acquainted with how those last few tasty little “O”s tend to clump together in the bowl: either drifting to the center or to the outer edges. The “Cheerios effect is found throughout nature, such as in grains of pollen (or, alternatively, mosquito eggs or beetles) floating on top of a pond; small coins floating in a bowl of water; or fire ants clumping together to form life-saving rafts during floods. A 2005 paper in the American Journal of Physics outlined the underlying physics, identifying the culprit as a combination of buoyancy, surface tension, and the so-called “meniscus effect.”
It all adds up to a type of capillary action. Basically, the mass of the Cheerios is insufficient to break the milk’s surface tension. But it’s enough to put a tiny dent in the surface of the milk in the bowl, such that if two Cheerios are sufficiently close, the curved surface in the liquid (meniscus) will cause them to naturally drift toward each other. The “dents” merge and the “O”s clump together. Add another Cheerio into the mix, and it, too, will follow the curvature in the milk to drift toward its fellow “O”s.
Physicists made the first direct measurements of the various forces at work in the phenomenon in 2019. And they found one extra factor underlying the Cheerios effect: The disks tilted toward each other as they drifted closer in the water. So the disks pushed harder against the water’s surface, resulting in a pushback from the liquid. That’s what leads to an increase in the attraction between the two disks.
On Wednesday, researchers reported that they had developed a drone they’re calling the CoulombFly, which is capable of self-powered hovering for as long as the Sun is shining. The drone, which is shaped like no aerial vehicle you’ve ever seen before, combines solar cells, a voltage converter, and an electrostatic motor to drive a helicopter-like propeller—with all components having been optimized for a balance of efficiency and light weight.
Before people get excited about buying one, the list of caveats is extensive. There’s no onboard control hardware, and the drone isn’t capable of directed flight anyway, meaning it would drift on the breeze if ever set loose outdoors. Lots of the components appear quite fragile, as well. However, the design can be miniaturized, and the researchers built a version that weighs only 9 milligrams.
Built around a motor
One key to this development was the researchers’ recognition that most drones use electromagnetic motors, which involve lots of metal coils that add significant weight to any system. So, the team behind the work decided to focus on developing a lightweight electrostatic motor. These rely on charge attraction and repulsion to power the motor, as opposed to magnetic interactions.
The motor the researchers developed is quite large relative to the size of the drone. It consists of an inner ring of stationary charged plates called the stator. These plates are composed of a thin carbon-fiber plate covered in aluminum foil. When in operation, neighboring plates have opposite charges. A ring of 64 rotating plates surrounds that.
The motor starts operating when the plates in the outer ring are charged. Since one of the nearby plates on the stator will be guaranteed to have the opposite charge, the pull will start the rotating ring turning. When the plates of the stator and rotor reach their closest approach, thin wires will make contact, allowing charges to transfer between them. This ensures that the stator and rotor plates now have the same charge, converting the attraction to a repulsion. This keeps the rotor moving, and guarantees that the rotor’s plate now has the opposite charge from the next stator plate down the line.
These systems typically require very little in the way of amperage to operate. But they do require a large voltage difference between the plates (something we’ll come back to).
When hooked up to a 10-centimeter, eight-bladed propeller, the system could produce a maximum lift of 5.8 grams. This gave the researchers clear weight targets when designing the remaining components.
Ready to hover
The solar power cells were made of a thin film of gallium arsenide, which is far more expensive than other photovoltaic materials, but offers a higher efficiency (30 percent conversion compared to numbers that are typically in the mid-20s). This tends to provide the opposite of what the system needs: reasonable current at a relatively low voltage. So, the system also needed a high-voltage power converter.
Here, the researchers sacrificed efficiency for low weight, arranging a bunch of voltage converters in series to create a system that weighs just 1.13 grams, but steps the voltage up from 4.5 V all the way to 9.0 kV. But it does so with a power conversion efficiency of just 24 percent.
The resulting CoulombFly is dominated by the large cylindrical motor, which is topped by the propeller. Suspended below that is a platform with the solar cells on one side, balanced out by the long, thin power converter on the other.
Meet the CoulombFly.
To test their system, the researchers simply opened a window on a sunny day in Beijing. Starting at noon, the drone took off and hovered for over an hour, and all indications are that it would have continued to do so for as long as the sunlight provided enough power.
The total system required just over half a watt of power to stay aloft. Given a total mass of 4 grams, that works out to a lift-to-power efficiency of 7.6 grams per watt. But a lot of that power is lost during the voltage conversion. If you focus on the motor alone, it only requires 0.14 watts, giving it a lift-to-power efficiency of over 30 grams per watt.
The researchers provide a long list of things they could do to optimize the design, including increasing the motor’s torque and propeller’s lift, placing the solar cells on structural components, and boosting the efficiency of the voltage converter. But one thing they don’t have to optimize is the vehicle’s size since they already built a miniaturized version that’s only 8 millimeters high and weighs just 9 milligrams but is able to generate a milliwatt of power that turns its propeller at over 15,000 rpm.
Again, all this is done without any onboard control circuitry or the hardware needed to move the machine anywhere—they’re basically flying these in cages to keep them from wandering off on the breeze. But there seems to be enough leeway in the weight that some additional hardware should be possible, especially if they manage some of the potential optimizations they mentioned.
Enlarge/ Right now, the software doesn’t do arms, so don’t go taking on any aliens with it.
20th Century Fox
Exoskeletons today look like something straight out of sci-fi. But the reality is they are nowhere near as robust as their fictional counterparts. They’re quite wobbly, and it takes long hours of handcrafting software policies, which regulate how they work—a process that has to be repeated for each individual user.
To bring the technology a bit closer to Avatar’s Skel Suits or Warhammer 40k power armor, a team at North Carolina University’s Lab of Biomechatronics and Intelligent Robotics used AI to build the first one-size-fits-all exoskeleton that supports walking, running, and stair-climbing. Critically, its software adapts itself to new users with no need for any user-specific adjustments. “You just wear it and it works,” says Hao Su, an associate professor and co-author of the study.
Tailor-made robots
An exoskeleton is a robot you wear to aid your movements—it makes walking, running, and other activities less taxing, the same way an e-bike adds extra watts on top of those you generate yourself, making pedaling easier. “The problem is, exoskeletons have a hard time understanding human intentions, whether you want to run or walk or climb stairs. It’s solved with locomotion recognition: systems that recognize human locomotion intentions,” says Su.
Building those locomotion recognition systems currently relies on elaborate policies that define what actuators in an exoskeleton need to do in each possible scenario. “Let’s take walking. The current state of the art is we put the exoskeleton on you and you walk on a treadmill for an hour. Based on that, we try to adjust its operation to your individual set of movements,” Su explains.
Building handcrafted control policies and doing long human trials for each user makes exoskeletons super expensive, with prices reaching $200,000 or more. So, Su’s team used AI to automatically generate control policies and eliminate human training. “I think within two or three years, exoskeletons priced between $2,000 and $5,000 will be absolutely doable,” Su claims.
His team hopes these savings will come from developing the exoskeleton control policy using a digital model, rather than living, breathing humans.
Digitizing robo-aided humans
Su’s team started by building digital models of a human musculoskeletal system and an exoskeleton robot. Then they used multiple neural networks that operated each component. One was running the digitized model of a human skeleton, moved by simplified muscles. The second neural network was running the exoskeleton model. Finally, the third neural net was responsible for imitating motion—basically predicting how a human model would move wearing the exoskeleton and how the two would interact with each other. “We trained all three neural networks simultaneously to minimize muscle activity,” says Su.
One problem the team faced is that exoskeleton studies typically use a performance metric based on metabolic rate reduction. “Humans, though, are incredibly complex, and it is very hard to build a model with enough fidelity to accurately simulate metabolism,” Su explains. Luckily, according to the team, reducing muscle activations is rather tightly correlated with metabolic rate reduction, so it kept the digital model’s complexity within reasonable limits. The training of the entire human-exoskeleton system with all three neural networks took roughly eight hours on a single RTX 3090 GPU. And the results were record-breaking.
Bridging the sim-to-real gap
After developing the controllers for the digital exoskeleton model, which were developed by the neural networks in simulation, Su’s team simply copy-pasted the control policy to a real controller running a real exoskeleton. Then, they tested how an exoskeleton trained this way would work with 20 different participants. The averaged metabolic rate reduction in walking was over 24 percent, over 13 percent in running, and 15.4 percent in stair climbing—all record numbers, meaning their exoskeleton beat every other exoskeleton ever made in each category.
This was achieved without needing any tweaks to fit it to individual gaits. But the neural networks’ magic didn’t end there.
“The problem with traditional, handcrafted policies was that it was just telling it ‘if walking is detected do one thing; if walking faster is detected do another thing.’ These were [a mix of] finite state machines and switch controllers. We introduced end-to-end continuous control,” says Su. What this continuous control meant was that the exoskeleton could follow the human body as it made smooth transitions between different activities—from walking to running, from running to climbing stairs, etc. There was no abrupt mode switching.
“In terms of software, I think everyone will be using this neural network-based approach soon,” Su claims. To improve the exoskeletons in the future, his team wants to make them quieter, lighter, and more comfortable.
But the plan is also to make them work for people who need them the most. “The limitation now is that we tested these exoskeletons with able-bodied participants, not people with gait impairments. So, what we want to do is something they did in another exoskeleton study at Stanford University. We would take a one-minute video of you walking, and based on that, we would build a model to individualize our general model. This should work well for people with impairments like knee arthritis,” Su claims.
Enlarge/ A kitty named Clover prepares to play with a robot arm in the Cat Royale “multi-species” science/art installation .
Blast Theory – Stephen Daly
Cats and robots are a winning combination, as evidenced by all those videos of kitties riding on Roombas. And now we have Cat Royale, a “multispecies” live installation in which three cats regularly “played” with a robot over 12 days, carefully monitored by human operators. Created by computer scientists from the University of Nottingham in collaboration with artists from a group called Blast Theory, the installation debuted at the World Science Festival in Brisbane, Australia, last year and is now a touring exhibit. The accompanying YouTube video series recently won a Webby Award, and a paper outlining the insights gleaned from the experience was similarly voted best paper at the recent Computer-Human Conference (CHI’24).
“At first glance, the project is about designing a robot to enrich the lives of a family of cats by playing with them,” said co-author Steve Benford of the University of Nottingham, who led the research, “Under the surface, however, it explores the question of what it takes to trust a robot to look after our loved ones and potentially ourselves.” While cats might love Roombas, not all animal encounters with robots are positive: Guide dogs for the visually impaired can get confused by delivery robots, for example, while the rise of lawn mowing robots can have a negative impact on hedgehogs, per Benford et al.
Blast Theory and the scientists first held a series of exploratory workshops to ensure the installation and robotic design would take into account the welfare of the cats. “Creating a multispecies system—where cats, robots, and humans are all accounted for—takes more than just designing the robot,” said co-author Eike Schneiders of Nottingham’s Mixed Reality Lab about the primary takeaway from the project. “We had to ensure animal well-being at all times, while simultaneously ensuring that the interactive installation engaged the (human) audiences around the world. This involved consideration of many elements, including the design of the enclosure, the robot, and its underlying systems, the various roles of the humans-in-the-loop, and, of course, the selection of the cats.”
Based on those discussions, the team set about building the installation: a bespoke enclosure that would be inhabited by three cats for six hours a day over 12 days. The lucky cats were named Ghostbuster, Clover, and Pumpkin—a parent and two offspring to ensure the cats were familiar with each other and comfortable sharing the enclosure. The enclosure was tricked out to essentially be a “utopia for cats,” per the authors, with perches, walkways, dens, a scratching post, a water fountain, several feeding stations, a ball run, and litter boxes tucked away in secluded corners.
Enlarge/ (l-r) Clover, Pumpkin, and Ghostbuster spent six hours a day for 12 days in the installation.
E. Schneiders et al., 2024
As for the robot, the team chose the Kino Gen3 lite robot arm, and the associated software was trained on over 7,000 videos of cats. A decision engine gave the robot autonomy and proposed activities for specific cats. Then a human operator used an interface control system to instruct the robot to execute the movements. The robotic arm’s two-finger gripper was augmented with custom 3D-printed attachments so that the robot could manipulate various cat toys and accessories.
Each cat/robot interaction was evaluated for a “happiness score” based on the cat’s level of engagement, body language, and so forth. Eight cameras monitored the cat and robot activities, and that footage was subsequently remixed and edited into daily YouTube highlight videos and, eventually, an eight-hour film.
Reinforcement-learning algorithms in systems like ChatGPT or Google’s Gemini can work wonders, but they usually need hundreds of thousands of shots at a task before they get good at it. That’s why it’s always been hard to transfer this performance to robots. You can’t let a self-driving car crash 3,000 times just so it can learn crashing is bad.
But now a team of researchers at Northwestern University may have found a way around it. “That is what we think is going to be transformative in the development of the embodied AI in the real world,” says Thomas Berrueta who led the development of the Maximum Diffusion Reinforcement Learning (MaxDiff RL), an algorithm tailored specifically for robots.
Introducing chaos
The problem with deploying most reinforcement-learning algorithms in robots starts with the built-in assumption that the data they learn from is independent and identically distributed. The independence, in this context, means the value of one variable does not depend on the value of another variable in the dataset—when you flip a coin two times, getting tails on the second attempt does not depend on the result of your first flip. Identical distribution means that the probability of seeing any specific outcome is the same. In the coin-flipping example, the probability of getting heads is the same as getting tails: 50 percent for each.
In virtual, disembodied systems, like YouTube recommendation algorithms, getting such data is easy because most of the time it meets these requirements right off the bat. “You have a bunch of users of a website, and you get data from one of them, and then you get data from another one. Most likely, those two users are not in the same household, they are not highly related to each other. They could be, but it is very unlikely,” says Todd Murphey, a professor of mechanical engineering at Northwestern.
The problem is that, if those two users were related to each other and were in the same household, it could be that the only reason one of them watched a video was that their housemate watched it and told them to watch it. This would violate the independence requirement and compromise the learning.
“In a robot, getting this independent, identically distributed data is not possible in general. You exist at a specific point in space and time when you are embodied, so your experiences have to be correlated in some way,” says Berrueta. To solve this, his team designed an algorithm that pushes robots be as randomly adventurous as possible to get the widest set of experiences to learn from.
Two flavors of entropy
The idea itself is not new. Nearly two decades ago, people in AI figured out algorithms, like Maximum Entropy Reinforcement Learning (MaxEnt RL), that worked by randomizing actions during training. “The hope was that when you take as diverse set of actions as possible, you will explore more varied sets of possible futures. The problem is that those actions do not exist in a vacuum,” Berrueta claims. Every action a robot takes has some kind of impact on its environment and on its own condition—disregarding those impacts completely often leads to trouble. To put it simply, an autonomous car that was teaching itself how to drive using this approach could elegantly park into your driveway but would be just as likely to hit a wall at full speed.
To solve this, Berrueta’s team moved away from maximizing the diversity of actions and went for maximizing the diversity of state changes. Robots powered by MaxDiff RL did not flail their robotic joints at random to see what that would do. Instead, they conceptualized goals like “can I reach this spot ahead of me” and then tried to figure out which actions would take them there safely.
Berrueta and his colleagues achieved that through something called ergodicity, a mathematical concept that says that a point in a moving system will eventually visit all parts of the space that the system moves in. Basically, MaxDiff RL encouraged the robots to achieve every available state in their environment. And the results of first tests in simulated environments were quite surprising.
Racing pool noodles
“In reinforcement learning there are standard benchmarks that people run their algorithms on so we can have a good way of comparing different algorithms on a standard framework,” says Allison Pinosky, a researcher at Northwestern and co-author of the MaxDiff RL study. One of those benchmarks is a simulated swimmer: a three-link body resting on the ground in a viscous environment that needs to learn to swim as fast as possible in a certain direction.
In the swimmer test, MaxDiff RL outperformed two other state-of-the-art reinforcement learning algorithms (NN-MPPI and SAC). These two needed several resets to figure out how to move the swimmers. To complete the task, they were following a standard AI learning process divided down into a training phase where an algorithm goes through multiple failed attempts to slowly improve its performance, and a testing phase where it tries to perform the learned task. MaxDiff RL, by contrast, nailed it, immediately adapting its learned behaviors to the new task.
The earlier algorithms ended up failing to learn because they got stuck trying the same options and never progressing to where they could learn that alternatives work. “They experienced the same data repeatedly because they were locally doing certain actions, and they assumed that was all they could do and stopped learning,” Pinosky explains. MaxDiff RL, on the other hand, continued changing states, exploring, getting richer data to learn from, and finally succeeded. And because, by design, it seeks to achieve every possible state, it can potentially complete all possible tasks within an environment.
But does this mean we can take MaxDiff RL, upload it to a self-driving car, and let it out on the road to figure everything out on its own? Not really.
3D model of a 100-year-old shipwreck off the western coast of Australia. Credit: Daniel Adams, Curtin University HIVE.
A small underwater drone called Hydrus has located the wreckage of a 100-year-old coal hulk in the deep waters off the coast of western Australia. Based on the data the drone captured, scientists were able to use photogrammetry to virtually “rebuild” the 210-foot ship into a 3D model (above). You can explore an interactive 3D rendering of the wreckage here.
The use of robotic submersibles to locate and explore historic shipwrecks is well established. For instance, researchers relied on remotely operated vehicles (ROVs) to study the wreckage of the HMS Terror, Captain Sir John S. Franklin‘s doomed Arctic expedition to cross the Northwest Passage in 1846. In 2007, a pair of brothers (printers based in Norfolk) discovered the wreck of the Gloucester, which ran aground on a sandbank off the coast of Norfolk in 1682 and sank within the hour. Among the passengers was James Stuart, Duke of York and future King James II of England, who escaped in a small boat just before the ship sank.
In 2022, the Falklands Maritime Heritage Trust and National Geographicannounced the discovery of British explorer Sir Ernest Shackleton‘s ship Endurance. In 1915, Shackleton and his crew were stranded for months on the Antarctic ice after the ship was crushed by pack ice and sank into the freezing depths of the Weddell Sea. The wreckage was found nearly 107 years later, 3,008 meters down, roughly four miles (6.4 km) south of the ship’s last recorded position. The wreck was in pristine condition partly because of the lack of wood-eating microbes in those waters. In fact, the lettering “ENDURANCE” was clearly visible in shots of the stern.
And just last year, an ROV was used to verify the discovery of the wreckage of a schooner barge called Ironton, which collided with a Great Lakes freighter called Ohio in Lake Huron’s infamous “Shipwreck Alley” in 1894. The wreck was so well-preserved in the frigid waters of the Great Lakes that its three masts were still standing and its rigging still attached. That discovery could help resolve unanswered questions about the ship’s final hours.
Enlarge/ Deployment of one of Advanced Navigation’s Micro Autonomous Underwater Vehicles (AUV).
Advanced Navigation
According to Advanced Navigation, there are some 3 million undiscovered shipwrecks around the world—1,819 recorded wrecks lying off the coast of Western Australia alone. That includes the Rottnest ship graveyard just southwest of Rottnest Island, with a seabed some 50 to 200 meters below sea level (164 to 656 feet). The island is known for the number of ships wrecked near its shore since the 17th century. The Rottnest graveyard is more of a dump site for scuttling obsolete ships, at least 47 of which would be considered historically significant.
However, this kind of deep ocean exploration can be both time-consuming and expensive, particularly at depths of more than 50 meters (164 feet). Hydrus was designed to reduce the cost of this kind of ocean exploration significantly. One person can deploy the drone because of its compact size, so there is no need for large vessels or complicated launch systems. And Hydrus can capture georeferenced 4K video and still images at the same time. Once this latest expedition realized they had found a shipwreck, they were able to deploy a pair of the drones to take a complete survey in just five hours.
Hydrus captured this footage of the 210-foot wreck of a 19th-century coal hulk. Credit: Advanced Navigation
Ross Anderson, curator of the Western Australia Museum, was able to identify the wreck as an iron coal hulk once used in Freemantle Port to service steamships, probably built in the 1860s–1890s and scuttled in the graveyard sometime in the 1920s. The geolocation data provided to scientists at Curtin University HIVE enabled them to use photogrammetry to convert that data into a 3D digital model. “It can’t be overstated how much this structure in data assists with constraining feature matching and reducing the processing time, especially in large datasets,” Andrew Woods, a professor at the university, said in a statement.
The expedition team’s next target using the Hydrus technology is the wreck of the luxury passenger steamship SS Koombana, which disappeared somewhere off Port Hedland en route to Broome during a tropical cyclone in 1912, with 150 on board presumed to have perished. The only wreckage recovered at the time was part of a starboard bow planking, a stateroom door, a panel from the promenade deck, and a few air tanks. There were a couple of reports in the 1980s of “magnetic anomalies” in the seabed off Bedout Island, part of the route the Koombana would have taken. But despite several deep-water expeditions in the early 2010s, to date the actual shipwreck has not been found.
