The "stumbling" robot dog died in just one hour...

To avoid predators, animals such as newborn giraffes or foals must learn to walk as fast as possible on their legs.

However, learning to precisely coordinate the muscles and tendons in your legs takes some time.

Initially, young animals rely heavily on innate spinal reflexes, motor control reflexes that help them avoid falling and getting injured when they first try to walk.

Afterwards, they must learn more advanced and precise muscle control until the nervous system eventually adapts to the leg muscles and tendons.

Eventually, they become like adult animals, with no more uncontrolled stumbling.

This process may be very short (for example, cows and sheep can walk as soon as they are born), or it may take a few days to a few weeks (for example, cats and dogs need some time to learn), or it may be as long as 1 year (for example, human toddlers learn to walk very slowly).

(Source: Pixabay)

So, the question is, how do animals learn to walk and learn from their stumbling?

To this end, a research team at the Max Planck Institute for Intelligent Systems (MPI-IS) conducted a study in which they built a four-legged, dog-sized robot in the hope of answering this question.

Figure｜Walking on a treadmill (Source: Morti)

The related research paper, titled “Learning plastic matching of robot dynamics in closed-loop central pattern generators”, has been published in the scientific journal Nature Machine Intelligence.

"As engineers and roboticists, we set out to find the answer by building a robot that had animal-like reflexes and learned from its mistakes," said Felix Ruppert, first and corresponding author of the paper.

"If the animal falls, is it a bug? If it happens only once, it's not a bug. But if it falls often, it gives us a measure of how well the robot can walk."

Using algorithms to optimize the "virtual spinal cord"

According to the paper, the robot dog, named Morti, took only one hour to learn to walk and mastered the complex leg mechanics.

Figure｜Robot dog Morti (Source: MPI-IS)

During this process, the research team used a Bayesian optimization algorithm to guide the robot dog's learning: the measured foot sensor information was matched with the target data of the virtual spinal cord model, which runs as a program in the robot's "brain".

The robot learns to walk by constantly comparing sent and expected sensor information, running reflex loops, and adjusting its motion control patterns.

The learning algorithm is similar to the control parameters of a central pattern generator (CPG).

In humans and animals, these CPGs are networks of neurons in the spinal cord that generate periodic muscle contractions without input from the brain. CPG networks help produce rhythmic tasks, such as walking, blinking, or digesting.

Furthermore, reflexes are involuntary motor control behaviors triggered by hard-coded neural pathways connecting leg sensors to the spinal cord.

As long as the animal walks on a perfectly flat surface, the CPG is sufficient to control movement signals from the spinal cord.

However, just a small collision with the ground can change the way they walk.

At this point, their own reflexes kick in to help adjust movement patterns and prevent falls.

These momentary changes in motor signals are reversible, or "elastic," and the movement patterns return to their original form after being regulated.

But if they still stumble after multiple cycles of movement—albeit as an active reflex—then those movement patterns must be relearned and made “irreversible.”

When animals are first born, their CPGs are not yet well adjusted, and they stumble on both flat and uneven terrain. However, the animals quickly learn how the CPGs and reflexes control the muscles and tendons in their legs.

(Source: MPI-IS)

The same is true for "Morti," a Labrador-sized robot dog.

What’s more, Morti optimizes its movement patterns faster than a small animal, taking only about an hour.

Morti's CPG is simulated on a small computer that controls the movement of the robot's legs.

This virtual spinal cord was placed on Morti's back, where the head would be.

As the robot walks steadily, sensor data from Morti's feet is constantly compared with the expected touchdown predicted by its own CPG.

If the robot falls, the learning algorithm changes how far the legs swing back and forth, how fast they swing, and how long the legs stay on the ground.

The adjusted movement will also tell Morti how to better utilize leg mechanics later on.

During the learning process, Morti's CPG sends adaptive motion signals to optimize its walking and reduce stumbling.

In this framework, Morti’s virtual spinal cord has no knowledge of its own leg design, power source, or body structure. Without any knowledge of its own physical structure, Morti lacks a robot “model.”

To this, Ruppert explains: “Morti doesn’t actually know the anatomy of its legs and how they work.”

"CPGs are similar to the built-in automatic walking intelligence provided by nature, which we have transferred to the robot. The computer generates signals to control the motors in the legs, and the robot walks and stumbles. Data is sent from sensors to the virtual spinal cord and compared with the CPG data. If the sensor data does not match the expected data, the learning algorithm changes the walking behavior until the robot walks well and does not stumble. A core part of the learning process is changing the output of the CPG while keeping the reaction active and monitoring the robot's stumbling."

Energy-saving robot dog control

Morti's small computer consumes only 5 watts of power while walking.

But most existing industrial quadruped robots have much greater power requirements. Their controllers use a model of the robot, encoded with the robot's precise mass and body geometry, and typically consume tens to hundreds of watts.

Both types of robots operate dynamically and efficiently, but Morti's energy consumption is much lower and it also provides important insights into the animal's anatomy.

"We can't easily study the spinal cord in living animals. But we can build a model in a robot," says Alexander Badri-Spröwitz, one of the authors of the paper.

"We know these CPGs exist in many animals. We know the reflexes are intrinsic; but how do you combine the two so that an animal learns both the reflex and the CPG movement? This is fundamental research at the intersection of robotics and biology. Our robotic model provides answers to questions that biology cannot answer."

In future work, the research team will continue to expand CPG to take into account the body pitch motion when generating the hip trajectory. The body pitch can be fed back into the CPG through the inertial measurement unit.

References:

https://www.nature.com/articles/s42256-022-00505-4