Drawing like humans, GT team develops graffiti robot system

Recently, a research team from Georgia Institute of Technology built the first graffiti robot system that can mimic the fluidity of human movement. The robot, named GTGraffiti, uses motion capture technology to record human painting movements, and then programs a cable-driven robot for spraying graffiti artworks by combining and processing these gestures, thereby completing the painting graffiti of human artists.

The project was designed by Gerry Chen, a PhD student in robotics at Georgia Tech, and was completed by graduate student Juan-Diego Florez, Frank Dellaert, professor of robotics in the School of Interactive Computing, Seth Hutchinson, professor of KUKA Robotics, and Sang-won Leigh, assistant professor in the School of Industrial Design. A peer-reviewed study of the robotic system will be published in the Proceedings of the International Conference on Robotics and Automation in June 2022.

How do robots learn to doodle?

For a robot that can draw in human style, both the robot and the art must be considered in the design. The GTGraffiti robot system consists of three parts: drawing capture, robot hardware, and planning control.

First, the research team used motion capture technology to record human artists painting, which provided insight into the types of movements required to create graffiti artwork. In this study, Chen and his team invited two artists to draw alphabetic symbols in a bubble letter graffiti style. While the human artists were painting, they recorded the movements of the artists' hands on the canvas, as well as the movements of the spray paint cans themselves.

(Source: GT)

Capturing the trajectory of the hand and the spray can was crucial for the robot to be able to paint using similar layering, composition, and movement as a human artist. The team then processed the data to analyze the speed, acceleration, and force level of each movement and used this information in the next phase - designing the robot.

Given this data, as well as the portability and accuracy required for the artwork, the researchers chose to use a cable-driven robot. A cable-driven robot on a cable and pulley system is able to move flexibly over a wide range of motion. The team then mounted the robot on a 9 x 10-foot steel frame, which the researchers say can also be mounted directly on a flat structure of almost any size, such as the side of a building.

In a third step, the researchers converted the artist's composition into electrical signals to generate text for the robot to paint. The researchers then used mathematical equations to generate trajectories for the robot to follow. These algorithmic paths ensure that the robot paints at the correct speed, position, direction, and perspective. Finally, these paths are converted into motion instructions to be executed.

To solve the problem of winding of the machine system motors, the robot central controller is programmed to recalculate the motor commands 1,000 times per second so that the robot can operate safely and reliably. Once assembled, the robot can draw artwork in the style of a human graffiti artist.

(Source: GT)

In the test session, a human artist selected the letters "ATL" and used them to create a piece of graffiti artwork.

Why art? Why graffiti?

Some of the most iconic industries for robotics applications include manufacturing, biomedicine, automotive, agriculture, and the military. But it turns out that art can showcase robotics in a particularly powerful way.

“Art, especially painting or dancing, embodies some of the most complex and subtle movements that humans can make. So if we want to create robots that can do highly technical things that humans do, creating robots that can dance or paint is a good goal. These are the types of skills that demonstrate the extraordinary capabilities of robots and can also be applied to a variety of other applications,” Chen said.

On a personal level, Chen’s idea is for people to see robots as helpful to humanity, rather than as job replacers or entities that cause fear, sadness or a sense of doom, as they are often portrayed in movies.

“Graffiti is an art form that is essentially meant to be seen by the public. In this regard, I hope we can use graffiti to convey the idea that robots working alongside humans can make a positive contribution to society.”

Currently, Chen and his team’s plans for the robot revolve around two main aspects: preserving and amplifying art. To this end, they are currently experimenting with reproducing pre-recorded shapes at different scales and testing the robot’s ability to map larger surfaces.

These capabilities will enable the robot to paint enlarged versions of original works in different geographical locations and provide assistance to artists who are unable to engage in on-site spray painting. In theory, an artist can paint a work of art in one part of the world, and the GTGraffiti robot can execute the art in another location.

In the future, Chen hopes to use GTGraffiti to capture artists painting graffiti in the wild. Using the captured motion data, GTGraffiti will be able to replicate the artwork without worrying about it being painted over or vandalized.

“The robots themselves aren’t creating art, but rather collaborating with human artists to enable them to achieve more than they could without the robots,” Chen said.

Chen envisions that the robotic system will eventually have capabilities that allow artists to interact with the robot in real time. For example, he hopes to develop technology that would allow an artist standing at the foot of a building to spray-paint graffiti in a small space while a cable-driven robot simultaneously paints the side of the building in giant brushstrokes.

“We hope that our research can help artists create artworks executed by superhuman robots that convey messages more powerfully than anything they could draw in person,” Chen said.

References:

https://research.gatech.edu/introducing-gtgraffiti-robot-paints-human

https://dx.doi.org/10.48550/arXiv.2109.06238