Creating art is an intensive and time-consuming process. It’s not just envisioning and designing the piece that’s challenging — the labor of painting also takes a lot of time. But what if robots could help with this, and maybe even expand an artist’s repertoire?

It may seem far-fetched, but in a new study, researchers paved the way for exactly this: they trained a swarm of robots to be used in producing art.

María Santos was always fascinated by the intersection of engineering and arts. A musician herself, she loves to explore this overlap of seemingly different worlds, she tells ZME Science.

“During my PhD at the School of Electrical Engineering at Georgia Tech, I was given the opportunity of combining my research on control theory and multi-robot systems with different forms of art,” she says.

It all started in a previous study with her doctoral advisor, Professor Magnus Egersted. The two first studied the expressive capabilities of robot swarms to convey basic emotions and then moved on to look at the individual trajectories executed by the swarm of robots.

Is there some artistic merit to this, or could this approach be applied in an artistic setting as a tool? Santos believes so.

“In this study we explore how the integration of such trajectories over time can lead to artistic paintings by making the robots leave physical trails as they move,” Santos explains in an email.

“We envisioned the multi-robot system as an extension of an artist’s creative palette. The presented painting swarm along with all its control knobs embody new means of interaction between artists and the piece of art, whereby artists can explore new creative directions, intuitively interacting with a robotic system while not having to concern themselves with aspects such as individual robot control or available paints to each robot.”

At first glance, using robots for art seems like a weird idea, but it makes sense once you look at it. Painting is typically labor-intensive, and despite the world around us becoming more and more automated, painting has remained exclusively a manual endeavor. The idea is not to have the robots create art, but rather for artists to use the robots as a tool to ease their workload or explore new artistic directions.

The robots in the project move about a canvas leaving color trails, and the artist can select the areas of the canvas to be painted in a certain color — the robots will oblige in real time. It’s a bit like applying digital techniques into the real-life analog world and can serve as an interesting tool for artists.

The way Santos envisions the approach, the artist would control the swarm behavior, but not necessarily every individual robot.

“In this approach, the robotic swarm can be thought of as an “active” brush for the human artist to paint with, where the individual robots (active bristles) move over the canvas according to the color specifications given by the human at each point in time. Thus, the artist can control the collective behavior of the swarm and potentially some other general parameters (how much paint to release, how sharp the trajectories of the robots may be), but not the individual movements of each robot.”

This leaves a wide array of parameters the artist can influence to produce the desired effect, and explore different variations. It’s akin to how a composer writes variations on a theme, Santos tells me.

In the experiments, the researchers used a projector to simulate the colored paint trail with a digital input, although they will soon replace this with a robot that handles actual paint. They found that even when the robot doesn’t have access to the desired color, it is capable to collaborate with other robots and approximate the color. This means the artist doesn’t need to worry whether the robots have access to all the possible colors.

Now, the researchers hope to collaborate with artists to see how this approach could be best tweaked to make it work in real life. The current pandemic, however, has proven to be quite a hurdle.

“We would love to get feedback from artists! In fact, when we started this project, our idea was to get artists to come to the lab and interact with the robotic swarm. This way we could see what they could come up with creatively in terms of generated paints, but also to get their input about which features would be most interesting to develop as the project progresses further.”

“However, due to COVID19, this part was infeasible during the last months, so we focus on studying the characteristics of the paintings as a function of different parameters in the swarm.”

Ultimately, the team hopes to develop this into a full-scale artistic project and allow artists and the public to experiment with it

“As of now, the artworks were created to evaluate the operation of the system, but we would love to exhibit them! Once we can get people back in the lab to try the system, we would love to see what people would come up with.”

Journal Reference:  Interactive Multi-Robot Painting Through Colored Motion Trails, Frontiers in Robotics and AI(2020). DOI: 10.3389/frobt.2020.580415