How can robots create art?

Creating physical art is a hard problem for tech as there are no clear rules and the actual painting process can be unpredictable. Robots that paint operate in two general ways– by direct human involvement (e.g., manual or even telekinetic remote operation of the robot arm) or by getting painting commands that are created by software.

The robots who paint with direct human involvement can get input multiple ways. The most obvious way is to create a physical tool that a human artist can move and have a robot mimic the motions remotely in a similar way that remote doctors can perform surgery. When we first setup the competition, we thought that this would be a popular technique but it turned out that the teams wanted more of a challenge – using AI to decide what and how to paint.

The software-generated paintings commands can be considered as the technique of the “artist” and often are similar to how human artist paint. The human designers of the robotic artist approach the creation of the techniques in different ways. The most direct way is to start with a photo or image of what is desired to painted and then write software that can look for patterns and shapes that can be converted into brush strokes. A straightforward technique is to simplify a photo into several different colors and then tell the robot to paint each color separately. Paintings done like this can have a “paint by number” look.  A more elaborate technique is to let the robot paint a bit “outside the lines” and then use a camera to see what the robot needs to do to make adjustments so that the painting more closely matches the desired image, called a corrective feedback loop, to improve a painting over time. Other techniques include strategies that decompose an image into large abstract shapes and paint them first in what human artist call an underlayer. They then paint additional layers on top to add detail and richness to the painting.  The human designer gets to create different painting techniques for a wide range of painting styles. By looking at the different styles of artwork submitted in this years contest, you can see how each technique becomes the signature style of the robot.

Starting with a source photo or image isn’t the only way to create art with robots. It is possible to completely generate the desired image using AI (artificial intelligence) and then use either specially designed or some of the previously mentioned painting techniques to create the physical painting.   We anticipate that future RobotArt teams will even try to synthesize existing art using machine learning techniques like deep learning to either determine and repurpose its core essence or extend the creative process in interesting new ways (see Google’s “inception” art for more examples).

What is possible?

Is it possible to resurrect famous painters or even create new artists using artificial intelligence and robotics?  We think so and have seen amazing progress in just the last few years.  An artist typically starts with a concept, be it their unique way of seeing things or a message to be expressed, before converting it into a tangible form.  A recent breakthrough in AI, called image style transfer, addresses the first part of the artistic process – that is, codifying the mental process of “seeing the world” from an artist viewpoint.  It is now easy to take an example of an artist’s work and apply it to any image or video to create the appearance that the artist painted it.  The following shows an example of how image style transfer can imagine how Picasso might have interpreted the source image.  See more examples here.

 

 

 

 

 

 

Our competition highlights the messy final step of converting an idea of a painting into it’s physical form.

How to know if a robot succeeds in creating art?

Whether or not a robot has succeeded in creating something beautiful is easy to judge. You don’t need to be an expert to know if you like an artwork. This is why we decided to open up part of the judging process to the general public. We had over 2200 people register to vote on their favorite artwork. Each person was given 10 votes that they could distribute among their favorites – with up to 3 votes per artwork.

What are some benefits of students participating?

For the most part, engineering graduates rarely get an opportunity to take art-related classes. The RobotArt competition is a way to give students an opportunity to get hands-on with the art creation process.   Students quickly find that science and engineering share many of the innovative problem solving skills used by artist such as balancing details and abstraction, looking at reality in different ways, and planning steps that build on each other to become greater than the sum of the parts.  Oh yah, and it’s cool.  Who doesn’t like watching a robot whip out a painting?

Does robot art invalidate human generated art?

Art can be appreciated on many levels – technique mastery, aesthetics, and even from a personal connection with the artist (ask any parent with their child’s artwork taped to the refrigerator).  The camera didn’t invalidate the portrait artist as the portrait artist was often trying to capture a deeper emotion that a perfect copy of the sitter.  However, the camera opened up a whole new form of art – photography.  Likewise, human generate art will always be highly respected not only for its creativity but in our shared human experience.  While we may be impressed by AI chess software, we are thrilled and impressed by human grandmasters.

That being said, robots and AI will enable artist to attempt art that are perhaps intellectually or physically more ambition than before.  AI advances in human mimicry or extension might also affect the fundamental connection between the artwork and those who interact with it.

While this contest can be though a “John Henry” moment where we are investigating the fuzzy, less analytical, comparisons between human and machine, in art, there are no losers.  We all win when we see something beautiful.

Why a robot art competition NOW?

There are several trends that are converging that makes the timing right for this competition.  First, as there is growing access to inexpensive yet powerful robots, we are at a point where robots will become commonplace.  It’s a bit similar to getting critical mass to easy online tools for creating web pages.  Once there is access, people can become more creative.  Second, computer processing speed and advancements in AI, especially in image processing and object recognition, will enable the software that controls robots to think in ways similar to humans.

Where is this competition headed?

We are excited to see how teams build upon their technical successes of this year to focus more on the creative process and artistry in the next competitions.  We are also going to open a gallery to sell selected artworks following the announcement of winners in the 2018 competition.

How do I get in contact with the teams?

If you are wanting to write a story about any of the teams, feel free to email me at abconru at yahoo.com or find their contact information online.  I am happy to connect you with the teams ASAP.

How do I get high-res images of the artwork for journalistic publications?

You can simply click on the medium sized photo of each artwork page to see the high-res image.  Please note that the copyright for the images are owned by the team but you have permission to use the images and videos in publications with credits.

Notes

Feel free to quote the website or me.